alibaba/canal

Canal is a distributed database binlog incremental subscription and consumption framework that captures real-time data changes from MySQL and MariaDB sources. It provides a modular architecture for routing these changes to various message queues like Kafka, RocketMQ, and RabbitMQ, while also supporting direct synchronization to downstream systems such as Elasticsearch, HBase, and Kudu. The system includes comprehensive administrative interfaces, remote configuration management, and monitoring capabilities to facilitate large-scale data integration and ETL workflows.

62 changes.

This period focused on modernizing the codebase with Java 8+ features, implementing GTID-based synchronization, and supporting MySQL 8.0 authentication protocols. The architecture was refactored to support remote configuration, dynamic MQ partitioning, and enhanced security through SSL and encrypted passwords.

37 changes.

This period focused on expanding Canal's ecosystem by introducing a new client adapter framework for data synchronization to targets like HBase, RDB, and message queues. Concurrently, the project integrated Prometheus metrics and authentication into the server and embedded components, while also adding Docker support and RDS binlog parsing capabilities.

35 changes.

This period focused on establishing a centralized management interface and expanding the system's connectivity. A new Canal Admin API and web UI were introduced to control server instances, while the connector core was refactored to support modular message queue integrations. Additionally, the codebase grew to include adapters for Elasticsearch, Kudu, and TableStore, alongside new client and server connectors for Kafka, RabbitMQ, RocketMQ, and Pulsar.

6 changes.

This period focused on modernizing the deployment and operational capabilities of the project. Key additions included Helm charts for Kubernetes-based orchestration, multi-architecture Dockerfile support for amd64 and arm64 platforms, and enhanced security features like SSL/TLS for MySQL connections. Additionally, the codebase expanded its observability by exposing JVM metrics via Prometheus and improved reliability through comprehensive unit testing.

Added new socket channel implementations for the MySQL driver: BioSocketChannel and NettySocketChannel, along with their respective pools. The SocketChannel interface and SocketChannelPool abstract class provide a unified API for socket operations. The BioSocketChannel supports both standard and SSL/TLS connections, while the NettySocketChannel uses Netty's NIO for non-blocking I/O. The SocketChannelPool allows selecting between BIO and Netty implementations via the 'canal.socketChannel' environment variable or system property, defaulting to BIO. This change introduces the underlying infrastructure for socket communication in the MySQL driver.

driver/src/main/java/com/alibaba/otter/canal/parse/driver/mysql/socket · high confidence

The Docker image now includes new shell and Python scripts to handle container initialization and lifecycle management. A new entrypoint (main.sh) runs initialization scripts from the /alidata/init/ directory, while exec\_rc\_local.sh and lark-wait manage startup conditions and graceful shutdowns. Additionally, fix-hosts.py corrects /etc/hosts entries, and 02init-sshd.sh configures SSH settings. These changes support the container's internal bootstrapping and process monitoring.

docker/image/alidata · high confidence

Introduced new Docker build and run scripts (build.sh, run.sh, run\_admin.sh) and Dockerfiles (Dockerfile, Dockerfile\_admin) for the Canal Server and Canal Admin components. These files enable users to build and run the respective services in Docker containers, exposing the necessary ports (11110-11112, 9100 for server; 8089 for admin) and configuring the entry points for the application.

docker · high confidence

Introduces a new ES6x adapter implementation for Canal, enabling synchronization to Elasticsearch 6.x via both Transport and REST clients. The change adds the ES6xAdapter, ESEtlService, ES6xTemplate, and ESConnection classes to handle ES 6.x specific APIs, including support for index aliases and parent-child relations in ETL processes.

client-adapter/es6x/src/main · high confidence

Introduces a new HBase adapter in the client-adapter module, enabling real-time synchronization of database changes to HBase and supporting ETL operations. The implementation includes configuration loading, a config monitor for hot-reloading YAML mappings, and services for handling INSERT, UPDATE, and DELETE operations, as well as data import and count queries against HBase tables.

client-adapter/clickhouse, client-adapter/hbase, client-adapter/phoenix · high confidence

Added new example classes for the Canal Kafka client integration, including a base test class, a standard client example, a flat message client example, and an offset client example, along with a running test, to demonstrate how to consume Canal data via Kafka.

example/src/main/java/com/alibaba/otter/canal/example/kafka · high confidence

Added a new Kudu adapter to the client-adapter module, enabling real-time synchronization and ETL (Extract, Transform, Load) operations for Kudu tables. This includes the main \KuduAdapter\ class, configuration loaders, a file-based config monitor for hot-reloading, and services for handling insert, update, and delete events. The implementation supports mapping source database tables to Kudu tables, handling batch commits, and managing Kudu client connections.

client-adapter/kudu, client-adapter/tablestore · high confidence

The Prometheus module now provides comprehensive monitoring for Canal instances by exposing metrics for store, entry, meta, sink, and parser components. The implementation includes a new \CanalInstanceExports\ class that manages the lifecycle of collectors for each component, allowing users to monitor instance-level metrics such as transaction counts, delays, and byte counts via the Prometheus HTTP server.

prometheus · high confidence

Added new request classes for Alibaba Cloud RDS API interactions: AbstractRequest provides common HTTP request handling, signature generation, and parameter formatting; DescribeBackupPolicyRequest queries RDS backup policy details; DescribeBinlogFilesRequest retrieves binlog file lists with pagination and date range support. These classes enable Canal to fetch RDS metadata and binlog information directly from the RDS API.

parse/src/main/java/com/alibaba/otter/canal/parse/inbound/mysql/rds/request · high confidence

A new REST endpoint has been added to the Canal adapter launcher, providing endpoints to trigger ETL (Extract, Transform, Load) processes and manage synchronization switches. Users can now use the \/etl/\ endpoints to manually trigger data synchronization tasks, with built-in locking to prevent concurrent execution. Additionally, the \/syncSwitch/\ endpoint allows users to enable or disable synchronization for specific destinations, while the \/destinations\ endpoint returns the current status of all configured instances.

client-adapter/launcher/src/main/java/com/alibaba/otter/canal/adapter/launcher/rest · high confidence

Users can now connect to RabbitMQ as a message queue target for Canal. This change introduces the \RabbitMQCanalConnector\ and \AliyunCredentialsProvider\ classes, enabling the client to publish and consume messages via the AMQP protocol, including support for Alibaba Cloud credentials for authentication.

client/src/main/java/com/alibaba/otter/canal/client/rabbitmq · high confidence

Users can now connect Canal to RabbitMQ for both message production and consumption. The change introduces a new RabbitMQ connector module containing configuration constants, producer and consumer implementations, and SPI registration files, enabling users to route Canal data to RabbitMQ queues and exchanges with configurable host, port, virtual host, credentials, and routing keys.

connector/rabbitmq-connector · high confidence

Users can now connect to RocketMQ to receive Canal data. The new \RocketMQCanalConnector\ supports both standard and flat message formats, handles Alibaba Cloud commercial service integration via the 'cloud' access channel, and allows configuration of namespace, batch size, and authentication credentials.

client/src/main/java/com/alibaba/otter/canal/client/rocketmq · high confidence

The connector/rocketmq-connector module now includes the Java implementation for the RocketMQ integration, introducing new configuration classes (RocketMQConstants, RocketMQProducerConfig) and SPI service files to wire the producer and consumer components. This change enables users to configure and use RocketMQ as a message queue destination for Canal data synchronization.

connector/rocketmq-connector · high confidence

Users can now configure SSL/TLS settings for MySQL connections, including enabling or disabling SSL, specifying TLS versions, and providing trust and client certificates. This introduces new configuration options for secure database connections.

driver/src/main/java/com/alibaba/otter/canal/parse/driver/mysql/ssl · high confidence

The TCP connector module now includes a new \CanalTCPConsumer\ class that handles message consumption via TCP, supporting both direct host/port connections and ZooKeeper-based cluster discovery. Configuration is centralized in \TCPConstants\ with keys for host, username, password, ZooKeeper hosts, and batch size. The SPI registration in \META-INF/canal/com.alibaba.otter.canal.connector.core.spi.CanalMsgConsumer\ wires the \tcp\ implementation, enabling users to consume Canal data over TCP.

connector/tcp-connector · high confidence

The docker/base directory now includes complete build files for both amd64 and arm64 architectures. This adds support for building the osbase and osadmin images on both platforms, including architecture-specific configurations for MySQL 8.0, JDK, and system dependencies, enabling multi-architecture image generation.

docker/base · high confidence

The Canal adapter launcher now supports loading remote configuration from a database. New classes in the \monitor/remote\ package (\DbRemoteConfigLoader\, \RemoteConfigLoader\, \RemoteAdapterMonitor\, etc.) enable the adapter to fetch \application.yml\ and adapter configs from a remote database (via \canal.manager.jdbc.\*\ properties) and apply them to the local file system, allowing centralized management of adapter configurations.

client-adapter/launcher/src/main/java/com/alibaba/otter/canal/adapter/launcher/monitor/remote · high confidence

Added new example classes (CanalRocketMQClientExample, CanalRocketMQClientFlatMessageExample, and a shared test base) that demonstrate how to connect to and consume Canal binlog events using the RocketMQ client. The examples show how to configure connection parameters (name servers, topic, group ID) and process both standard and flat message formats from RocketMQ.

example/src/main/java/com/alibaba/otter/canal/example/rocketmq · high confidence

A new PulsarMQCanalConnector implementation has been added to the client module, enabling Canal to connect to and consume messages from Apache Pulsar message queues. This introduces a new integration point for users who rely on Pulsar for their messaging infrastructure.

client/src/main/java/com/alibaba/otter/canal/client/pulsarmq · high confidence

Users can now connect to Apache Pulsar as a message queue backend. This change introduces a new \pulsarmq\ connector implementation, including configuration constants, producer and consumer classes, and SPI registrations. The connector supports configuring the Pulsar server URL, role token, topic tenant prefix, listener name, batch processing timeouts, retry policies, and message compression/chunking options.

connector/pulsarmq-connector · high confidence

The driver now supports connecting to MySQL servers using the Auth Switch protocol, SHA-2 password authentication, and SSL/TLS negotiation, enabling compatibility with modern MySQL versions (including 8.0.30+) and secure connections. Additionally, the driver implements new packet handlers for semi-synchronous replication acknowledgments, GTID-based binlog dumping, and slave registration, allowing more robust and feature-complete replication capabilities.

driver/src/main/java/com/alibaba/otter/canal/parse/driver/mysql/packets/client · high confidence

Added new components to enable Canal to automatically download MySQL binlog files from Alibaba Cloud RDS OSS storage and parse them locally. This includes a new \RdsBinlogEventParserProxy\ to manage the lifecycle, \RdsBinlogOpenApi\ to query available binlog files via the RDS Open API, \BinlogDownloadQueue\ to handle concurrent HTTP downloads with SSL/TLS support, and \RdsLocalBinlogEventParser\ to process the downloaded files. This allows users to recover from data loss or continue replication when the primary MySQL instance is unavailable or binlogs have been purged.

parse/src/main/java/com/alibaba/otter/canal/parse/inbound/mysql/rds · high confidence

The binlog parser now supports several new MySQL 8.0+ log event types, including HeartbeatV2LogEvent, TransactionContextLogEvent, TransactionPayloadLogEvent, ViewChangeEvent, and XaPrepareLogEvent. Additionally, the GtidLogEvent has been extended to parse the new 'last committed' and 'sequence number' fields, and the FormatDescriptionLogEvent has been updated with post-header lengths for these new event types.

dbsync/src/main/java/com/taobao/tddl/dbsync/binlog/event · high confidence

Users can now sync data to Elasticsearch 8.x using the new ES8x adapter. This includes a new adapter implementation (ES8xAdapter) and supporting classes (ESConnection, ES8xTemplate, ESEtlService) that handle connection, data transformation, and bulk operations for ES 8.x. The adapter supports certificate-based authentication and handles index and alias mapping lookups. Example configuration files for customer and order data are also provided.

client-adapter/es7x, client-adapter/es8x · high confidence

A new \CanalAdminController\ class has been introduced in the deployer module to provide programmatic control over Canal instances. This controller implements the \CanalAdmin\ interface, enabling users to start, stop, restart, and check the status of specific Canal instances via the admin interface. It also includes authentication logic for user/password access and methods to retrieve running instances and logs.

deployer/src/main/java/com/alibaba/otter/canal/deployer/admin · high confidence

Introduced new shell scripts (app.sh, app\_admin.sh) and a SQL schema (canal\_manager.sql) to manage the lifecycle of Canal components within the Docker image. The scripts handle starting and stopping the Canal server, the admin interface, and an optional MySQL instance for local development, while the SQL file defines the default database structure for the Canal manager.

docker/image · high confidence

The driver now supports synchronization via Global Transaction IDs (GTID) for both MySQL and MariaDB. This change introduces new packet handling classes (Capability, GTIDSet, MariaGTIDSet, MysqlGTIDSet, UUIDSet) to parse and manage GTID sets, enabling more reliable and precise binlog replication. The implementation handles edge cases such as empty GTID strings in MariaDB and multiple GTID entries separated by newlines, ensuring stable synchronization behavior.

driver/src/main/java/com/alibaba/otter/canal/parse/driver/mysql/packets · high confidence

New Helm chart packages have been introduced for the canal-admin and canal-server components. Each chart includes standard Kubernetes templates (Deployment/StatefulSet, Service, Ingress, HPA, etc.) and default values, enabling users to deploy these services on Kubernetes via Helm.

charts/canal-admin, charts/canal-server · high confidence

A new Maven wrapper configuration file has been added to the project, specifying the use of Apache Maven version 3.5.0. This ensures that all developers and CI systems use the same Maven version, improving build consistency.

.mvn · high confidence

A new utility class, MigrateMap, has been added to the common library. It provides a method to create a ConcurrentMap backed by a Guava LoadingCache, allowing users to perform standard map operations (get, put, remove, etc.) while automatically computing values on demand. This enables a smoother transition or integration point between standard Java concurrent maps and Guava's caching mechanisms.

common/src/main/java/com/google · high confidence

Introduced the CanalMetricsProvider and CanalMetricsService interfaces to support a service-provider-based metrics export mechanism, along with a default no-op implementation (NopCanalMetricsService) to ensure backward compatibility.

server/src/main/java/com/alibaba/otter/canal/spi · high confidence

The deployer now includes SQL scripts to create the \meta\_snapshot\ and \meta\_history\ tables, which store metadata about table structure changes and binlog offsets. This provides the necessary database schema for tracking historical and current metadata states in the time-series database.

deployer/src/main/resources/spring/tsdb/sql · medium confidence

The admin Docker image now includes a cron-based log cleanup script (clean\_log.sh) that automatically removes old files from /tmp and the Canal server/admin log directories when disk usage exceeds defined thresholds (80% or 90%). Additionally, a new health check script (health.sh) has been added to verify the availability of the admin service by polling the appropriate HTTP endpoint (either the admin index page or Canal metrics), returning a clear \[OK\] or \[FAILED\] status for monitoring systems.

docker/image/admin · high confidence

Added several new utility classes in the connector core module to support message handling and data conversion. This includes \CanalMessageSerializerUtil\ for serializing and deserializing Canal messages, \MessageUtil\ for converting internal message formats, \JdbcTypeUtil\ for JDBC type conversions, \DateUtil\ for date parsing, and \Callback\ and \TimeZone\ helper classes. These changes provide the foundational utilities required for the modular MQ connector implementation.

connector/core/src/main/java/com/alibaba/otter/canal/connector/core/util · medium confidence

Added DDL scripts to define the \meta\_history\ and \meta\_snapshot\ tables for Derby, H2, and MySQL databases. These schema files establish the necessary database structures for tracking metadata history and snapshots, ensuring compatibility across different database engines.

parse/src/main/resources · high confidence

A new logger adapter example has been added to the client-adapter module, allowing users to easily debug data synchronization by logging DML (Data Manipulation Language) events to the console. This implementation uses the Fastjson2 library (version 2.0.2) for JSON serialization and registers itself via the SPI mechanism, providing a simple way to inspect incoming data changes during development or troubleshooting.

client-adapter/logger · high confidence

Added three new exception classes to the parse module: PositionNotFoundException, ServerIdNotMatchException, and ServerLogPurgedException. These new error types allow the system to more specifically identify and handle issues related to missing positions, server ID mismatches, and purged server logs during binlog parsing.

parse/src/main/java/com/alibaba/otter/canal/parse/exception · high confidence

Introduced a new plain-text configuration mechanism for Canal instances, allowing remote configuration retrieval via HTTP. The change adds HttpHelper for HTTP communication, PlainCanal as a data model, and PlainCanalConfigClient to fetch and parse remote properties. This enables dynamic instance configuration updates without local file changes.

instance/manager/src/main/java/com/alibaba/otter/canal/instance/manager/plain · high confidence

The Canal Adapter launcher now includes shell and batch scripts to manage the application lifecycle. The new startup.sh and startup.bat files handle environment detection, Java path resolution, and JVM configuration (including G1 GC settings for Java 11+ and memory limits). The stop.sh script manages process termination by reading the adapter PID file and killing the Java process. The restart.sh script provides a convenience wrapper that stops and then starts the adapter. These scripts support both Linux and Windows environments, with specific handling for IPv4/IPv6 stack preferences and logging configurations.

client-adapter/launcher/src/main/bin · high confidence

The binlog parser now handles MariaDB-specific compressed row and query log events, including new classes for DeleteRowsCompressLogEvent, QueryCompressedLogEvent, UpdateRowsCompressLogEvent, WriteRowsCompressLogEvent, and StartEncryptionLogEvent. Additionally, the CharsetConversion class has been updated to include mappings for numerous new utf8mb4 and utf8 collations (including 0900\_ai\_ci, 0900\_as\_cs, and various language-specific collations), and the NameCache utility has been introduced to optimize string lookups. The DirectLogFetcher and FileLogFetcher classes have also been updated to improve performance and robustness in log fetching.

dbsync/src/main/java/com/taobao/tddl/dbsync/binlog · high confidence

Introduced new components to manage synchronization state and locking for ETL operations. The \SyncSwitch\ component enables dynamic, per-destination control of synchronization via a local or distributed (ZooKeeper-backed) switch, allowing users to enable or disable sync states. Additionally, the \EtlLock\ component provides both local and distributed (Curator-based) locking mechanisms to coordinate ETL tasks, ensuring safe concurrent execution across multiple instances.

client-adapter/launcher/src/main/java/com/alibaba/otter/canal/adapter/launcher/common · high confidence

The project introduces a comprehensive set of new configuration and documentation files to improve development workflow and security posture. A \.codecov.yml\ file is added to configure code coverage thresholds for pull requests. The \.travis.yml\ file is introduced to automate Java builds across multiple JDK versions (8, 11, 12) and integrate with Codecov. A \SECURITY.md\ file is added to establish a vulnerability reward program and reporting guidelines via the Alibaba Security Response Center. Additionally, \codeformat.xml\ and \codetemplates.xml\ are added to enforce consistent Java code formatting and Eclipse IDE templates, while \mvnw\ (Maven Wrapper) is added to ensure consistent build environments. The \.gitignore\ file is updated to exclude build artifacts, logs, and IDE-specific files.

(repo-wide) · high confidence

The Canal Adapter now exposes JVM metrics through a built-in Prometheus HTTP server. A new PrometheusService component starts an HTTP server on port 11113 by default, initializing default HotSpot collectors to make adapter metrics available for monitoring.

client-adapter/launcher/src/main/java/com/alibaba/otter/canal/adapter/launcher/prometheus · high confidence

The admin/admin-web module now provides a complete web application for managing Canal instances and nodes. This includes a Spring Boot entry point, startup scripts for Linux and Windows, and a set of RESTful API controllers for managing cluster configurations, node servers, and instance settings. The implementation introduces a new connector layer for communicating with Canal servers, along with configuration classes for Ebean and Spring context, enabling users to remotely start, stop, and monitor Canal instances through the admin web interface.

admin/admin-web · high confidence

The adapter launcher module now includes a new set of configuration classes that manage the adapter's runtime state and initialization. A new \AdapterCanalConfig\ class handles the loading of data source connections using Druid, while \AdapterConfigHolder\ provides a singleton holder for remote configuration items. Additionally, \BootstrapConfiguration\ is introduced to load remote configurations and start monitoring, and \CuratorClient\ manages the Zookeeper connection for remote config retrieval. These changes support the adapter's ability to load and monitor remote configurations dynamically.

client-adapter/launcher/src/main/java/com/alibaba/otter/canal/adapter/launcher/config · high confidence

The MySQL binlog parser now supports MySQL master heartbeat events to detect physical TCP connection failures, improving connection stability. It also adds support for parsing GTID, MariaDB GTID, and heartbeat V2 log events, allowing clients to access last committed and sequence number information. Additionally, the TableMetaCache has been refactored to support a persistent TableMetaTSDB store, with fallback to database queries and retry logic, and includes optimizations for string and charset caching.

parse/src/main/java/com/alibaba/otter/canal/parse/inbound/mysql/dbsync · medium confidence

The MySQL parser now utilizes a multi-stage processing architecture (MysqlMultiStageCoprocessor) that separates network reception, basic parsing, deep DML parsing, and storage into distinct stages, improving concurrency and performance. The update introduces support for Alibaba RDS binlog downloads, including new data models for binlog files and backup policies, and adds configuration options for filtering DML operations (insert, update, delete) and validating binlog formats. Additionally, the TableMeta TSDB integration is refined to support timestamp-based schema lookups and improved error handling for connection and binlog stream issues.

parse/src/main/java/com/alibaba/otter/canal/parse/inbound/mysql · high confidence

The MySQL driver now supports SSL/TLS connections, allowing users to configure secure connections via the new SslInfo and SslMode parameters. The MysqlConnector logs SSL status upon connection. Additionally, MysqlQueryExecutor now supports executing multiple statements in a single query via the new queryMulti method, and the driver has been refactored to use a custom SocketChannel implementation instead of the JDK's default, improving connection management and error handling.

driver/src/main/java/com/alibaba/otter/canal/parse/driver/mysql · high confidence

The client library now supports connecting to Canal via Message Queue (MQ) with a new \CanalMQConnector\ interface and \CanalMessageDeserializer\ for parsing MQ messages. Additionally, the idle timeout for both single and cluster connectors has been increased from 30 seconds to 1 hour to prevent premature disconnections, and the \CanalConnectors\ factory methods now accept a \destination\ parameter for more precise connection targeting.

client/src/main/java/com/alibaba/otter/canal/client · high confidence

The example startup scripts (startup.sh, startup.bat, init.sh) have been updated to support Java 11 and the G1 garbage collector, while also enabling IPv6 connectivity. The shell script now detects the Java version to apply specific JVM flags (e.g., G1GC for Java 11+), and both shell and batch scripts set the \java.net.preferIPv4Stack\ property to \false\ to allow IPv6. A new \init.sh\ script was added to automatically fetch MySQL master binlog file and sync position.

example/src/main/bin · high confidence

A new ApplicationConfigMonitor component was added to the adapter launcher to watch for changes in application configuration files (YAML/YML) in the configuration directory. When a matching file is modified, the system validates the YAML syntax, refreshes the Spring context, and reinitializes the CanalAdapterService, enabling dynamic reloading of adapter configurations without restarting the application.

client-adapter/launcher/src/main/java/com/alibaba/otter/canal/adapter/launcher/monitor · high confidence

The client adapter launcher now includes its foundational configuration files. The \application.yml\ provides a comprehensive template for connecting to various data sources and message queues, including TCP, Kafka, RocketMQ, RabbitMQ, and various database adapters (MySQL, Oracle, PostgreSQL, HBase, Elasticsearch, Kudu, Phoenix, Clickhouse). The \bootstrap.yml\ configures the connection to the Canal manager database. Additionally, \logback.xml\ is introduced to manage logging, routing output to both console and rolling log files for adapters and RocketMQ clients, while \spring.factories\ registers the application's bootstrap configuration for Spring Cloud.

client-adapter/launcher/src/main/resources · high confidence

The admin UI is initialized with a complete frontend structure, including environment configuration files (.env.development, .env.production, .env.staging) and build tooling (babel, eslint, postcss). The layout is established with components for the header (Navbar), sidebar (SidebarItem, Logo, Link, Item), and main content area (AppMain). A global SVG icon system is implemented via the SvgIcon component and a collection of SVG assets. API modules are added for managing Canal clusters, configurations, instances, and node servers, alongside standard user and table data interfaces.

admin/admin-ui · high confidence

The change introduces the \CanalAdmin\ interface and its \CanalAdminWithNetty\ implementation, which provide a new programmatic API for managing the Canal Server (start, stop, restart, check status) and its instances (start, stop, release, restart). This is supported by a new Netty-based server architecture that enforces client authentication via a handshake and seed-based password verification before allowing access to administrative commands.

server/src/main/java/com/alibaba/otter/canal/admin · high confidence

Added the core components for the Elasticsearch client adapter, including the ESAdapter, ESSyncService, and configuration loaders. This enables the system to synchronize data from databases to Elasticsearch indices based on YAML configuration files, supporting insert, update, and delete operations with support for complex SQL parsing and batch processing.

client-adapter/escore · high confidence

Added PlainCanalInstanceGenerator to generate CanalInstance from manager configuration via Spring XML, supporting dynamic property merging and bean factory retrieval. Refactored CanalInstanceWithManager to extend AbstractCanalInstance, simplifying lifecycle management and integrating support for RDS binlog, TableMetaTSDB, and FileMixedMetaManager.

instance/manager/src/main/java/com/alibaba/otter/canal/instance/manager · medium confidence

The client-adapter/common module now includes a set of new support classes that form the foundation for data synchronization adapters. This includes the \Dml\ data model for representing change events, \AbstractEtlService\ for handling data import logic, and utility classes like \MessageUtil\ for parsing Canal messages and \JdbcTypeUtil\ for type conversion. These additions enable the client adapter to process and transform data for downstream systems.

client-adapter/common/src/main/java/com/alibaba/otter/canal/client/adapter/support · high confidence

The Kafka connector module is refactored with new configuration and SPI registration files. Users can now configure Kerberos authentication for Kafka by setting \kafka.kerberos.enable\, \kafka.kerberos.krb5.file\, and \kafka.kerberos.jaas.file\ properties. The connector now supports both flat and non-flat message formats, with deserialization handled by custom \KafkaMessageDeserializer\ and serialization by \KafkaMessageSerializer\. The implementation includes new classes for producer (\CanalKafkaProducer\), consumer (\CanalKafkaConsumer\), and configuration (\KafkaProducerConfig\, \KafkaConstants\).

connector/kafka-connector · high confidence

Added a new time-series database (TSDB) module for managing MySQL table metadata, including the \TableMetaTSDB\ interface and implementations like \MemoryTableMeta\ and \DatabaseTableMeta\. This change enables Canal to track and roll back table schema changes over time, supporting features such as snapshot expiration, schema filtering, and improved DDL parsing via Druid. The new components allow for better handling of table structure evolution during event parsing.

parse/src/main/java/com/alibaba/otter/canal/parse/inbound/mysql/tsdb · high confidence

Users can now receive change data in a simplified 'FlatMessage' format, which includes fields for primary keys, DDL flags, and GTID, making it easier to process events without deep parsing. Additionally, a new AdminPacket protocol and associated Java classes provide a structured way to manage Canal instances, including support for client authentication, heartbeat, and log retrieval, improving administrative control and security.

protocol/src/main/java/com/alibaba/otter/canal/protocol · high confidence

A new \ChannelFutureAggregator\ class has been added to the Netty listener package. This component implements \ChannelFutureListener\ to capture and aggregate client request results, including destination, packet type, request payload, data amount, latency, error codes, and channel errors. This enables more detailed profiling and metrics collection for network operations.

server/src/main/java/com/alibaba/otter/canal/server/netty/listener · high confidence

A new \OuterAdapter\ interface and a \ProxyOuterAdapter\ implementation have been added to the client-adapter module. The \ProxyOuterAdapter\ acts as a wrapper around any \OuterAdapter\ implementation, temporarily switching the thread's context classloader before delegating calls. This ensures that external adapters execute in their own classloader context, preventing classpath conflicts and improving isolation for third-party or plugin-based adapters.

client-adapter/common/src/main/java/com/alibaba/otter/canal/client/adapter · high confidence

The Kafka client module now includes new connector implementations: KafkaCanalConnector, which handles standard message retrieval, and KafkaOffsetCanalConnector, which allows consumers to read from specific message offsets. Supporting classes include KafkaMessage and KafkaFlatMessage wrappers that expose Kafka offsets, along with a MessageDeserializer for deserializing Canal messages from Kafka payloads.

client/src/main/java/com/alibaba/otter/canal/client/kafka · high confidence

The RDB client adapter now supports hot-reloading of mapping configurations via a file monitor, allowing changes to YAML config files to take effect without restarting the adapter. It also introduces parallel batch submission for data synchronization, improving throughput, and adds support for case-insensitive target table names. Additionally, the adapter now supports mirroring databases and handles ETL operations with improved error handling and retry mechanisms.

client-adapter/rdb/src/main · high confidence

The AuthenticationInfo class now supports encrypted database passwords via Druid, controlled by the new enableDruid and pwdPublicKey fields, and also carries SSL configuration through a new SslInfo field. Users can now configure password encryption and SSL settings directly on the authentication object, enabling secure credential handling and encrypted database connections.

parse/src/main/java/com/alibaba/otter/canal/parse/support · high confidence

The server module refactored its MQ handling by introducing new components: a \CanalMQStarter\ class that manages the lifecycle of MQ workers and handles destination-level start/stop operations, alongside \CanalServerStarter\ and \CanalService\ interfaces that define the server's operational contract. This change isolates the MQ worker management and message retrieval logic from the main server entry point, allowing for cleaner separation of concerns in the Canal server architecture.

server/src/main/java/com/alibaba/otter/canal/server · high confidence

The MySQL driver now supports GTID-based synchronization and MySQL 8.0's caching\_sha2\_password authentication. A new \GtidUtil\ class parses GTID sets for both MySQL and MariaDB, while \MySQLPasswordEncrypter\ implements the \caching\_sha2\_password\ scramble and RSA encryption logic. Additionally, \CharsetUtil\ provides a comprehensive mapping of MySQL character sets, and \PacketManager\ has been refactored to use a new \SocketChannel\ interface, improving how packets are read and written.

driver/src/main/java/com/alibaba/otter/canal/parse/driver/mysql/utils · high confidence

The main entry point for the Canal Adapter launcher has been restructured into a new \CanalAdapterApplication\ class. This change configures the Spring Boot application to exclude default data source auto-configuration and sets the banner mode to OFF. Additionally, it explicitly enables SLF4J logging for the RocketMQ client by setting the \rocketmq.client.logUseSlf4j\ system property.

client-adapter/launcher/src/main/java/com/alibaba/otter/canal/adapter/launcher · medium confidence

The Canal server startup logic has been refactored into a new \CanalStarter\ class, which now handles the initialization of the embedded server, Netty server, and MQ producers. The \CanalController\ has been updated to support remote configuration polling from a manager, allowing the server to dynamically reload its configuration without a full restart. Additionally, the \CanalLauncher\ now sets a global uncaught exception handler and configures logging for RocketMQ clients.

deployer/src/main/java/com/alibaba/otter/canal/deployer · high confidence

The shell and batch scripts in deployer/src/main/bin have been updated to improve compatibility and performance. A new restart.sh script was added to simplify service restarts. The startup.sh script now detects the Java version to apply appropriate JVM arguments: it enables G1 garbage collection for Java 11+, adjusts the thread stack size to 1m to fix issues on ARM64, and adds GC logging. The startup.bat script was corrected to remove a duplicate java command invocation and updated to support a 'local' configuration profile. Additionally, the stop.sh script was fixed to correctly identify Linux processes using the 'ps' command, resolving syntax errors on some Unix-like systems. The startup.sh script also now supports SASL authentication for ZooKeeper if a jaas.conf file is present.

deployer/src/main/bin · medium confidence

The embedded server implementation is now a singleton, ensuring only one instance exists. It adds configurable metrics server port and initializes the metrics service on startup. Additionally, it introduces user/password authentication via the new \auth\ method, which validates credentials using a scramble algorithm. The class also implements the \CanalService\ interface and registers/unregisters instances with the metrics service during start/stop operations.

server/src/main/java/com/alibaba/otter/canal/server/embedded · high confidence

The server now enforces a seed-based authentication check before processing client requests, rejecting connections if the seed is missing or credentials are invalid. Additionally, the default idle timeout for disconnected subscribers has been increased from 5 minutes to 1 hour, and the idle timeout configuration now correctly interprets the client's timeout parameter as milliseconds rather than seconds.

server/src/main/java/com/alibaba/otter/canal/server/netty/handler · high confidence

The \PropertyPlaceholderConfigurer\ now enforces a specific precedence for configuration values: system properties (when in OVERRIDE mode) are resolved first, followed by thread-local properties, then file-based properties, and finally defaults. This ensures that runtime environment variables or thread-local contexts can override static file configurations. Additionally, the \SocketAddressEditor\ now uses a dedicated \AddressUtils.splitIPAndPort\ method for parsing socket addresses, replacing the previous generic string split logic.

instance/spring/src/main/java/com/alibaba/otter/canal/instance/spring/support · medium confidence

The Event model now exposes granular metadata fields (executeTime, entryType, journalName, position, serverId, gtid, eventType, rowsCount) and a rawEntry ByteString, allowing users to access detailed event information without parsing the full entry object. Additionally, the Events class uses a diamond operator for generic type inference.

store/src/main/java/com/alibaba/otter/canal/store/model · high confidence

The default configuration file (canal.properties) has been significantly expanded to support a wider range of message queue backends, including Kafka, RocketMQ, RabbitMQ, and Pulsar, with specific properties for each (e.g., kafka.bootstrap.servers, rocketmq.namesrv.addr). Additionally, a new local configuration file (canal\_local.properties) is introduced to manage admin registration settings. Logging is also enhanced by adding a dedicated appender for RocketMQ client logs and configuring specific log levels for Kafka, RocketMQ, and RabbitMQ connectors.

deployer/src/main/resources · high confidence

The example instance.properties file has been significantly expanded to document new capabilities. Users can now configure GTID-based synchronization (canal.instance.gtidon), connect to Alibaba Cloud RDS via OSS binlog (canal.instance.rds.\), and use the internal TSDB for table metadata (canal.instance.tsdb.\). Additionally, the file now includes commented-out examples for SSL/TLS settings, Druid password encryption, table field filtering, and various Message Queue (MQ) routing and partitioning configurations.

deployer/src/main/resources/example · high confidence

The \EntryEventSink\ now tracks sink blocking time via a new \eventsSinkBlockingTime\ metric, and refactors the transaction filtering logic to use a timestamp-based strategy for empty transactions, ensuring begin/commit pairs are handled correctly. Additionally, the \TimelineTransactionBarrier\ fixes a potential deadlock by correcting the order of transaction state checks, and switches to \ThreadLocal.withInitial\ for cleaner initialization.

sink/src/main · high confidence

The ZooKeeper client implementation was refactored to improve connection management and cleanup. A new \clearClients()\ method was added to \ZkClientx\ to explicitly clear cached client instances, which addresses issues where shutdown or restart operations would fail due to closed ZK connections. Additionally, \ZooKeeperx\ was changed to extend \ZkConnection\ rather than implementing \IZkConnection\, and internal state management was updated to use reflection to access private fields from the parent class, simplifying the connection lifecycle.

common/src/main/java/com/alibaba/otter/canal/common/zookeeper · high confidence

The Canal client library now supports graceful shutdown via a new \stopRunning\ method on the \CanalConnector\ interface, allowing blocking operations like \subscribe\ to be interrupted cleanly. Additionally, the \ClusterCanalConnector\ and \SimpleCanalConnector\ have been updated to handle connection timeouts more robustly: an \idleTimeout\ (defaulting to 1 hour) is now enforced, and the client will no longer retry indefinitely when no alive Canal server is found, instead throwing a specific \ServerNotFoundException\. The client also implements a more secure password authentication mechanism using a server-provided seed for scrambling, and the codebase has been modernized with Java 8+ features like lambda expressions and the diamond operator.

client/src/main/java/com/alibaba/otter/canal/client/impl · high confidence

The ClientRunningMonitor now handles errors more gracefully to prevent deadlocks and excessive retries. Specifically, the initRunning method is now synchronized to ensure sequential execution and includes logic to reset the mutex if the active data belongs to the current instance, avoiding potential deadlocks. Additionally, a try-catch block was added to handle unexpected exceptions, ensuring that the running node is released and a CanalClientException is thrown, which prevents infinite retry loops when the server is unavailable. The shutdown process also explicitly shuts down the delay executor to prevent resource leaks.

client/src/main/java/com/alibaba/otter/canal/client/impl/running · high confidence

The regex filter now correctly handles commas inside regex grouping constructs (such as {1,2} or (a,b)), preventing the split logic from breaking quantifiers and other grouped expressions. This fix ensures that filter patterns containing commas within curly, square, or round brackets are preserved intact, resolving issues where valid regex syntax was incorrectly parsed. Additionally, the implementation replaces StringBuffer with StringBuilder for better performance and updates generic type declarations to use diamond operators.

connector/core/src/main/java/com/alibaba/otter/canal/connector/core/filter, filter/src/main/java/com/alibaba/otter/canal/filter/aviater · high confidence

The Zookeeper-based server running state monitor now ensures that if the start process fails, the system resets its state to avoid interference with subsequent start attempts. The monitor's lifecycle methods have been refactored to properly initialize and shut down the scheduled executor service, preventing resource leaks. Additionally, the \release\ method now returns a boolean to indicate success, and the \stop\ method ensures the executor is properly terminated. Logging has been updated to use the server address instead of the deprecated \cid\ field.

common/src/main/java/com/alibaba/otter/canal/common/zookeeper/running · medium confidence

The instance configuration monitor has been significantly refactored to support remote configuration polling via a new \PlainCanalConfigClient\. The \ManagerInstanceConfigMonitor\ now uses a scheduled executor to periodically scan for instance changes, supporting start, stop, and reload actions based on remote configuration updates. The \InstanceAction\ interface has been updated to include a \release\ method, and the \InstanceConfigMonitor\ interface has corrected the spelling of \register\ and \unregister\ methods. Additionally, the \SpringInstanceConfigMonitor\ has been updated to use lambda expressions and modern Java features, improving code readability and maintainability.

deployer/src/main/java/com/alibaba/otter/canal/deployer/monitor · medium confidence

Added new SPI interfaces \CanalMQProducer\ and \CanalMsgConsumer\ to standardize message queue integration, along with a custom \ExtensionLoader\ and classloader proxies (\ProxyCanalMQProducer\, \ProxyCanalMsgConsumer\) to manage extension loading and classloader isolation. This refactors the connector core to support modular MQ implementations via the \@SPI\ annotation.

connector/core/src/main/java/com/alibaba/otter/canal/connector/core/spi · high confidence

Added new utility classes in the common package: CommonUtils for configuration path resolution, directory deletion, and Druid password decryption; ExecutorTemplate for simplified concurrent task execution; FileUtils for safe file reading and path validation; MQUtil for validating message queue topics and tags; and PropertiesUtils for resolving configuration values with environment variable priority. Updated JsonUtils to migrate from Fastjson 1.x to Fastjson2, applying an auto-type filter to restrict deserialization to trusted packages, and added an InetAddressWriter for JSON serialization. Also updated AddressUtils to split IP and port, refined BooleanMutex synchronization, and adjusted thread factory exception handling.

common/src/main/java/com/alibaba/otter/canal/common/utils · high confidence

Adds new configuration classes (CanalConstants, MQProperties) and base producer classes (AbstractMQProducer, MQDestination, MQMessageUtils) to the connector core. This refactors how MQ settings (such as flat message format, database hashing, thread pool sizes, and batch sizes) are loaded and applied, providing a structured way to configure message queue interactions.

connector/core/src/main/java/com/alibaba/otter/canal/connector/core/producer · high confidence

The memory-based event store now supports acknowledging events by specifying a sequence ID, allowing for more granular control over which events are considered processed. This change introduces a new method signature that accepts both a position and a sequence ID, enabling the system to track and manage event states more precisely.

store/src/main/java/com/alibaba/otter/canal/store · medium confidence

Replaced deprecated Guava MapMaker and Function-based lambda patterns with Java standard library equivalents (e.g., ConcurrentHashMap, lambdas) and Guava's MigrateMap. This includes fixing a ConcurrentModificationException in MemoryMetaManager.listAllSubscribeInfo by returning a defensive copy of the list, and updating charset handling to use StandardCharsets.UTF\_8. Additionally, the codebase migrates from anonymous inner classes to lambda expressions for executor tasks, improving code readability and reducing boilerplate.

meta/src/main · high confidence

The binlog parsing pipeline has been refactored into a multi-stage architecture that separates network reception, basic event parsing, and deep DML parsing. This enables parallel processing of DML events to improve performance. Additionally, the system now supports filtering specific table columns (field-level filtering) and handles GTID-based synchronization, heartbeat events, and improved error handling for missing positions or table metadata.

parse/src/main/java/com/alibaba/otter/canal/parse/inbound · medium confidence

The Netty server implementation now includes a singleton-based profiling mechanism to track client request results, enabling better observability. Additionally, the server manages child socket channels via a ChannelGroup to ensure explicit closure during shutdown, preventing the server from hanging when abnormal network connections cause the IO worker thread to loop indefinitely. The code also introduces a write method in NettyUtils that supports a ChannelFutureListener for asynchronous write operations and adds a VERSION field to the Packet builder, alongside a typo fix in the embedded server class name.

server/src/main/java/com/alibaba/otter/canal/server/netty · medium confidence

The \CanalParameter\ model now supports several new capabilities for instance management. You can now configure SSL/TLS connections for the master, standby, and general connections using new fields for modes, TLS versions, and certificate stores. Metadata storage options have been extended with a \LOCAL\_FILE\ mode and a flush period parameter. Additionally, the model includes settings for a table metadata time-series database (TSDB), RDS-specific credentials, GTID support, parallel processing, custom alarm handlers, and multi-stream consumption.

instance/manager/src/main/java/com/alibaba/otter/canal/instance/manager/model · high confidence

The \CanalInstanceWithSpring\ class now extends \AbstractCanalInstance\ instead of \CanalInstanceSupport\, removing a large amount of duplicated boilerplate code for starting, stopping, and managing internal components. Additionally, a setter for \CanalMQConfig\ has been added to the class. In \SpringCanalInstanceGenerator\, the Spring context loading logic has been updated to use \ClassPathXmlApplicationContext\ and includes a synchronized block to safely set the \canal.instance.destination\ system property during instance generation.

instance/spring/src/main/java/com/alibaba/otter/canal/instance/spring · medium confidence

The log position manager classes in the \canal.parse.index\ package have been refactored to extend a new \AbstractLogPositionManager\ base class and replace setter-based dependency injection with constructor injection. This change removes the reliance on Spring's setter injection for fields like \zkClientx\, \metaManager\, and \memoryLogPositionManager\, making the components easier to instantiate and test. Additionally, the \FailbackLogPositionManager\ now uses a \Logger\ for error reporting instead of relying on the \Assert\ utility, and \FileMixedLogPositionManager\ switches from \MapMaker\ to \MigrateMap\ for caching.

parse/src/main/java/com/alibaba/otter/canal/parse/index · high confidence

The Canal adapter launcher module has been restructured by introducing three new core classes: \AdapterProcessor\, \CanalAdapterLoader\, and \CanalAdapterService\. \CanalAdapterService\ acts as the Spring-managed entry point, initializing the adapter lifecycle. \CanalAdapterLoader\ is responsible for discovering and initializing \OuterAdapter\ instances based on configuration, while \AdapterProcessor\ manages the actual message processing, including batch synchronization and retry logic. This refactoring separates concerns, allowing for better management of adapter groups and concurrent processing of messages.

client-adapter/launcher/src/main/java/com/alibaba/otter/canal/adapter/launcher/loader · medium confidence

The MySQL DDL parsing logic has been refactored to improve how table rename operations are handled. A new \DdlResult\ class has been introduced to represent parsed DDL events, replacing the previous inner-class approach. This change enables the parser to correctly process multi-table RENAME statements by linking multiple rename operations together via a \renameTableResult\ field. Additionally, the \SimpleDdlParser\ has been moved to the \ddl\ package and updated to use the new \DdlResult\ model, ensuring that complex rename operations are captured accurately in the change stream.

parse/src/main/java/com/alibaba/otter/canal/parse/inbound/mysql/ddl · high confidence

The TableIdNotFoundException class has been moved from the old com.alibaba.otter.canal.parse.exception package to the new com.taobao.tddl.dbsync.binlog.exception package, reflecting a structural reorganization of the codebase.

dbsync/src/main/java/com/taobao/tddl/dbsync/binlog/exception · low confidence

The \CanalLifeCycle\ and \CanalAlarmHandler\ interfaces have been updated to remove the \public\ modifier from their method signatures. In Java, interface methods are implicitly public, so this change is a code cleanup that removes redundant modifiers. Additionally, the \sendAlarm\ method in \CanalAlarmHandler\ had its Javadoc comment simplified by removing the \@param title\ parameter which was not present in the method signature.

common/src/main/java/com/alibaba/otter/canal/common, common/src/main/java/com/alibaba/otter/canal/common/alarm · high confidence

The BinLogFileQueue now enforces that binlog filenames must end with a digit, filtering out non-conforming files that were previously accepted. Additionally, error handling has been improved by capturing and propagating exceptions from the background file-reloading task, ensuring that issues with binlog file discovery are surfaced rather than silently ignored.

parse/src/main/java/com/alibaba/otter/canal/parse/inbound/mysql/local · high confidence

Added new packet classes, AuthSwitchRequestMoreData and AuthSwitchRequestPacket, to support the MySQL authentication switch protocol. The HandshakeInitializationPacket was updated to correctly parse server capabilities, the authentication plugin name, and the scramble buffer based on those capabilities. Additionally, the FieldPacket now skips filter bytes during parsing, and generic Java type improvements (diamond operator) were applied to ResultSetPacket and RowDataPacket.

driver/src/main/java/com/alibaba/otter/canal/parse/driver/mysql/packets/server · medium confidence

The \AbstractCanalInstance\ class has been introduced to centralize instance lifecycle management, replacing the previous \CanalInstanceSupport\ base class. This change includes adding \CanalMQConfig\ to support dynamic queue partitioning via the \enableDynamicQueuePartition\ configuration, allowing systems like Alibaba Cloud RocketMQ to handle dynamic scaling without manual partition configuration. Additionally, the \subscribeChange\ method in \AbstractCanalInstance\ now explicitly checks for \AbstractEventParser\ types to ensure compatibility, particularly for \MysqlEventParser\ Gtid handling.

instance/core · medium confidence

The internal pattern caching mechanism in PatternUtils has been migrated from the deprecated MapMaker to the newer CacheBuilder API. This change updates the underlying implementation for storing compiled patterns, ensuring compatibility with current Guava versions and removing reliance on legacy caching constructs.

filter/src/main/java/com/alibaba/otter/canal/filter · high confidence

The EntryPosition class now tracks the server ID and GTID (Global Transaction ID) alongside the journal name and position, allowing for more precise identification of the replication state. Additionally, the PositionRange class now supports an 'endSeq' (sequence number) field to more accurately record positions within the ring buffer, improving data consistency during master-slave switches.

protocol/src/main/java/com/alibaba/otter/canal/protocol/position · medium confidence

The \CanalEventUtils\ class was updated to improve how event positions are created and compared. The code now extracts \serverId\ and \gtid\ from the \Event\ object and stores them in the \EntryPosition\. Additionally, the \checkPosition\ method was refactored to use the new \Event\ accessor methods (\getJournalName\, \getPosition\, \getExecuteTime\) instead of the previous \CanalEntry\ header access, and includes a short-circuit check for exact matches. This change ensures that position tracking includes more granular identifiers, which can help in accurately identifying primary/standby switches and preventing data loss during synchronization.

store/src/main/java/com/alibaba/otter/canal/store/helper · medium confidence

The table metadata storage implementation was fixed to support both MySQL and H2 databases. The new \MetaBaseDAO\ class now detects the database type and loads the appropriate SQL schema from \ddl/mysql/\ or \ddl/h2/\ directories, ensuring that the \meta\_history\ and \meta\_snapshot\ tables are correctly initialized for both database types.

parse/src/main/java/com/alibaba/otter/canal/parse/inbound/mysql/tsdb/dao · high confidence

Resolves a NullPointerException and incorrect sequence handling in the memory event store. The code now correctly handles GTID mode by ensuring the acknowledgment point is set at the transaction end rather than the head, preventing data loss. Additionally, the \ack\ and \cleanUntil\ methods are updated to support a sequence ID parameter, and variable names are corrected to fix compiler errors.

store/src/main/java/com/alibaba/otter/canal/store/memory · high confidence

Added DDL test cases for MySQL ALTER TABLE operations; Added Helm chart test scripts and sample values files; Added and reorganized server-side unit tests; Added and updated tests for MySQL parsing components; Added and updated tests for common utilities and server monitoring; Added and updated tests for the Aviater filter components; Added and updated unit tests for binlog parsing and JSON conversion; Added integration tests for the PlainCanalConfigClient; Added performance test for DirectLogFetcher; Added performance tests for MySQL binlog parsing and dumping; Added tests for BooleanMutex concurrency behavior; Added tests for MySQL TSDB components; Added tests for RDB adapter configuration loading; Added tests for the ES6x client adapter; Added unit tests for LogHeader and TableMapLogEvent; Added unit tests for MySQL DDL parsing; Added unit tests for MySQL driver components; Added unit tests for MySQL packet handling; Added unit tests for MySQL parsing utilities; Added unit tests for RDB sync adapter; Added unit tests for client-adapter support utilities; Added unit tests for protocol and security utilities; Disabled failing Spring instance integration tests; Refactored and added Canal client example tests; Refactored memory event store tests; Removed unused test stub for CanalLogPositionManager; Updated Spring test configuration files to use bean references and remove default database names; Updated test configuration and refactored test code; Updated test suite for Canal event parsing and storage; Updated test suite for log position managers; Updated test suite to use JUnit 4 and suppress skipped tests; Updated test suite with modernized concurrency patterns and stub implementation; Updated unit tests for EventTransactionBuffer and TableMetaCache.

The Canal admin module is now split into separate admin-ui and admin-web components, with the admin-ui introducing a Vue.js frontend using Element UI, Axios, and Vue Router. The client-adapter module is restructured into distinct sub-modules for each supported data sink, including ClickHouse, Elasticsearch (versions 6x, 7x, and 8x), HBase, Kudu, Logger, and Phoenix. These changes introduce new dependencies such as Spring Boot 2.5.15, Caffeine, Ebean, and various database connectors, while also updating the frontend build process to compile the Vue.js application into the admin-web static resources.

(dependencies) · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• CAI 35 → 33 (-2.1)
• Rubric changed (rubric-2026.08.19 → rubric-2026.09.15) — scores are not directly comparable.

• Code Health 39 → 42 (+2.5)
• Architecture 100 → 85 (-14.7)
• Maturity 55 → 55 (-0.3)
• Readiness 21 → 23 (+1.9)
• Security 59 → 57 (-2.1)
• Domain Modelling 75 → 79 (+3.8)
• Accessibility 39 → 27 (-12.4)

• AbstractCanalClientTest.process (cognitive 16) (example/src/main/java/com/alibaba/otter/canal/example/AbstractCanalClientTest.java)
• BaseCanalClientTest.printEntry (cognitive 35) (example/src/main/java/com/alibaba/otter/canal/example/BaseCanalClientTest.java)
• CanalKafkaClientExample.process (cognitive 23) (example/src/main/java/com/alibaba/otter/canal/example/kafka/CanalKafkaClientExample.java)
• CanalKafkaClientFlatMessageExample.process (cognitive 23) (example/src/main/java/com/alibaba/otter/canal/example/kafka/CanalKafkaClientFlatMessageExample.java)
• CanalKafkaOffsetClientExample.process (cognitive 33) (example/src/main/java/com/alibaba/otter/canal/example/kafka/CanalKafkaOffsetClientExample.java)
• CanalRocketMQClientExample.process (cognitive 17) (example/src/main/java/com/alibaba/otter/canal/example/rocketmq/CanalRocketMQClientExample.java)
• CanalRocketMQClientFlatMessageExample.process (cognitive 17) (example/src/main/java/com/alibaba/otter/canal/example/rocketmq/CanalRocketMQClientFlatMessageExample.java)
• Coverage not included — suite not readable by the collector
• Dependency hygiene not measured — dependency manifest found but not parsed for hygiene
• Duplicated block (10 lines × 2) (admin/admin-web/src/main/java/com/alibaba/otter/canal/admin/connector/SimpleAdminConnector.java)
• Duplicated block (10 lines × 2) (admin/admin-web/src/main/java/com/alibaba/otter/canal/admin/connector/SimpleAdminConnector.java)
• Duplicated block (10 lines × 2) (client-adapter/clickhouse/src/main/java/com/alibaba/otter/canal/client/adapter/clickhouse/support/SingleDml.java)
• Duplicated block (10 lines × 2) (client-adapter/common/src/main/java/com/alibaba/otter/canal/client/adapter/support/AbstractEtlService.java)
• Duplicated block (10 lines × 2) (client-adapter/common/src/main/java/com/alibaba/otter/canal/client/adapter/support/ExtensionLoader.java)
• Duplicated block (10 lines × 2) (client-adapter/common/src/main/java/com/alibaba/otter/canal/client/adapter/support/JdbcTypeUtil.java)
• Duplicated block (10 lines × 2) (client-adapter/es7x/src/main/java/com/alibaba/otter/canal/client/adapter/es7x/etl/ESEtlService.java)
• Duplicated block (10 lines × 2) (client-adapter/es7x/src/main/java/com/alibaba/otter/canal/client/adapter/es7x/support/ES7xTemplate.java)
• Duplicated block (10 lines × 2) (client-adapter/escore/src/main/java/com/alibaba/otter/canal/client/adapter/es/core/service/ESSyncService.java)
• Duplicated block (10 lines × 2) (client/src/main/java/com/alibaba/otter/canal/client/CanalMessageDeserializer.java)
• Duplicated block (10 lines × 2) (client/src/main/java/com/alibaba/otter/canal/client/impl/ClusterCanalConnector.java)
• …and 282 more

• (anonymous) (cognitive 17) (admin/admin-ui/src/permission.js)
• ClassTooLong: CanalParameter (instance/manager/src/main/java/com/alibaba/otter/canal/instance/manager/model/CanalParameter.java)
• ClassTooLong: ESSyncService (client-adapter/escore/src/main/java/com/alibaba/otter/canal/client/adapter/es/core/service/ESSyncService.java)
• ClassTooLong: LogBuffer (dbsync/src/main/java/com/taobao/tddl/dbsync/binlog/LogBuffer.java)
• ClassTooLong: LogEventConvert (parse/src/main/java/com/alibaba/otter/canal/parse/inbound/mysql/dbsync/LogEventConvert.java)
• ClassTooLong: MQMessageUtils (connector/core/src/main/java/com/alibaba/otter/canal/connector/core/producer/MQMessageUtils.java)
• ClassTooLong: MemoryEventStoreWithBuffer (store/src/main/java/com/alibaba/otter/canal/store/memory/MemoryEventStoreWithBuffer.java)
• ClassTooLong: MysqlConnection (parse/src/main/java/com/alibaba/otter/canal/parse/inbound/mysql/MysqlConnection.java)
• ClassTooLong: MysqlEventParser (parse/src/main/java/com/alibaba/otter/canal/parse/inbound/mysql/MysqlEventParser.java)
• ClassTooLong: PhTypeUtil (client-adapter/hbase/src/main/java/com/alibaba/otter/canal/client/adapter/hbase/support/PhTypeUtil.java)
• Dependency hygiene PARTLY measured — Maven/Gradle declarations read, no dependency graph resolved
• Documentation: no installation or build instructions (README.md)
• Documentation: no usage examples (README.md)
• Duplicated block (10 lines × 2) (client-adapter/clickhouse/src/main/java/com/alibaba/otter/canal/client/adapter/clickhouse/service/ClickHouseBatchSyncService.java)
• Duplicated block (10 lines × 2) (client-adapter/clickhouse/src/main/java/com/alibaba/otter/canal/client/adapter/clickhouse/service/ClickHouseMirrorDbBatchSyncService.java)
• Duplicated block (10 lines × 2) (client-adapter/common/src/main/java/com/alibaba/otter/canal/client/adapter/support/Dml.java)
• Duplicated block (10 lines × 2) (client-adapter/common/src/main/java/com/alibaba/otter/canal/client/adapter/support/URLClassExtensionLoader.java)
• Duplicated block (10 lines × 2) (client-adapter/es6x/src/main/java/com/alibaba/otter/canal/client/adapter/es6x/ES6xAdapter.java)
• Duplicated block (10 lines × 2) (client-adapter/escore/src/main/java/com/alibaba/otter/canal/client/adapter/es/core/service/ESSyncService.java)
• Duplicated block (10 lines × 2) (client-adapter/hbase/src/main/java/com/alibaba/otter/canal/client/adapter/hbase/HbaseAdapter.java)
• …and 559 more

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

alibaba/canal was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.