apache/rocketmq
58.4
Adequate · 13 November 2026
222.8k
lines of production code
Java
primary language
7
measurements over time
What this system is
Features
Add BitsArray, BloomFilter, and BloomFilterData utility classes
The filter module now includes new utility classes to support efficient bit manipulation and probabilistic filtering. BitsArray provides a wrapper for byte arrays to allow easy single-bit operations (set, get, and, or, xor, not). BloomFilter implements a simple Bloom filter using Guava's MurmurHash3 to calculate bit positions for strings, enabling fast membership checks. BloomFilterData holds the bit positions and total bit count generated by the filter, facilitating data exchange and validation between filter instances.
filter/src/main/java/org/apache/rocketmq/filter/util · high confidence
Add HTTP/2 protocol proxy handler
The proxy now includes an HTTP/2 protocol handler that detects HTTP/2 connections by checking for the 'PRI ' magic string and proxies them to a local gRPC server. This handler supports TLS configuration based on the proxy's TLS mode settings and forwards HAProxy messages to preserve client connection information.
proxy/src/main/java/org/apache/rocketmq/proxy/remoting/protocol/http2proxy · high confidence
Add OpenMessaging (OMS) example code
Added three new example classes—SimpleProducer, SimplePullConsumer, and SimplePushConsumer—in the openmessaging package to demonstrate how to use the OpenMessaging API with RocketMQ. These examples show how to create a producer for sending messages (sync, async, and oneway) and how to create pull and push consumers for receiving messages, using the OMS access point URL format.
example/src/main/java/org/apache/rocketmq/example/openmessaging · high confidence
Add OpenMessaging Promise implementation for RocketMQ
The \openmessaging\ module now includes a new \DefaultPromise\ class and \FutureState\ enum in the \io.openmessaging.rocketmq.promise\ package. This implementation provides the core asynchronous promise/future behavior required to support the OpenMessaging specification within RocketMQ, handling state transitions (doing, done, cancelled), result retrieval with timeouts, and listener notifications.
openmessaging/src/main/java/io/openmessaging/rocketmq/promise · high confidence
Add OpenMessaging domain model implementations for RocketMQ
This change introduces the core domain objects required to implement the OpenMessaging specification within the RocketMQ connector. It adds \BytesMessageImpl\ to handle binary message bodies and headers, \SendResultImpl\ to expose message IDs and properties from send operations, and \ConsumeRequest\ to encapsulate the context (message, queue, process queue) for consumption tasks. Additionally, it defines \NonStandardKeys\ and \RocketMQConstants\ to manage RocketMQ-specific metadata (such as consumer/producer groups and delivery timing) that maps to the OpenMessaging abstraction.
openmessaging/src/main/java/io/openmessaging/rocketmq/domain · high confidence
Add OpenMessaging utility classes for bean population and message conversion
Introduces BeanUtils and OMSUtil helper classes in the OpenMessaging RocketMQ integration. BeanUtils provides reflection-based population of JavaBeans from Properties and KeyValue objects, supporting primitive type conversion. OMSUtil handles bidirectional conversion between OpenMessaging BytesMessage and RocketMQ Message objects, mapping system and user headers, and converting send results.
openmessaging/src/main/java/io/openmessaging/rocketmq/utils · high confidence
Add batch message sending examples
New example classes, SimpleBatchProducer and SplitBatchProducer, are added to demonstrate how to send messages in batches. SimpleBatchProducer shows sending a small batch of messages, while SplitBatchProducer illustrates handling large batches by splitting them into smaller chunks to stay within size limits.
example/src/main/java/org/apache/rocketmq/example/batch · high confidence
Add default configuration files for broker, proxy, and tools
The distribution package now includes default configuration files for the broker (broker.conf), the proxy (rmq-proxy.json), and the tools module (tools.yml). These files provide out-of-the-box settings for cluster names, broker roles, and default access credentials, simplifying initial setup and deployment.
distribution/conf · high confidence
Add gRPC v2 EndTransaction activity
The proxy now exposes a new gRPC v2 endpoint for ending transactions. This change introduces the \EndTransactionActivity\ class, which validates the transaction ID and topic, maps the requested resolution (commit or rollback) to the internal transaction status, and delegates the operation to the messaging processor. Users can now complete distributed transactions via the v2 gRPC interface.
proxy/src/main/java/org/apache/rocketmq/proxy/grpc/v2/transaction · high confidence
Add transactional message example
Added example code for using RocketMQ transactional messages, including a producer that sends messages with a custom transaction listener implementation to handle local transaction execution and status checking.
example/src/main/java/org/apache/rocketmq/example/transaction · high confidence
Added ACL 1.0 to 2.0 migration tooling
The auth module now includes an \AuthMigrator\ component that automatically converts legacy ACL 1.0 configurations (defined in YAML files under the \conf/acl\ directory) into the new ACL 2.0 format. This migration reads existing \PlainAccessConfig\ data—such as access keys, secret keys, admin status, and topic/group permissions—and creates corresponding users and authorization policies in the new system. The migration is controlled by the \authConfig.isMigrateAuthFromV1Enabled\ flag and runs asynchronously to ensure existing access control settings are preserved during the transition.
auth/src/main/java/org/apache/rocketmq/auth/migration · high confidence
Added MessageRequestModeManager for pop consuming mode configuration
The broker now includes a new MessageRequestModeManager component to manage and persist the message request mode settings for topics and consumer groups. This manager stores configuration data in a concurrent map structure and handles serialization/deserialization to support the Pop Consuming feature, allowing the broker to track and apply specific consumption modes.
broker/src/main/java/org/apache/rocketmq/broker/loadbalance · high confidence
Added OpenTracing and standard trace message examples
The tracemessage example package now includes new demonstration programs for integrating with OpenTracing (Jaeger). OpenTracingProducer, OpenTracingPushConsumer, and OpenTracingTransactionProducer show how to register OpenTracing hooks to trace message sending, consumption, and transactional operations. Additionally, TraceProducer and TracePushConsumer provide standard examples for basic message tracing functionality.
example/src/main/java/org/apache/rocketmq/example/tracemessage · high confidence
Added PR merge utility script
A new Python script, dev/merge\_rocketmq\_pr.py, has been added to the repository. This tool automates the process of merging GitHub pull requests into the Apache RocketMQ project by handling git operations such as fetching branches, resolving conflicts, creating squashed commits with proper attribution, and pushing the result to the upstream repository.
dev · high confidence
Added SQL and Tag filter example code
New example files have been added to demonstrate message filtering capabilities: SqlFilterProducer and SqlFilterConsumer show how to use SQL expressions to filter messages based on tags and properties, while TagFilterProducer and TagFilterConsumer illustrate filtering by specific tags using logical operators.
example/src/main/java/org/apache/rocketmq/example/filter · high confidence
Added Windows and Linux deployment scripts for controller quick-start and independent modes
New shell (.sh) and Windows batch (.cmd) scripts have been added to the distribution/bin/controller directory to simplify local testing and deployment. Users can now quickly start a full stack (NameServer and Brokers) using the \fast-try\ scripts, or deploy a standalone 3-node controller cluster independently using \fast-try-independent-deployment\. Additionally, scripts for running NameServers in a controller plugin configuration (\fast-try-namesrv-plugin\) are provided for both platforms, enabling easier validation of controller modes without manual command-line invocation.
distribution/bin/controller · high confidence
Added authorization model classes and proxy metadata providers for ACL 2.0
This change introduces the core data models and proxy integration points for the new RocketMQ ACL 2.0 authorization system. In the auth module, new classes define the authorization context (AuthorizationContext, DefaultAuthorizationContext), decision and policy types (Decision, PolicyType), and the hierarchical policy structure (Acl, Policy, PolicyEntry, Resource, Environment). Additionally, a RequestContext model and an AuthorizationException are added. In the proxy module, new providers (ProxyAuthenticationMetadataProvider, ProxyAuthorizationMetadataProvider) are implemented to bridge the proxy layer with the metadata service for user and ACL operations.
(repo-wide) · high confidence
Added broadcast push consumer example with default NameServer address
A new PushConsumer example has been added to the broadcast package, demonstrating how to configure a consumer for broadcasting message delivery. The example sets the message model to BROADCASTING, subscribes to TopicTest with specific tag filters, and includes a commented-out line to set the default NameServer address (127.0.0.1:9876), addressing previous issues where the example lacked a default configuration.
example/src/main/java/org/apache/rocketmq/example/broadcast · high confidence
Added configuration files for 2-master 2-slave broker deployments
New property files have been added to the distribution configuration for both asynchronous and synchronous 2-master 2-slave broker topologies. These files define the cluster settings for broker-a and broker-b, specifying their roles (ASYNC\_MASTER, SYNC\_MASTER, or SLAVE) and ensuring all brokers use ASYNC\_FLUSH for disk persistence, allowing users to deploy these specific high-availability architectures.
distribution/conf/2m-2s-async, distribution/conf/2m-2s-sync · high confidence
Added configuration files for 2-master no-slave broker deployment
New configuration files have been added to the distribution package to support a two-master, no-slave broker topology. This includes specific property files for broker-a, broker-b, and a trace-enabled broker (broker-trace), all configured as ASYNC\_MASTER nodes within the DefaultCluster with ASYNC\_FLUSH disk settings.
distribution/conf/2m-noslave · high confidence
Added configuration files for controller mode deployments
New configuration files are now included in the distribution to support running the controller in various modes. This includes standalone and 3-node cluster configurations for the controller itself, as well as setups where the controller runs as a plugin within the NameServer. Additionally, quick-start configuration files for brokers and NameServers are provided to facilitate easy setup of controller-mode environments.
distribution/conf/controller · high confidence
Added core classes for SQL92 message filtering support
Introduced the \UnaryType\ enum and \PropertyExpression\ class within the filter module to support message filtering based on SQL92. The \UnaryType\ enum defines unary operators such as NEGATE, IN, NOT, BOOLEANCAST, and LIKE, while \PropertyExpression\ provides the logic to evaluate property names against the message context, forming the foundational building blocks for the new filtering capability.
filter/src/main/java/org/apache/rocketmq/filter/constant, rocketmq-filter · high confidence
Added dledger cluster management script
A new shell script, fast-try.sh, has been added to the distribution/bin/dledger directory to facilitate the quick start and stop of a dledger cluster. This script automates the launching and termination of the NameServer and three Broker instances (n0, n1, n2) by executing the corresponding mqbroker and mqnamesrv binaries with specific Java memory configurations, ensuring the required configuration files are present before starting services.
distribution/bin/dledger · high confidence
Added order message producer and consumer examples
New example files (Consumer.java and Producer.java) were added to the ordermessage package to demonstrate how to send and consume ordered messages using RocketMQ's DefaultMQProducer and DefaultMQPushConsumer. The producer example shows how to use a MessageQueueSelector to ensure messages with the same order ID are sent to the same queue, while the consumer example illustrates orderly consumption with auto-commit and suspension logic.
example/src/main/java/org/apache/rocketmq/example/ordermessage · high confidence
Added quickstart examples for sending and consuming messages
New example files, Consumer.java and Producer.java, have been added to the quickstart package to demonstrate how to use DefaultMQPushConsumer and DefaultMQProducer. These examples show users how to configure a consumer group, subscribe to a topic, and register a message listener, as well as how to instantiate a producer, send messages synchronously, and handle the resulting SendResult.
example/src/main/java/org/apache/rocketmq/example/quickstart · high confidence
Added request-response RPC example code
New example files (AsyncRequestProducer, RequestProducer, ResponseConsumer) were added to demonstrate the request-response pattern in RocketMQ. These examples show how to send synchronous and asynchronous requests using DefaultMQProducer.request() and how to handle incoming messages and send replies using MessageUtil.createReplyMessage() and a dedicated reply producer.
example/src/main/java/org/apache/rocketmq/example/rpc · high confidence
Added sample configuration for multi-broker container deployment
The distribution now includes a sample configuration set in the \2container-2m-2s\ directory to support deploying multiple brokers within containerized environments. This set provides configuration files for two NameServers and two BrokerContainers, each hosting a master-slave pair (broker-a and broker-b). These files demonstrate how to configure broker roles, storage paths, and container-specific settings like \brokerConfigPaths\ to enable high-availability setups using the BrokerContainer feature.
distribution/conf/container · high confidence
Benchmark tools now support ACL authentication and configurable message compression
The benchmark examples (Producer, Consumer, BatchProducer, TransactionProducer) now include built-in support for RocketMQ Access Control List (ACL) authentication via the new AclClient helper, allowing users to enable ACL by passing the -a flag and providing access/secret keys. Additionally, the producer benchmarks now support configurable message compression (ZLIB, LZ4, SNAPPY) with adjustable levels and size thresholds via command-line options, enabling users to benchmark performance under compressed message scenarios.
example/src/main/java/org/apache/rocketmq/example/benchmark · high confidence
Broker configuration storage migrates to RocksDB with LMQ support
The broker now stores topic, consumer offset, and subscription group configurations in RocksDB instead of JSON files, introducing new managers (RocksDBTopicConfigManager, RocksDBConsumerOffsetManager, RocksDBSubscriptionGroupManager) that handle loading, persisting, and merging data with version checks. This change adds support for LMQ (Local Message Queue) topics and subscription groups within the RocksDB-backed config system, allowing these entities to be managed alongside standard configurations without requiring a migration to RocksDB for the consume queue store.
broker/src/main/java/org/apache/rocketmq/broker/config/v1 · high confidence
Controller adds jRaft task and closure classes
The controller module introduces new implementation classes to support jRaft-based controller operations. This includes \ControllerClosure\, which handles asynchronous responses from jRaft tasks, and a set of request/response pairs in the \task\ package (\BrokerCloseChannelRequest/Response\, \CheckNotActiveBrokerRequest/Response\, \GetBrokerLiveInfoRequest/Response\, \GetSyncStateDataRequest\, \RaftBrokerHeartBeatEventRequest/Response\) that define the data structures for broker communication and state synchronization within the jRaft controller.
controller/src/main/java/org/apache/rocketmq/controller/impl/closure, controller/src/main/java/org/apache/rocketmq/controller/impl/task · high confidence
Controller introduces event classes for broker ID assignment, master election, and sync state updates
The controller module now includes new event classes (AlterSyncStateSetEvent, ApplyBrokerIdEvent, ElectMasterEvent) that formalize internal state changes for broker registration, master election, and sync state set alterations. These events carry structured data such as broker names, addresses, IDs, and sync state sets, enabling the controller to track and propagate these specific operational changes within the cluster management logic.
rocketmq-controller · high confidence
Controller metrics support via OpenTelemetry
The controller now exposes operational metrics using OpenTelemetry, including gauges for disk usage and active broker count, counters for request and DLedger operations, and histograms for request and operation latency. Metrics are exported via OTLP gRPC, Prometheus HTTP server, or JSON logging, with configuration for cardinality limits and aggregation strategies.
controller/src/main/java/org/apache/rocketmq/controller/metrics · high confidence
Initial Bazel build support for RocketMQ
This change introduces the foundational Bazel build configuration files (\bazel/BUILD.bazel\, \bazel/GenTestRules.bzl\, and \bazel/rocketmq\_apis.BUILD\) to enable building and testing RocketMQ using the Bazel build system. It includes a helper macro for generating Java test rules and a minimal build file to expose the \rocketmq\_apis\ submodule's protocol buffer definitions as an external repository, allowing the project to be built with Bazel alongside existing build methods.
bazel · high confidence
Initial OpenMessaging 0.3.0 implementation for RocketMQ
The RocketMQ connector now implements the OpenMessaging specification version 0.3.0, replacing the previous 0.1.0-alpha support. This change introduces the \MessagingAccessPointImpl\ class, which serves as the entry point for creating standard OMS producers, push consumers, and pull consumers, while explicitly throwing exceptions for unsupported features like \StreamingConsumer\ and \ResourceManager\. Additionally, a new \ClientConfig\ class has been added to manage connection settings, including access points, namespaces, operation timeouts, and RocketMQ-specific parameters such as consumer groups, redelivery limits, and thread pool sizes.
openmessaging/src/main/java/io/openmessaging/rocketmq/config, rocketmq-openmessaging · high confidence
Initial SQL92 message filtering support
This change introduces the core expression evaluation engine for SQL92-based message filtering in RocketMQ. It adds the \FilterFactory\ and \FilterSpi\ interface to manage filter registration, along with the \SqlFilter\ implementation that delegates to the new parser. The \expression\ package provides the full set of expression classes (binary, unary, logic, comparison, and constant) required to parse and evaluate SQL92 filter conditions against message properties.
filter/src/main/java/org/apache/rocketmq/filter/expression · high confidence
Initial implementation of BrokerOuterAPI for cluster communication
The broker now includes a new \BrokerOuterAPI\ class that handles all outbound network communication with NameServers and Controllers. This component manages broker registration, heartbeat reporting, route table retrieval, and controller-specific operations such as master election and sync state set updates, replacing previous ad-hoc or distributed implementations with a centralized, configurable client.
broker/src/main/java/org/apache/rocketmq/broker/out · high confidence
Initial implementation of OpenMessaging 0.1.0-alpha push and pull consumers
This change introduces the core consumer implementations for the OpenMessaging (OMS) bridge, adding \PushConsumerImpl\ and \PullConsumerImpl\ along with a new \ClientConfig\ class to manage RocketMQ-specific settings (such as consumer groups, thread pools, and redelivery limits). The consumers are configured to identify themselves with the OMS language code and support attaching/detaching queues, with push consumers registering message listeners and pull consumers utilizing a scheduled pull service with a local message cache. A specific behavioral detail is that the nameserver address is only set directly from access points when the \OMS\_RMQ\_DIRECT\_NAME\_SRV\ environment variable is enabled.
openmessaging/src/main/java/io/openmessaging/rocketmq/consumer · high confidence
Initial project scaffolding and build configuration
This change introduces the foundational project structure and build tooling for Apache RocketMQ. It adds the Apache License and NOTICE files, establishes a Bazel build system (including WORKSPACE, MODULE.bazel, and .bazelrc configurations), and provides essential developer documentation such as BUILDING and CONTRIBUTING guides. The repository is also configured with standard CI/CD files (.asf.yaml, .gitignore) and updated documentation (README.md) to reflect the current release version and quick-start instructions.
(repo-wide) · high confidence
Introduce ACL 2.0 authorization context builder
Added the \AuthorizationContextBuilder\ interface and its \DefaultAuthorizationContextBuilder\ implementation to support the new ACL 2.0 model. This component extracts authorization context (such as topic, group, and action) from both gRPC telemetry requests (e.g., \SendMessageRequest\, \HeartbeatRequest\) and legacy Remoting commands, enabling the system to correctly identify resources for permission checks under the updated authorization framework.
auth/src/main/java/org/apache/rocketmq/auth/authorization/builder · high confidence
Introduce Bazel build support for the tools module
The \tools\ module now includes a \BUILD.bazel\ file, enabling builds via the Bazel build system. This configuration defines the \tools\ and \tests\ Java libraries, specifying their source files and dependencies on internal modules (such as \remoting\, \client\, \common\, and \srvutil\) and external Maven artifacts (including Netty, Commons CLI, RocksDB, and Fastjson2).
tools · high confidence
Introduce ControllerRequestProcessor for centralized request handling
The controller module now uses a dedicated ControllerRequestProcessor to handle incoming network requests. This processor implements NettyRequestProcessor and routes commands such as broker registration, heartbeat, master election, and configuration updates to the ControllerManager. It also adds OpenTelemetry-based metrics collection for request latency and success/failure status, providing better observability for controller operations.
controller/src/main/java/org/apache/rocketmq/controller/processor · high confidence
Introduce GrpcChannelManager for gRPC client channel and response lifecycle management
The proxy now includes a new GrpcChannelManager component that centralizes the creation, retrieval, and removal of gRPC client channels per client ID. It also manages the lifecycle of asynchronous response futures by assigning nonces, storing them in a concurrent map, and running a scheduled task every second to scan for and timeout expired requests based on the configured grpcProxyRelayRequestTimeout.
proxy/src/main/java/org/apache/rocketmq/proxy/grpc/v2/channel · high confidence
Introduce LMQ subscription group support
The broker now includes a dedicated LmqSubscriptionGroupManager that extends the standard SubscriptionGroupManager to handle Light Message Queue (LMQ) groups. This change allows the broker to recognize, create, and manage subscription groups specifically identified as LMQs, ensuring they are handled separately from regular consumer groups during lookup and update operations.
broker/src/main/java/org/apache/rocketmq/broker/subscription · high confidence
Introduce MessageService abstraction for proxy message operations
The proxy now uses a new \MessageService\ interface to abstract message operations, with two implementations: \ClusterMessageService\ for remote broker communication and \LocalMessageService\ for local broker processing. This change enables the proxy to support local mode operation, where message sending, popping, and acknowledgment are handled directly by the embedded broker controller rather than via network calls, improving performance and enabling standalone proxy testing.
proxy/src/main/java/org/apache/rocketmq/proxy/service/message · high confidence
Introduce NettyEvent and NettyRemotingAbstract for remoting layer
The remoting module now includes a new NettyEvent class to encapsulate Netty channel events (type, remote address, and channel) and a new NettyRemotingAbstract abstract class that serves as the core implementation for handling remoting commands, managing request/response tables, semaphores for oneway/async limits, SSL contexts, RPC hooks, and metrics integration.
rocketmq-remoting · high confidence
Introduce ReplicasManager for controller-mode broker coordination
The broker now includes a new ReplicasManager component that handles registration with the controller, synchronizes metadata, and manages role transitions (master/slave) in controller mode. This component ensures reliable broker registration, handles controller address discovery, and coordinates high-availability state changes, replacing previous ad-hoc logic with a dedicated manager for replica coordination.
broker/src/main/java/org/apache/rocketmq/broker/controller · high confidence
Introduce RocketMQ BrokerContainer for hosting multiple brokers
Adds a new BrokerContainer module that allows a single process to host and manage multiple broker instances (masters, slaves, and DLedger brokers) via a shared remoting server and outer API. This introduces a new startup entry point (BrokerContainerStartup) and configuration model (BrokerContainerConfig) for deploying multiple brokers in a single JVM, along with lifecycle hooks (BrokerBootHook) for pre- and post-start customization.
container/src/main · high confidence
Introduce RocksDB-backed Pop consumer state management
The broker now uses a new RocksDB-based storage layer (PopConsumerRocksdbStore) to persist Pop consumer records, replacing or augmenting previous in-memory or file-based approaches. This change introduces a dedicated cache (PopConsumerCache) and lock service (PopConsumerLockService) to manage in-flight messages, visibility timeouts, and offset commits for Pop consumers, improving reliability and performance for FIFO and standard Pop consumption patterns.
broker/src/main/java/org/apache/rocketmq/broker/pop · high confidence
Introduce Tiered Storage module for offloading message data
This change introduces the Tiered Storage module, a technical preview feature that allows RocketMQ brokers to offload message data from local disk to cheaper, larger storage mediums (such as POSIX file systems or object stores like S3). By enabling this via the \messageStorePlugIn\ configuration, users can extend message retention times at a lower cost and flexibly specify different TTLs per topic. The module includes a new \TieredMessageStore\ plugin, configuration options for backend providers and read-ahead caching, and specific metrics for monitoring dispatch and storage operations.
tieredstore · high confidence
Introduce adaptive cold read flow control strategies
The broker now supports configurable cold data read flow control strategies. Administrators can enable an adaptive PID-based strategy (via \usePIDColdCtrStrategy\) that dynamically adjusts per-consumer-group read thresholds based on global accumulation error, or fall back to a simpler static strategy. This change introduces the \ColdCtrStrategy\ interface, \PIDAdaptiveColdCtrStrategy\, and \SimpleColdCtrStrategy\ implementations, alongside the \ColdDataCgCtrService\ which manages runtime thresholds and the \ColdDataPullRequestHoldService\ which temporarily holds pull requests for cold data to enforce the flow control.
broker/src/main/java/org/apache/rocketmq/broker/coldctr · high confidence
Introduce centralized JSON-based proxy configuration system
The proxy now uses a new configuration framework centered on \ConfigurationManager\ and \Configuration\ classes to load settings from a JSON file (defaulting to \rmq-proxy.json\). This system supports locating the config file via the \RMQ\_PROXY\_HOME\ environment variable or the \com.rocketmq.proxy.configPath\ system property, and it unifies the management of both \ProxyConfig\ and \AuthConfig\. Additionally, a new \MetricCollectorMode\ enum is introduced to allow users to explicitly configure whether metrics are collected by the proxy itself, sent to an external address, or disabled entirely.
proxy/src/main/java/org/apache/rocketmq/proxy/config · high confidence
Introduce dedicated long-polling services for Lite and LMQ consumption modes
The broker now includes new long-polling infrastructure classes to support Lite consumption and Local Message Queue (LMQ) scenarios. PopLiteLongPollingService handles long-polling for Lite consumers using a clientId-based key and a lightweight in-memory map, while LmqPullRequestHoldService extends the standard PullRequestHoldService to manage hold requests specifically for LMQ topics, including logic to clean up empty polling entries. Supporting classes such as PollingHeader, PollingResult, NotificationRequest, ManyPullRequest, and PullRequest provide the necessary data structures and result enums for these new polling flows.
broker/src/main/java/org/apache/rocketmq/broker/longpolling · high confidence
Introduce expression-based message filtering with bloom filter optimization
The broker now supports advanced message filtering using SQL92 expressions (in addition to existing tag filtering). This change introduces a new filtering architecture in the broker, including \ConsumerFilterManager\ to manage filter data, \ConsumerFilterData\ to store compiled expressions and metadata, and \ExpressionMessageFilter\ to evaluate messages against these expressions. A key performance enhancement is the integration of Bloom filters to quickly determine if a message might match a consumer's filter before full evaluation, reducing unnecessary processing. The system also handles retry topics correctly by resolving the original topic for filter evaluation.
broker/src/main/java/org/apache/rocketmq/broker/filter · high confidence
Introduce gRPC v2 proxy implementation with auto-renew and lite consumer support
The proxy now includes a new gRPC v2 implementation (GrpcMessagingApplication) that handles message sending, receiving, and client management via the v2 protocol. This change introduces support for auto-renewal of message visibility, allowing consumers to extend visibility without explicit calls, and adds support for Lite Consumers, which use a selective subscription model. The implementation includes new components like ReceiptHandleGroup for managing message locks, PopMessageResultFilterImpl for filtering messages based on retry attempts, and GrpcClientChannel for managing client connections. It also integrates authentication and authorization pipelines, and adds configuration for gRPC thread pools and rate limiting for lite subscription synchronization.
rocketmq-proxy · high confidence
Introduce in-process relay pipeline for proxy operations
The proxy now supports in-process relay for specific operations (GetConsumerRunningInfo, ConsumeMessageDirectly, CheckTransactionState) via a new relay service layer. This allows the proxy to handle these requests locally without requiring a co-located broker, improving functionality in cluster mode where the proxy acts as the caller. The implementation includes a new \ProxyRelayService\ interface, concrete implementations for local and cluster modes, and supporting classes like \ProxyRelayRequest\ to carry response futures within the relay pipeline.
proxy/src/main/java/org/apache/rocketmq/proxy/service/relay · high confidence
Introduce new ACL 2.0 authentication evaluation engine
The authentication module now uses a new evaluation engine (AuthenticationEvaluator) that delegates to pluggable strategies (defaulting to StatelessAuthenticationStrategy) and a handler chain (DefaultAuthenticationHandler) for user lookup and signature verification. Context is built via AuthenticationContextBuilder (with a DefaultAuthenticationContextBuilder implementation) from both gRPC Metadata and RemotingCommand sources, and user metadata is managed through a new AuthenticationMetadataManager (implemented by AuthenticationMetadataManagerImpl) that coordinates with authorization providers. This replaces the previous authentication flow in this package with a structured, extensible pipeline for ACL 2.0.
auth/src/main/java/org/apache/rocketmq/auth/authentication · high confidence
Introduce new proxy startup, configuration, and admin gRPC interfaces
The proxy now features a new startup entry point (ProxyStartup) that accepts command-line arguments for configuration paths and proxy mode (local or cluster), and initializes both a data-plane gRPC server and a dedicated admin gRPC server for management operations. This includes new model converters for admin responses and a remoting protocol handler interface, enabling users to start the proxy with explicit configuration and interact with its new administrative capabilities.
proxy/src/main/java/org/apache/rocketmq/proxy · high confidence
Introduce proxy common infrastructure for message renewal and context management
The proxy now includes a new set of common classes to support message receipt handling and renewal strategies. This includes a \ProxyContext\ for managing request-scoped data (such as client ID, namespace, and action), a \MessageReceiptHandle\ to track message consumption state and renewal attempts, and a \RenewStrategyPolicy\ that defines the back-off intervals for message visibility renewal. Additionally, supporting classes like \Address\ for host/port parsing, \ContextVariable\ constants, and \RenewEvent\/\ReceiptHandleGroupKey\ for event tracking have been added to facilitate these operations.
proxy/src/main/java/org/apache/rocketmq/proxy/common · high confidence
Introduce srvutil module with server utilities and file watching
The new srvutil module provides shared server utilities, including command-line argument parsing (help and namesrvAddr options) via ServerUtil, a standardized shutdown hook mechanism via ShutdownHookThread, and a FileWatchService for monitoring file changes. The module is built with Bazel and includes unit tests for these components.
srvutil · high confidence
Introduce standalone Controller module for broker management
Adds a new standalone Controller component that manages broker registration, heartbeat monitoring, and master election. The module introduces a \ControllerManager\ to initialize and run the service, a \Controller\ interface defining the broker management API (register, elect, sync state), and a \BrokerHeartbeatManager\ to track broker liveness. It supports two backend implementations, DLedger and jRaft, and includes a startup class (\ControllerStartup\) for launching the server and an \ElectPolicy\ interface to allow configurable master election strategies.
controller/src/main/java/org/apache/rocketmq/controller · high confidence
Introduce transactional message metrics tracking and flushing
The broker now tracks and persists metrics for transactional messages. A new \TransactionMetrics\ class stores per-topic counts, and a \TransactionMetricsFlushService\ background thread periodically persists these metrics to disk, ensuring that transactional message activity is observable and survives broker restarts.
broker/src/main/java/org/apache/rocketmq/broker/transaction · high confidence
Introduce v2 Config Manager backed by RocksDB for broker metadata
The broker now includes a new \ConfigManagerV2\ implementation in \broker/src/main/java/org/apache/rocketmq/broker/config/v2\ that stores core metadata—specifically topic configurations, subscription groups, and consumer/pull offsets—in RocksDB. This v2 storage layer replaces the previous heap-based caching approach with a persistent, WAL-enabled RocksDB store to improve data integrity and reliability. The package introduces helper classes (\ConfigHelper\, \ConfigStorage\) and specific managers (\TopicConfigManagerV2\, \SubscriptionGroupManagerV2\, \ConsumerOffsetManagerV2\) that handle the serialization (JSON), key layout, and versioning of these configurations.
broker/src/main/java/org/apache/rocketmq/broker/config/v2 · high confidence
Introduces ACL 2.0 client authentication and refactors core client components
The client now supports the new ACL 2.0 protocol via the new \AclClientRPCHook\, which automatically signs requests with AccessKey, SecurityToken, and signatures before transmission. This change also introduces several foundational refactors: a new \ClientRemotingProcessor\ handles broker-to-client requests (such as transaction checks and offset resets), distinct rebalance implementations are added for push, pull, and lite-pull consumers to better manage queue assignment and offset persistence, and the \MQClientInstance\ is updated to support concurrent heartbeats and dynamic name server address fetching. Additionally, a new \ThreadLocalIndex\ utility is provided for thread-safe index generation.
rocketmq-client · high confidence
Introduces AbortProcessException to enforce immediate hook termination
The \common\ module now includes the \AbortProcessException\ class, a new runtime exception designed specifically for broker hooks (SendMessageHook, ConsumeMessageHook, and RPCHook). When thrown by a hook implementation, this exception forces the broker to immediately return an error response to the client with a specific response code, bypassing normal processing flow. This provides a standardized mechanism for hooks to enforce strict validation or policy decisions that require immediate request rejection.
common/src/main · high confidence
Introduces EscapeBridge for remote message escaping in Slave Acting Master mode
The new EscapeBridge component in the broker's failover package enables the asynchronous escaping of messages to a remote broker when the local broker is acting as a master for a slave. This change supports the Slave Acting Master feature by providing mechanisms to send messages (including transactional half-messages) to remote brokers via an async executor, ensuring message durability and availability during failover scenarios.
broker/src/main/java/org/apache/rocketmq/broker/failover · high confidence
Introduces Lite Event Dispatcher, Batch Splitting Metric Exporter, and configurable Pop Order Lock Timer
The broker now includes a new LiteEventDispatcher to handle event dispatching for the Lite Topic model, supporting wildcard subscriptions and prefix index maintenance for LMQs. To prevent OTLP gRPC export failures caused by large metric batches (e.g., high-cardinality consumer lag), a BatchSplittingMetricExporter is added to split exports based on data point counts and concurrency limits. Additionally, the HashedWheelTimer parameters for the QueueLevelConsumerOrderInfoLockManager are now configurable via broker settings, allowing tuning of tick intervals and wheel ticks for pop-order lock notifications.
rocketmq-broker · high confidence
Introduces RocketMQ ACL 2.0 authorization and authentication pipelines
The broker now integrates a new ACL 2.0 authorization and authentication framework. This change adds an \AuthenticationPipeline\ and an \AuthorizationPipeline\ to the broker's request processing chain, which enforce access control using the new \AuthorizationEvaluator\ and \AuthorizationCompatibility\ logic. To support this, the broker includes \AclConverter\ and \UserConverter\ classes that translate between the legacy \AclInfo\/\UserInfo\ protocol structures and the new ACL 2.0 model objects (Subject, Policy, Resource, Action). This enables the broker to evaluate authorization contexts for both legacy Remoting commands and the new gRPC-based telemetry requests.
auth/src/main/java/org/apache/rocketmq/auth/authorization, broker/src/main/java/org/apache/rocketmq/broker/auth · high confidence
Introduces RocksDB-backed compaction and flush-watcher components
The store module adds new internal classes to support RocksDB-based consume queue storage and optimized disk flushing. Specifically, it introduces FlushDiskWatcher to manage asynchronous flush requests with timeout handling, CompactionLog to handle message compaction logic for RocksDB-backed stores, and OffsetInitializerRocksDBImpl to initialize offsets by querying the RocksDB store. These changes enable the store to leverage RocksDB for consume queue operations and improve the reliability of disk flush operations.
rocketmq-store · high confidence
Introduces a new request processing pipeline for the proxy remoting layer
The proxy remoting layer now uses a structured pipeline architecture to handle incoming requests. This change adds the \RequestPipeline\ interface and three concrete implementations: \ContextInitPipeline\ (which initializes the \ProxyContext\ with channel metadata like client ID and language), \AuthenticationPipeline\ (which performs authentication when enabled), and \AuthorizationPipeline\ (which performs authorization when enabled). This replaces the previous ad-hoc handling with a composable chain of responsibilities, allowing for cleaner separation of context initialization, authentication, and authorization logic within the proxy.
proxy/src/main/java/org/apache/rocketmq/proxy/remoting/pipeline · high confidence
Introduces broker-side infrastructure for the new Lite Topic message model
This change adds the core broker components required to support the new Lite Topic message model. It introduces lifecycle managers (AbstractLiteLifecycleManager, LiteLifecycleManager, and RocksDBLiteLifecycleManager) to handle topic TTL, subscription validity, and offset tracking for both file-based and RocksDB-backed message stores. A new LmqPrefixIndex (backed by a PatriciaTrie) accelerates wildcard dispatch by indexing LMQ names. Exclusive subscription eviction is now enforced via server-side tombstones managed by ExclusiveEvictionTombstones, and metadata utilities (LiteMetadataUtil) provide configuration checks for Lite-specific group and topic behaviors.
broker/src/main/java/org/apache/rocketmq/broker/lite · high confidence
Introduces default broker election policy for controller mode
Adds the DefaultElectPolicy implementation, which defines the logic for selecting a new master broker when the current one fails. The policy prioritizes the existing master if it remains valid, respects a preferred broker ID if specified, and otherwise selects the best candidate from the sync state set (or all replicas if necessary) based on a ranking of epoch, max offset, and election priority.
controller/src/main/java/org/apache/rocketmq/controller/elect/impl · high confidence
Introduces gRPC request pipeline for authentication, authorization, and context initialization
The gRPC proxy now processes incoming requests through a structured pipeline consisting of context initialization, authentication, and authorization stages. ContextInitPipeline extracts client metadata (such as address, ID, and protocol type) from gRPC headers into the proxy context. AuthenticationPipeline validates credentials using the new ACL 2.0 authentication evaluator and injects the authenticated identity into the request headers. AuthorizationPipeline enforces access control policies via the ACL 2.0 authorization evaluator. This pipeline architecture provides a modular foundation for securing gRPC traffic in the proxy.
proxy/src/main/java/org/apache/rocketmq/proxy/grpc/pipeline · high confidence
Introduction of AuthorizationFactory for centralized authorization component creation
A new AuthorizationFactory class has been added to the auth module to centralize the creation and caching of authorization components, including providers, metadata providers, evaluators, and strategies. This factory uses a singleton map to ensure that instances are created only once per configuration name, improving efficiency and consistency in how authorization logic is initialized and accessed within the system.
auth/src/main/java/org/apache/rocketmq/auth/authorization/factory · high confidence
Introduction of AuthorizationMetadataManager for ACL operations
The authorization module now includes a new \AuthorizationMetadataManager\ interface and its \AuthorizationMetadataManagerImpl\ implementation to manage Access Control Lists (ACLs). This component provides methods for creating, updating, and deleting ACLs, as well as retrieving them by subject. The implementation validates that subjects (such as users) exist before performing operations and delegates storage tasks to configured \AuthorizationMetadataProvider\ and \AuthenticationMetadataProvider\ instances, ensuring that ACL operations are backed by the appropriate metadata services.
auth/src/main/java/org/apache/rocketmq/auth/authorization/manager · high confidence
Introduction of Broker Plugin and Processor Architecture
The broker now supports an extensible plugin architecture and a refactored request processing model. A new \BrokerAttachedPlugin\ interface allows external modules to hook into the broker lifecycle (load, start, shutdown, metadata sync, and status changes). Additionally, core request handling has been reorganized into dedicated processor classes (such as \AckMessageProcessor\, \PeekMessageProcessor\, and \AbstractSendMessageProcessor\) and a \PullMessageResultHandler\ interface, replacing the previous monolithic processing logic to improve modularity and maintainability.
broker/src/main/java/org/apache/rocketmq/broker/processor · high confidence
Introduction of RebalanceLockManager for message queue locking
The broker now includes a new RebalanceLockManager component in the rebalance package to manage distributed locks for message queues. This manager handles lock acquisition, expiration checks, and batch locking operations for consumer groups, ensuring that message queues are correctly assigned and locked during the rebalancing process.
broker/src/main/java/org/apache/rocketmq/broker/client/rebalance · high confidence
Introduction of RocketMQ ACL 2.0 client support
The client module now includes the foundational classes for the new ACL 2.0 security model. This change introduces the \AclSigner\ for calculating request signatures, \AclUtils\ for handling network address validation (including IPv6), and \SessionCredentials\ for managing access keys and secret keys. It also adds the \AccessChannel\ configuration to distinguish between local and cloud deployments, and provides the \MQAdmin\ and \MqClientAdmin\ interfaces to expose management capabilities such as topic creation, offset querying, and cluster statistics.
client/src/main/java · high confidence
Introduction of static topic queue mapping and LMQ support
The broker now supports static topic queue mapping and Light Message Queues (LMQ). A new TopicQueueMappingManager and TopicQueueMappingCleanService manage static topic queue mappings, allowing for persistent, versioned queue assignments that are cleaned up based on broker load and epoch changes. Additionally, a new LmqTopicConfigManager extends TopicConfigManager to handle LMQ-specific topic configurations, ensuring that LMQ topics are treated as existing and selected appropriately without requiring full configuration updates.
broker/src/main/java/org/apache/rocketmq/broker/topic · high confidence
Multi-protocol remoting server supports HTTP/2 alongside Remoting
The proxy now includes a new MultiProtocolRemotingServer that allows both the standard RocketMQ Remoting protocol and HTTP/2 to be served on the same port. This is enabled by a new MultiProtocolTlsHelper that configures SSL/TLS with ALPN to negotiate HTTP/2, and a ProtocolNegotiationHandler that routes connections to the appropriate protocol handler. A new RemotingProxyOutClient interface is also introduced to support client-side invocation.
proxy/src/main/java/org/apache/rocketmq/proxy/remoting · high confidence
Namespace v2 example code added
The namespace example directory now includes three new Java files demonstrating how to use the namespace v2 feature: ProducerWithNamespace, PullConsumerWithNamespace, and PushConsumerWithNamespace. These examples show how to configure DefaultMQProducer, DefaultMQPullConsumer, and DefaultMQPushConsumer with the setNamespaceV2 method to operate within a specific namespace instance.
example/src/main/java/org/apache/rocketmq/example/namespace · high confidence
Namesrv introduces KVConfigManager for namespace-scoped key-value configuration
The NameServer now supports storing and retrieving key-value configuration data organized by namespace. A new KVConfigManager handles loading, persisting, and managing these configurations in memory and on disk, while KVConfigSerializeWrapper provides the serialization structure for the config table. This enables users to manage specific configuration items within isolated namespaces via the NameServer.
namesrv/src/main/java/org/apache/rocketmq/namesrv/kvconfig · high confidence
Nearby routing support via zone-based filtering in NameServer
The NameServer now supports a 'nearby routing' feature that filters topic route data based on the consumer's availability zone. A new ZoneRouteRPCHook intercepts GET\_ROUTEINFO\_BY\_TOPIC responses when zone mode is enabled, removing broker data and associated queue data that do not match the requested zone name, while preserving masters and slaves in the same zone. This allows clients to receive route information containing only brokers within their specific zone, optimizing network latency for zone-aware deployments.
namesrv/src/main/java/org/apache/rocketmq/namesrv/route · high confidence
New AdminService interface for proxy-side cluster management
The proxy now exposes an AdminService interface and its DefaultAdminService implementation, enabling administrators to manage the RocketMQ cluster directly through the proxy. This new capability allows querying cluster topology, topic routes, and consumer connections, as well as performing mutations like creating topics, resetting consumer offsets, and managing subscription groups. All operations are asynchronous and routed through the proxy's internal broker client, ensuring that administrative tasks do not block the main gRPC execution threads.
proxy/src/main/java/org/apache/rocketmq/proxy/service/admin · high confidence
New Broker2Client component for managing client communication
The broker now includes a new \Broker2Client\ class in the \org.apache.rocketmq.broker.client.net\ package to handle interactions with connected clients. This component centralizes logic for notifying consumers of subscription changes, checking transaction states for producers, and resetting consumer offsets based on time or force flags. It also supports specific serialization formats for C++ clients during offset resets, providing a structured way for the broker to send commands to client channels.
broker/src/main/java/org/apache/rocketmq/broker/client/net · high confidence
New BrokerFastFailure component for proactive queue cleanup
A new BrokerFastFailure class has been added to the broker latency package. It introduces a scheduled background task that proactively cleans expired requests from various thread-pool queues (send, pull, lite-pull, heartbeat, transaction, ack, and admin broker). When a request exceeds its configured wait time in a queue, it is removed and the client receives a SYSTEM\_BUSY response, helping to prevent thread starvation and improve broker responsiveness under load.
broker/src/main/java/org/apache/rocketmq/broker/latency · high confidence
New DLedger role change handler for broker state transitions
Added DLedgerRoleChangeHandler to manage broker role transitions (leader, follower, candidate) within the DLedger replication subsystem. This component ensures that when a broker's role changes, it correctly updates its internal state, registers with the cluster, and manages slave synchronization tasks, thereby supporting high-availability scenarios such as slave acting as master.
broker/src/main/java/org/apache/rocketmq/broker/dledger · high confidence
New LMQ example applications demonstrating multi-dispatch and consumption patterns
Added four new example programs in the LMQ (Local Message Queue) package to demonstrate how to use RocketMQ's multi-dispatch feature. LMQProducer shows how to send messages to a parent topic while dispatching copies to multiple LMQ sub-topics using the INNER\_MULTI\_DISPATCH property. LMQPushConsumer, LMQPullConsumer, and LMQPushPopConsumer provide examples of consuming these dispatched messages using push, pull, and pop-consume modes respectively, including the necessary client-side route compensation logic required for LMQ topics.
example/src/main/java/org/apache/rocketmq/example/lmq · high confidence
New OpenMessaging producer implementation for RocketMQ
Added AbstractOMSProducer and ProducerImpl classes to implement the OpenMessaging specification for RocketMQ producers. The implementation supports synchronous, asynchronous, and oneway message sending, with specific handling for BytesMessage types. Configuration includes conditional name server address resolution via the OMS\_RMQ\_DIRECT\_NAME\_SRV environment variable, and client language is tagged as 'OMS'.
openmessaging/src/main/java/io/openmessaging/rocketmq/producer · high confidence
New OpenTelemetry-based broker metrics framework
The broker now exposes a comprehensive set of observability metrics using the OpenTelemetry standard, replacing or augmenting previous internal metrics. This change introduces a new metrics framework in the broker module, including constants, managers, and calculators for tracking broker health (topic/consumer group counts, permissions), message flow (throughput, message sizes, commit/rollback counts), client connections, and detailed consumer lag and latency. It also adds specific metrics for the Pop consumption model (buffer sizes, revive lag/latency) and remoting stats. The implementation supports multiple export backends, including Prometheus, OTLP gRPC, and JSON logging, and includes optimizations like pre-built AttributeKey instances to reduce allocation overhead.
broker/src/main/java/org/apache/rocketmq/broker/metrics · high confidence
New ReceiptHandleManager interface and DefaultReceiptHandleManager implementation
The proxy now introduces a dedicated \ReceiptHandleManager\ interface and its \DefaultReceiptHandleManager\ implementation to centralize the management of message receipt handles. This component maintains a concurrent map of receipt handle groups keyed by channel and consumer group, providing methods to add, remove, and query unacked message counts. It includes a scheduled task that scans for messages nearing their visibility timeout and submits renewal tasks to a dedicated worker thread pool, while also listening for client unregistration events to automatically clear stale handle data.
proxy/src/main/java/org/apache/rocketmq/proxy/service/receipt · high confidence
New SlaveSynchronize component for master-slave data replication
A new SlaveSynchronize class has been added to the broker module to handle the synchronization of configuration and state data from the master broker to the slave. This component implements specific sync methods for topic configurations, consumer offsets, delay offsets, subscription group configurations, and message request modes, ensuring that the slave broker maintains consistency with the master's state. It also includes logic to sync timer metrics when the timer wheel is enabled, providing a centralized mechanism for maintaining high availability through data replication.
broker/src/main/java/org/apache/rocketmq/broker/slave · high confidence
New authorization provider interfaces and local RocksDB-backed metadata storage
The auth module introduces a new provider layer for authorization logic and metadata management. This includes the \AuthorizationProvider\ interface for handling authorization contexts and the \AuthorizationMetadataProvider\ interface for ACL CRUD operations. The \DefaultAuthorizationProvider\ implements the core authorization flow using a handler chain (User and ACL handlers) and adds audit logging for allow/deny decisions. Additionally, \LocalAuthorizationMetadataProvider\ provides a concrete implementation that stores ACL metadata in a local RocksDB instance, backed by a Caffeine cache for performance, with proper lifecycle management for the database and cache refresh executor.
auth/src/main/java/org/apache/rocketmq/auth/authorization/provider · high confidence
New authorization strategy framework with stateful caching
The authorization module introduces a new strategy pattern for access control, featuring an \AbstractAuthorizationStrategy\ base class that handles whitelist bypasses and delegates to providers. Two concrete implementations are provided: \StatelessAuthorizationStrategy\ for direct evaluation and \StatefulAuthorizationStrategy\, which caches authorization results using Caffeine to improve lookup efficiency for repeated requests based on channel ID, subject, resource, actions, and source IP.
auth/src/main/java/org/apache/rocketmq/auth/authorization/strategy · high confidence
New benchmark scripts for batch and transaction producers
The distribution/benchmark directory now includes dedicated startup scripts for batch and transaction producers (batchproducer.sh and tproducer.sh), alongside a centralized shutdown.sh script that supports stopping producer, consumer, transaction producer, and batch producer instances. These scripts leverage an updated runclass.sh utility that handles JVM configuration, GC logging (including RAM disk support on macOS and /dev/shm on Linux), and classpath setup, providing a more structured way to run and manage benchmark workloads.
distribution/benchmark · high confidence
New broker logging configuration and transaction metadata schema
The broker now ships with a dedicated \rmq.broker.logback.xml\ configuration file that defines structured, asynchronous logging for specific subsystems (default, broker, protection, watermark, RocksDB, and store) with configurable log directories and file rotation policies. Additionally, a new \transaction.sql\ schema file is included to define the \t\_transaction\ table structure for storing transaction metadata.
broker/src/main/resources · high confidence
New broker utility classes for message validation and atomic counting
Added two new utility classes to the broker module: HookUtils, which centralizes pre-send message validation logic (including checks for topic length, batch consistency, timer message handling, and LMQ quota enforcement), and PositiveAtomicCounter, a thread-safe counter that ensures values remain positive by masking the sign bit. These utilities support the broker's message processing pipeline by providing reusable, standardized checks and counters.
broker/src/main/java/org/apache/rocketmq/broker/util · high confidence
New broker-side message tracing context and hook interfaces
The broker now exposes dedicated context objects and hook interfaces for tracing message operations. New classes \ConsumeMessageContext\ and \SendMessageContext\ capture detailed metadata for consumption and sending events respectively, including fields for namespace, account statistics, commercial usage, and message identifiers. Corresponding interfaces \ConsumeMessageHook\ and \SendMessageHook\ allow external components to intercept and observe these events before and after message processing, enabling enhanced monitoring and auditing capabilities for message flow within the broker.
broker/src/main/java/org/apache/rocketmq/broker/mqtrace · high confidence
New channel abstraction and serialization interfaces
The proxy module introduces a new set of interfaces and utilities in the \proxy.processor.channel\ package to standardize channel handling. This includes the \ChannelProtocolType\ enum to identify protocol variants (gRPC v1/v2, remoting), and interfaces \ChannelExtendAttributeGetter\ and \RemoteChannelConverter\ to abstract channel attributes and conversion logic. Additionally, \RemoteChannelSerializer\ provides JSON serialization and deserialization for \RemoteChannel\ objects using Fastjson2, enabling consistent handling of remote channel metadata across the proxy.
proxy/src/main/java/org/apache/rocketmq/proxy/processor/channel · high confidence
New common infrastructure for service lifecycle, typed attributes, logging configuration, and thread monitoring
The rocketmq-common module introduces several foundational components to support new message models and improve operational visibility. It adds a LifecycleAwareServiceThread base class to help service threads manage startup synchronization safely. A new attribute system (BooleanAttribute, EnumAttribute, LongRangeAttribute, StringAttribute) provides a standardized way to define, validate, and manage topic and message properties. Logging is enhanced via a DefaultJoranConfiguratorExt that supports component-specific configuration files (e.g., broker, namesrv, proxy) and warns on classpath duplicates. Additionally, a ThreadPoolQueueSizeMonitor is added to track thread pool queue sizes and trigger diagnostics when usage exceeds 85% of capacity.
rocketmq-common · high confidence
New controller event model and serialization infrastructure
The controller module now uses a new internal event system to manage broker state changes. This introduces a set of event types (AlterSyncStateSet, ApplyBrokerId, ElectMaster, CleanBrokerData, UpdateBrokerAddress) and corresponding message classes, along with serializers to handle their serialization and deserialization. This change provides the underlying mechanism for the controller to track and propagate broker lifecycle events, such as address updates and data cleanup, within the new jRaft-based controller implementation.
controller/src/main/java/org/apache/rocketmq/controller/impl/event · high confidence
New distribution/bin scripts for RocketMQ components and utilities
This change introduces a comprehensive set of new shell and batch scripts in the distribution/bin directory to manage RocketMQ services. It adds dedicated launchers for the Broker (mqbroker), NameServer (mqnamesrv), Controller (mqcontroller), Proxy (mqproxy), and BrokerContainer (mqbrokercontainer), along with their Windows counterparts. The Broker script now supports an --enable-proxy flag to switch between standard broker and proxy modes. New utility scripts include mqadmin for administrative tasks, mqshutdown for stopping services, and os.sh for system tuning. Additionally, helper scripts like play.sh/play.cmd for quick starts, cachedog.sh/cleancache.sh for memory management, and export.sh for configuration export are provided.
distribution/bin · high confidence
New examples for scheduled and timer message delivery
Added four new example classes in the schedule package to demonstrate message scheduling capabilities: ScheduledMessageProducer and ScheduledMessageConsumer show how to send and receive messages with a fixed delay using delay time levels, while TimerMessageProducer and TimerMessageConsumer illustrate timer-based delivery for RocketMQ 5.0+, supporting arbitrary time delays via specific delivery timestamps or millisecond offsets.
example/src/main/java/org/apache/rocketmq/example/schedule · high confidence
New gRPC v2 client activity handler for heartbeat and termination
The proxy now includes a new \ClientActivity\ class in the gRPC v2 client package to handle client lifecycle events. This component implements the \heartbeat\ method, which registers producers and consumers (including new Lite Push and Simple consumer types) based on client settings, and the \notifyClientTermination\ method, which unregisters these clients and cleans up channel resources when a client disconnects. This change introduces the core logic for managing client connections and subscriptions in the v2 gRPC protocol implementation.
proxy/src/main/java/org/apache/rocketmq/proxy/grpc/v2/client · high confidence
New gRPC v2 proxy common utilities for settings, conversion, and validation
The proxy now includes a new set of common classes in the gRPC v2 package to support the updated protocol. GrpcClientSettingsManager handles client-specific settings, merging producer and subscription data (including retry policies and long-polling configurations) and configuring metric collection endpoints. GrpcConverter translates internal message structures into the v2 protobuf format, handling system properties, message IDs, and topic types. GrpcValidator enforces rules for topics, consumer groups, tags, and lite topics. ResponseBuilder and ResponseWriter standardize error code mapping and gRPC response writing, while GrpcProxyException provides a unified exception type for gRPC-specific errors.
proxy/src/main/java/org/apache/rocketmq/proxy/grpc/v2/common · high confidence
New gRPC v2 route handling for topic and assignment queries
The proxy now implements the gRPC v2 route activity, introducing new endpoints for querying topic routes and consumer group assignments. This change enables clients to retrieve message queue locations and broker assignments via the v2 protocol, including support for FIFO and lite topic modes in assignment logic, and ensures that permission checks are correctly mapped to the gRPC response format.
proxy/src/main/java/org/apache/rocketmq/proxy/grpc/v2/route · high confidence
New heartbeat management implementations for the controller
The controller now includes two new implementations for managing broker heartbeats: DefaultBrokerHeartbeatManager, which handles broker registration, liveness checks, and lifecycle notifications using local state, and RaftBrokerHeartBeatManager, which delegates heartbeat events to the JRaft controller for distributed consistency. Supporting data classes BrokerIdentityInfo and BrokerLiveInfo have been added to represent broker identity and live status respectively, enabling the controller to track broker epochs, offsets, and election priorities.
controller/src/main/java/org/apache/rocketmq/controller/impl/heartbeat · high confidence
New jRaft-based controller implementation added alongside DLedger
A new jRaft-based controller implementation (JRaftController) has been added to the controller module, providing an alternative to the existing DLedger-based controller. This change introduces new state machine and controller classes that leverage the jRaft library for consensus, including specific handling for broker registration, master election, and sync state set alterations via jRaft's RaftGroupService. The DLedgerController remains present, allowing users to choose between the two underlying consensus mechanisms for controller high availability.
controller/src/main/java/org/apache/rocketmq/controller/impl · high confidence
New metadata service abstraction for proxy configuration and ACL caching
The proxy now introduces a \MetadataService\ interface with two implementations: \LocalMetadataService\ for direct broker metadata access and \ClusterMetadataService\ for cluster-wide metadata retrieval with caching. \ClusterMetadataService\ adds Guava-based caching for topic configurations, subscription group configs, users, and ACLs, with configurable cache sizes, expiration, and refresh intervals. This enables the proxy to efficiently cache and retrieve metadata across the cluster, improving performance for metadata lookups.
proxy/src/main/java/org/apache/rocketmq/proxy/service/metadata · high confidence
New offset management components for Lite Topics and broadcast consumption
The broker now includes new classes in the offset package to support advanced consumption models. BroadcastOffsetStore provides in-memory offset tracking for broadcast consumption mode. LmqConsumerOffsetManager extends the standard offset manager to handle Light Message Queue (LMQ) specific offset storage and reset logic. MemoryConsumerOrderInfoManager introduces a lightweight, non-persistent manager for tracking consumer order information and queue suspension in Lite Topics, prioritizing low latency over persistence for these specific use cases.
broker/src/main/java/org/apache/rocketmq/broker/offset · high confidence
New operation-mode Producer and Consumer examples added
Added new example classes for the operation package that demonstrate how to send and receive messages using the DefaultMQProducer and DefaultMQPushConsumer. These examples include command-line argument parsing via Apache Commons CLI to configure group names, topics, tags, and message counts, allowing users to quickly test basic messaging scenarios.
example/src/main/java/org/apache/rocketmq/example/operation · high confidence
New pagecache transfer classes for message I/O
The broker's pagecache package now includes three new classes—ManyMessageTransfer, OneMessageTransfer, and QueryMessageTransfer—that implement Netty's FileRegion interface to handle zero-copy file transfers. These classes manage the transfer of message data (headers and payloads) from internal buffer results to network channels, supporting scenarios involving multiple messages, single messages, and query results.
broker/src/main/java/org/apache/rocketmq/broker/pagecache · high confidence
New proxy channel management and invocation context infrastructure
The proxy now introduces a dedicated channel management layer to handle client connections and internal invocations more robustly. A new ChannelManager centralizes the creation and lifecycle of SimpleChannel instances, using client identifiers to map and track active connections while automatically cleaning up inactive channels. Additionally, new InvocationContext and InvocationContextInterface classes provide a structured way to manage asynchronous remoting command responses, including expiration handling and response completion, replacing ad-hoc context management in the proxy's remoting processors.
proxy/src/main/java/org/apache/rocketmq/proxy/service/channel · high confidence
New proxy utility classes for tag filtering, gRPC metadata, and constants
Added three new utility classes to the proxy common module: FilterUtils provides a method to check if a message's tags match a consumer group's subscription data; GrpcUtils offers helpers to safely add or replace gRPC metadata headers and retrieve server call attributes; and ProxyUtils defines constants for maximum message numbers in pop requests and broker addresses.
proxy/src/main/java/org/apache/rocketmq/proxy/common/utils · high confidence
New remoting activity handlers for proxy messaging operations
The proxy now includes a new set of remoting activity classes in the \proxy/remoting/activity\ package to handle specific messaging requests. \AbstractRemotingActivity\ provides a base implementation for request processing, error handling, and response writing. Specific activities like \SendMessageActivity\, \PopMessageActivity\, \PullMessageActivity\, \AckMessageActivity\, \ChangeInvisibleTimeActivity\, \RecallMessageActivity\, and \TransactionActivity\ now handle their respective request codes, delegating to the \MessagingProcessor\. \ClientManagerActivity\ manages client registration, heartbeats, and unregistration, while \ConsumerManagerActivity\ handles consumer group operations like getting consumer lists and batch locking/unlocking. \GetTopicRouteActivity\ provides topic route information to clients.
proxy/src/main/java/org/apache/rocketmq/proxy/remoting/activity · high confidence
New simple example clients for RocketMQ features
Added a suite of Java example programs in the simple examples package to demonstrate core RocketMQ capabilities. These include AclClient for authenticated access, AsyncProducer and OnewayProducer for different send modes, PopConsumer for the POP consumption protocol, and multiple LitePullConsumer variants (LitePullConsumerAssign, LitePullConsumerAssignWithSubExpression, LitePullConsumerSubscribe) for flexible pull-based consumption. The set also includes legacy and specialized consumers like PullConsumer, PullScheduleService, and PushConsumer, alongside helper classes like CachedQueue and RandomAsyncCommit to support these examples.
example/src/main/java/org/apache/rocketmq/example/simple · high confidence
New timer benchmark examples for timing messages
Added TimerConsumer and TimerProducer example programs in the benchmark timer module to demonstrate and measure performance for timing messages with arbitrary time delays. The producer sends messages with specific delivery timestamps using the timer deliver property, while the consumer tracks and reports metrics such as consume TPS, average/percentile delayed durations, and send latency.
example/src/main/java/org/apache/rocketmq/example/benchmark/timer · high confidence
New utility for identifying channel types and protocols
A new ChannelHelper utility class has been added to the proxy's common channel package. This component provides static methods to determine if a network channel is remote (synced from another proxy) and to identify the specific protocol type (gRPC v2, Remoting, or other remote types) associated with a connection, enabling more precise channel handling within the proxy infrastructure.
proxy/src/main/java/org/apache/rocketmq/proxy/common/channel · high confidence
Proxy metrics framework with OpenTelemetry support
The proxy now exposes operational metrics via a new metrics framework built on OpenTelemetry. A new ProxyMetricsManager initializes and manages metric exporters (OTLP gRPC, Prometheus, or logging) based on configuration, while ProxyMetricsConstant defines standard labels and gauges such as proxy status. This enables users to monitor proxy health and performance through standard observability backends.
proxy/src/main/java/org/apache/rocketmq/proxy/metrics · high confidence
Proxy now synchronizes consumer registration state across the cluster
The proxy introduces a new heartbeat synchronization mechanism to keep consumer group information consistent across multiple proxy instances. A new \ClusterConsumerManager\ intercepts consumer register and unregister events and delegates them to a \HeartbeatSyncer\. This syncer broadcasts client connection details (such as client ID, language, version, and subscription data) to a dedicated broadcast topic, allowing other proxies in the cluster to receive and apply these changes. This ensures that consumer routing and load balancing decisions remain accurate even when clients connect to different proxy nodes.
proxy/src/main/java/org/apache/rocketmq/proxy/service/sysmessage · high confidence
SQL message filtering now supports CONTAINS, STARTSWITH, and ENDSWITH operators
The SQL92-based message selector parser in the filter module has been updated to recognize and process the CONTAINS, STARTSWITH, and ENDSWITH string operators. This change, implemented via the regenerated JavaCC parser files (SelectorParser.jj and its generated Java counterparts), allows users to write more expressive filter expressions that check for substring presence or string prefix/suffix matches, extending the previous capabilities which were limited to standard comparison and logical operators.
filter/src/main/java/org/apache/rocketmq/filter/parser · high confidence
Support for forwarding messages to dead-letter queues and recalling delay messages
The gRPC v2 proxy now exposes two new producer operations: forwarding messages to a dead-letter queue (DLQ) and recalling delay messages. Users can now send a message to the DLQ via the new ForwardMessageToDeadLetterQueue activity, which validates the topic and consumer group, retrieves the receipt handle, and forwards the request to the messaging processor. Additionally, the new RecallMessage activity allows users to recall a previously sent delay message by providing the topic and a recall handle, with the operation completing asynchronously and returning the recalled message ID.
proxy/src/main/java/org/apache/rocketmq/proxy/grpc/v2/producer · high confidence
Tiered store fetcher cache, topic filtering, and commit log file management
The tiered storage module introduces a new MessageStoreFetcherImpl that supports a dual-TTL read-ahead cache policy (separating creation and post-read expiration) to optimize memory usage and cache freshness, alongside a MessageStoreTopicFilter that explicitly excludes system topics, LMQs, and broker-specific topics from tiered storage indexing. Additionally, FlatCommitLogFile now implements improved rolling and cleanup logic, including resetting the minimum ConsumeQueue offset after expired files are destroyed to ensure data consistency.
rocketmq-tiered-store · high confidence
Transactional messages now use RocksDB for storage
The broker now persists transactional (half) messages using RocksDB instead of the previous storage mechanism. This change introduces a new service that manages transaction message lifecycle, including scanning for pending transaction statuses and checking them against configurable thread pools and immunity times, providing a more efficient and scalable backend for transactional message handling.
broker/src/main/java/org/apache/rocketmq/broker/transaction/rocksdb · high confidence
gRPC v2 consumer support for message acknowledgment and visibility management
The proxy now exposes gRPC v2 consumer operations for acknowledging messages and modifying their visibility duration. AckMessageActivity handles both batch and individual message acknowledgments, returning detailed per-message results and a combined status (including MULTIPLE\_RESULTS when outcomes vary). ChangeInvisibleDurationActivity allows clients to extend or shorten the time a message remains invisible, supporting Lite Topic and consumer suspension flags. ReceiveMessageResponseStreamWriter manages the streaming response for message retrieval, handling various pop statuses (FOUND, POLLING\_FULL, NO\_NEW\_MSG) and automatically adjusting visibility on write errors.
proxy/src/main/java/org/apache/rocketmq/proxy/grpc/v2/consumer · high confidence
Architecture
Proxy processor layer refactored into dedicated service components
The proxy's messaging logic has been reorganized into a set of specialized processor classes—ProducerProcessor, ConsumerProcessor, TransactionProcessor, ReceiptHandleProcessor, ClientProcessor, and RequestBrokerProcessor—managed by a new DefaultMessagingProcessor. This change introduces a unified MessagingProcessor interface and an AbstractProcessor base class to standardize access to the ServiceManager and handle lifecycle management. For users, this represents a structural shift in how the proxy routes and processes messages, acknowledging, and transactions, laying the groundwork for improved modularity and support for new features like Lite Topic and v2 protocol handling.
proxy/src/main/java/org/apache/rocketmq/proxy/processor · high confidence
Proxy service layer refactored into pluggable local and cluster modes
The proxy's service management has been restructured to support distinct local and cluster execution modes via a new \ServiceManager\ interface and \ServiceManagerFactory\. The \ClusterServiceManager\ now initializes dedicated \MQClientAPIFactory\ instances for messaging, operations, transactions, and lite subscriptions, while the \LocalServiceManager\ delegates to the embedded \BrokerController\ for local testing or embedded scenarios. This change introduces a \LiteSubscriptionService\ to handle subscription synchronization for the new Lite Topic model and centralizes lifecycle management through the factory, allowing the proxy to dynamically switch its underlying remoting and client infrastructure based on the deployment mode.
proxy/src/main/java/org/apache/rocketmq/proxy/service · high confidence
Remoting module refactored into new package structure with Bazel build support
The remoting module has been reorganized into a new package structure (org.apache.rocketmq.remoting) and is now buildable with Bazel. This change introduces new core interfaces for the remoting layer, including RemotingClient, RemotingServer, and RemotingService, along with supporting types like InvokeCallback, RPCHook, and Configuration. It also adds new exception classes (RemotingException, RemotingConnectException, etc.) and utility classes (RemotingHelper, ServiceThread) to support the refactored architecture.
remoting · high confidence
Behavioural changes
Batched broker unregistration for faster offline processing
The NameServer now processes broker unregistration requests in batches rather than individually. A new \BatchUnregistrationService\ runs as a background thread, collecting \UnRegisterBrokerRequestHeader\ messages into a queue and processing them together via \RouteInfoManager.unRegisterBroker\. This change speeds up the broker-offline process, reducing latency when multiple brokers disconnect or are deregistered simultaneously.
namesrv/src/main/java/org/apache/rocketmq/namesrv/routeinfo · high confidence
Broker initialization and configuration loading restructured
The broker startup process has been refactored to use a new ConfigContext object for passing configuration, and the BrokerController now initializes a comprehensive set of services including the new BrokerPreOnlineService for coordinated HA handshakes, a PID-based adaptive cold-read control strategy, and a v2 RocksDB-based configuration storage layer (ConfigHelper, TableId, SerializationType) alongside the existing v1 managers.
broker/src/main/java/org/apache/rocketmq/broker · high confidence
Configurable TLS certificate reload interval
The proxy now allows users to configure the interval at which TLS certificate and key files are monitored for changes. A new TlsCertificateManager component watches the configured certificate and key paths and triggers an SSL context reload only when both files have changed, using the configurable interval (getTlsCertWatchIntervalMs) to determine the polling frequency. This enables tuning of how frequently the proxy checks for certificate updates without requiring a restart.
proxy/src/main/java/org/apache/rocketmq/proxy/service/cert · high confidence
Controller introduces dedicated replica and sync-state management classes
The controller's broker metadata handling is refactored to use new internal classes: BrokerReplicaInfo, ReplicasInfoManager, and SyncStateInfo. BrokerReplicaInfo now tracks broker instances by ID rather than just IP, storing IP addresses and registration check codes. ReplicasInfoManager acts as the central state machine for managing these replicas and their sync state sets, enforcing epoch checks for master and sync state set updates to prevent stale configurations. SyncStateInfo manages the current master broker ID, master epoch, and sync state set epoch for each broker group, ensuring consistent leader election and failover logic within the controller.
controller/src/main/java/org/apache/rocketmq/controller/impl/manager · high confidence
Distribution packaging restructured with dledger configs and logback XMLs
The distribution module now includes example configuration files for dledger (Raft-based) broker nodes (broker-n0.conf, broker-n1.conf, broker-n2.conf) to support high-availability setups. Additionally, the release assembly (release.xml) has been updated to bundle specific logback XML configuration files (for broker, client, controller, namesrv, tools, and proxy) directly into the distribution's conf directory, ensuring consistent logging behavior out-of-the-box. The release process also explicitly excludes Jaeger tracing dependencies from the final package.
distribution · high confidence
Introduction of queue-based transactional message implementation
The broker now uses a new queue-based architecture for transactional messages, replacing the previous implementation. This change introduces a dedicated batch service (TransactionalOpBatchService) to handle operation message batching, a new listener (DefaultTransactionalMessageCheckListener) that moves half-messages exceeding the maximum check limit to the TRANS\_CHECK\_MAXTIME\_TOPIC system topic, and a refactored service layer (TransactionalMessageServiceImpl) with updated bridge logic for fetching and processing half and operation messages.
broker/src/main/java/org/apache/rocketmq/broker/transaction/queue · high confidence
NamesrvController refactoring and embedded controller support
The NameServer startup logic has been restructured to introduce a dedicated NamesrvController component that centralizes initialization, network setup, and scheduled tasks. This change adds support for an embedded controller mode: when the enableControllerInNamesrv configuration is enabled, the startup process now also initializes and starts a ControllerManager alongside the NameServer. Additionally, the command-line interface now uses the DefaultParser instead of the deprecated PosixParser, and the -p flag has been updated to print configuration properties for the embedded controller when that mode is active.
namesrv/src/main/java/org/apache/rocketmq/namesrv · high confidence
New gRPC interceptors and request mapping in the proxy
The proxy now includes a set of new gRPC server interceptors (ContextInterceptor, GlobalExceptionInterceptor, HeaderInterceptor) and a RequestMapping utility. ContextInterceptor propagates metadata into the gRPC context, while GlobalExceptionInterceptor provides centralized error handling and logging for gRPC calls. HeaderInterceptor extracts and injects headers such as authorization keys, remote/local addresses, and proxy protocol information into the gRPC metadata. RequestMapping defines the correspondence between gRPC v2 request types (e.g., SendMessage, ReceiveMessage, AckMessage) and internal RocketMQ request codes, enabling the proxy to route gRPC requests to the appropriate remoting handlers.
proxy/src/main/java/org/apache/rocketmq/proxy/grpc/interceptor · high confidence
New gRPC server implementation with Proxy Protocol and TLS hot-reload support
The proxy's gRPC server has been refactored into a new architecture centered on GrpcServer and GrpcServerBuilder, introducing support for the HAProxy Proxy Protocol to correctly expose client connection details (IP, port) behind load balancers. The new server also features dynamic TLS certificate hot-reloading via TlsCertificateManager, which reloads the SSL context without restarting the server, and exposes configurable gRPC server parameters such as max concurrent calls, keep-alive settings, and event loop thread counts.
proxy/src/main/java/org/apache/rocketmq/proxy/grpc · high confidence
New remoting channel abstraction for proxy client communication
The proxy now introduces a dedicated \RemotingChannel\ abstraction to manage client connections, replacing the previous direct handling of Netty channels. This change adds a \RemotingChannelManager\ to track producer and consumer channels by group, and a \RemotingConverter\ to handle message serialization. For users, this provides a more structured and robust way for the proxy to forward requests and manage the lifecycle of client connections, ensuring that operations like transaction checks and consumer info retrieval are handled through this new channel layer.
proxy/src/main/java/org/apache/rocketmq/proxy/remoting/channel · high confidence
Proxy client module refactoring and new consumer management
The proxy's client service layer has been refactored to introduce \ClusterConsumerManager\, which extends the standard consumer manager to synchronize heartbeat data with the broker via a \HeartbeatSyncer\ during consumer registration and unregistration. Additionally, a new \ProxyClientRemotingProcessor\ handles specific remoting requests, including transaction state checks and lite-topic unsubscribe notifications, by routing them to the appropriate producer or consumer channels.
proxy/src/main/java/org/apache/rocketmq/proxy/service/client · high confidence
Proxy transaction state management refactored to use in-memory storage
The proxy's transaction service implementation has been rewritten to store transaction state in memory rather than encoding it in the transaction ID. This change introduces a new \TransactionDataManager\ and associated data models (\TransactionData\, \EndTransactionRequestData\) to handle transaction lifecycle, including configurable expiration and isolation by producer group. For users, this improves transaction handling reliability within the proxy and aligns with the v2 transaction protocol support.
proxy/src/main/java/org/apache/rocketmq/proxy/service/transaction · high confidence
Proxy validates topic message type consistency
The proxy now enforces that the message type specified in a request matches the topic's configured type. A new validation layer (TopicMessageTypeValidator) checks the TopicMessageType attribute and throws a MESSAGE\_PROPERTY\_CONFLICT\_WITH\_TYPE error if there is a mismatch, with the exception of topics configured as MIXED which accept any type. This prevents users from sending messages with an incompatible type to a topic that requires a specific one.
proxy/src/main/java/org/apache/rocketmq/proxy/processor/validator · high confidence
Refactor gRPC v2 messaging layer with new activity interfaces and base classes
The proxy's gRPC v2 messaging implementation has been restructured to improve code organization and maintainability. A new \GrpcMessagingActivity\ interface defines the contract for messaging operations (such as sending, receiving, and acknowledging messages), while \AbstractMessagingActivity\ provides a base class with shared validation logic for topics, consumer groups, and invisible times. \DefaultGrpcMessagingActivity\ implements this interface, delegating specific operations to dedicated activity classes (e.g., \SendMessageActivity\, \ReceiveMessageActivity\) and managing shared resources like \GrpcClientSettingsManager\ and \GrpcChannelManager\. Additionally, a new \ContextStreamObserver\ interface ensures that gRPC stream callbacks consistently receive the \ProxyContext\, enabling better context propagation throughout the messaging pipeline.
proxy/src/main/java/org/apache/rocketmq/proxy/grpc/v2 · high confidence
Refactored NameServer request handling into dedicated processor classes
The NameServer's request processing logic has been reorganized from a monolithic structure into specific processor components. A new \ClientRequestProcessor\ now handles client route queries, introducing logic to reject requests if the NameServer is not yet ready (based on startup time and wait configuration) and supporting standard JSON responses for newer clients. A \ClusterTestRequestProcessor\ extends this to fetch route info from a product environment if local data is missing. The \DefaultRequestProcessor\ now centrally manages administrative and broker-related requests (such as registration, KV config, and topic deletion) via a request-code switch, replacing the previous inline handling. This change improves code modularity and adds readiness checks for client requests.
namesrv/src/main/java/org/apache/rocketmq/namesrv/processor · high confidence
Refactored Pop Lite orderly consumption with new management interface
The broker's Pop Lite orderly consumption logic has been refactored to support more flexible concurrency strategies. A new \ConsumerOrderInfoManager\ interface defines the contract for managing ordered consumption state, replacing the previous monolithic implementation. The existing queue-level ordering logic is now encapsulated in \QueueLevelConsumerManager\, which implements this interface to handle message status updates, blocking checks, and offset commits specifically for queue-level ordering. This change isolates the ordering mechanism, facilitating future expansion to support message-group-level ordering and custom strategies without altering the core broker flow.
broker/src/main/java/org/apache/rocketmq/broker/pop/orderly · high confidence
Refactored broker client connection management with optimized channel handling
The broker's client connection management has been restructured to improve concurrency and performance. A new \ClientChannelAttributeHelper\ now stores consumer and producer group associations directly on Netty \Channel\ attributes, enabling faster lookups during channel close events. The \ConsumerManager\ and \ProducerManager\ have been updated to leverage this helper when \enableFastChannelEventProcess\ is enabled, reducing the overhead of scanning global tables. Additionally, a new \ClientHousekeepingService\ runs a scheduled task to periodically scan for and clean up inactive client channels, ensuring timely resource reclamation. The \DefaultConsumerIdsChangeListener\ also introduces a batching mechanism for consumer change notifications, allowing for more efficient updates when real-time notification is disabled.
broker/src/main/java/org/apache/rocketmq/broker/client · high confidence
Refactored proxy route service with queue penalization and priority selection
The proxy's topic routing logic in the \route\ package has been restructured to support more flexible and fault-aware message queue selection. A new \AddressableMessageQueue\ class now wraps standard queues with explicit broker addresses, enabling the \ClusterTopicRouteService\ to correctly resolve broker locations in cluster mode. Queue selection is now driven by a \MessageQueuePenalizer\ system that applies penalty scores (aggregated from multiple sources like fault strategies) to rank queues, and a \MessageQueuePriorityProvider\ that groups queues by priority levels. The \MessageQueueView\ and \MessageQueueSelector\ components now utilize these mechanisms to select the least-penalty, highest-priority queue for reads and writes, while \ProxyTopicRouteData\ and \TopicRouteHelper\ provide supporting data structures and error handling for topic route resolution.
proxy/src/main/java/org/apache/rocketmq/proxy/service/route · high confidence
Fixes
Fix ACL 2.0 authorization logic and user status checks
This change introduces the ACL 2.0 authorization handler and fixes critical logic in user authorization. The new AclAuthorizationHandler ensures that DENY policies take precedence over ALLOW policies when sorting matching entries, correcting previous behavior where this priority was not enforced. Additionally, the UserAuthorizationHandler now correctly checks for disabled user status, throwing an exception when a user is disabled, which fixes a bug where disabled users could still be authorized.
auth/src/main/java/org/apache/rocketmq/auth/authorization/chain · high confidence
Test coverage
Added BaseServiceTest base class for proxy service tests; Added test certificates for password-encrypted private keys; Added test for InvocationChannel context handling; Added test for master election offset overflow handling; Added test resources for ACL and logging configuration; Added test resources for RMQ proxy configuration; Added tests for FAQUrl utility methods; Added tests for GenericMapSuperclassDeserializer with fastjson2; Added tests for HTTP/2 proxy protocol handlers; Added tests for RecallMessageHandle encoding and decoding; Added tests for RemotingProtocolServer and AuthorizationPipeline; Added tests for TlsCertificateManager; Added tests for gRPC proxy server lifecycle, TLS negotiation, and admin service; Added tests for proxy startup and command-line argument parsing; Added tests for topic and group validation logic; Added unit tests for Action and RocketMQAction; Added unit tests for BitsArray utility; Added unit tests for BrokerContainer lifecycle and configuration; Added unit tests for ClientConfig and Validators; Added unit tests for ClusterMessageService and LocalMessageService; Added unit tests for ClusterMetadataService; Added unit tests for ConfigHelper; Added unit tests for DefaultAdminService; Added unit tests for DefaultReceiptHandleManager; Added unit tests for FilterUtil tag matching and subscription data building; Added unit tests for GrpcClientChannel behavior; Added unit tests for LatencyFaultToleranceImpl; Added unit tests for LiteSubscription; Added unit tests for LocalProxyRelayService and ProxyChannel; Added unit tests for MQClientAPIExt and MQClientAPIFactory; Added unit tests for MQClientInstance; Added unit tests for MqClientAdminImpl; Added unit tests for NamespaceRpcHook; Added unit tests for OpenMessaging RocketMQ module; Added unit tests for RemoteChannel encoding and decoding; Added unit tests for RemotingChannel and RemotingChannelManager; Added unit tests for RocketMQ consumer implementations; Added unit tests for StatsItemSet and MomentStatsItemSet; Added unit tests for ThreadLocalIndex; Added unit tests for broker identity and heartbeat management; Added unit tests for client message tracing and OpenTracing integration; Added unit tests for client-side ACL common utilities; Added unit tests for common chain, cold counter, and consumer receipt handle components; Added unit tests for common message components; Added unit tests for common module components; Added unit tests for common utility classes; Added unit tests for compression and pull consumer system flags; Added unit tests for consumer implementation classes; Added unit tests for consumer implementation classes; Added unit tests for consumer offset storage components; Added unit tests for controller implementation components; Added unit tests for controller replica and heartbeat management; Added unit tests for gRPC ClientActivity; Added unit tests for gRPC proxy authentication and authorization pipelines; Added unit tests for gRPC v2 consumer activities; Added unit tests for gRPC v2 proxy messaging components; Added unit tests for gRPC v2 route activity; Added unit tests for message queue allocation strategies; Added unit tests for message queue selectors and producer implementation; Added unit tests for message reply utilities; Added unit tests for proxy client services; Added unit tests for proxy configuration and initialization; Added unit tests for proxy remoting activity handlers; Added unit tests for proxy system message handling; Added unit tests for the NameServer module; Added unit tests for the auth module's authentication and authorization components; Added unit tests for the common attribute module; Added unit tests for the gRPC EndTransaction activity; Added unit tests for the message filtering module; Expanded unit test coverage for broker core components; New Bazel build infrastructure and test harness for integration testing; Updated Mockito configuration for proxy tests.
Dependencies
Enable Bazel build system for the store module
The store module now supports building and testing via Bazel. A new BUILD.bazel file defines the Java library and test targets, explicitly listing dependencies such as fastjson2, Netty, Guava, RocksDB, and OpenTelemetry. The build configuration also includes a GenTestRules macro to manage test execution, specifically excluding known flaky or slow tests (AutoSwitchHATest, DLedgerCommitlogTest, DLedgerMultiPathTest) and classifying others as medium tests to optimize CI performance.
store · high confidence
Introduce Maven multi-module structure with dependency management
The project has been restructured into a Maven multi-module build, adding dedicated pom.xml files for core components including auth, broker, client, common, container, controller, distribution, example, filter, namesrv, openmessaging, and tools. This change centralizes dependency management in the root pom.xml and explicitly declares inter-module dependencies (e.g., broker depending on auth and tiered-store), ensuring consistent versioning and cleaner build isolation across the RocketMQ codebase.
(dependencies) · high confidence
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
How this codebase got here
Score
- CAI 58 → 58 (+0.4)
- Rubric changed (rubric-2026.10.5 → rubric-2026.11.3) — scores are not directly comparable.
Lenses
- Code Health 54 → 52 (-2.4)
- Architecture 90 → 90 (+0.2)
- Maturity 69 → 72 (+3.1)
- Readiness 56 → 58 (+2.5)
- Security 58 → 62 (+3.9)
- Performance 79 → 80 (+1.1)
Resolved (121)
- 7.run (cognitive 34) (tools/src/main/java/org/apache/rocketmq/tools/admin/DefaultMQAdminExtImpl.java)
- 7.run (cyclomatic 16) (tools/src/main/java/org/apache/rocketmq/tools/admin/DefaultMQAdminExtImpl.java)
- BrokerConsumeStatsSubCommand.execute (cognitive 29) (tools/src/main/java/org/apache/rocketmq/tools/command/broker/BrokerConsumeStatsSubCommand.java)
- ClassTooLong: DefaultMQAdminExt (tools/src/main/java/org/apache/rocketmq/tools/admin/DefaultMQAdminExt.java)
- ClassTooLong: DefaultMQAdminExtImpl (tools/src/main/java/org/apache/rocketmq/tools/admin/DefaultMQAdminExtImpl.java)
- ClusterListSubCommand.printClusterBaseInfo (cognitive 57) (tools/src/main/java/org/apache/rocketmq/tools/command/cluster/ClusterListSubCommand.java)
- ClusterListSubCommand.printClusterBaseInfo (cyclomatic 17) (tools/src/main/java/org/apache/rocketmq/tools/command/cluster/ClusterListSubCommand.java)
- ClusterListSubCommand.printClusterMoreStats (cognitive 18) (tools/src/main/java/org/apache/rocketmq/tools/command/cluster/ClusterListSubCommand.java)
- ClusterSendMsgRTCommand.execute (cognitive 37) (tools/src/main/java/org/apache/rocketmq/tools/command/cluster/ClusterSendMsgRTCommand.java)
- ClusterSendMsgRTCommand.execute (cyclomatic 17) (tools/src/main/java/org/apache/rocketmq/tools/command/cluster/ClusterSendMsgRTCommand.java)
- CommandUtil.fetchMasterAndSlaveDistinguish (cognitive 18) (tools/src/main/java/org/apache/rocketmq/tools/command/CommandUtil.java)
- ConsumeMessageCommand.execute (cognitive 22) (tools/src/main/java/org/apache/rocketmq/tools/command/message/ConsumeMessageCommand.java)
- ConsumeMessageCommand.execute (cyclomatic 19) (tools/src/main/java/org/apache/rocketmq/tools/command/message/ConsumeMessageCommand.java)
- ConsumerProgressSubCommand.execute (cognitive 65) (tools/src/main/java/org/apache/rocketmq/tools/command/consumer/ConsumerProgressSubCommand.java)
- ConsumerProgressSubCommand.execute (cyclomatic 24) (tools/src/main/java/org/apache/rocketmq/tools/command/consumer/ConsumerProgressSubCommand.java)
- ConsumerStatusSubCommand.execute (cognitive 35) (tools/src/main/java/org/apache/rocketmq/tools/command/consumer/ConsumerStatusSubCommand.java)
- ConsumerSubCommand.execute (cognitive 33) (tools/src/main/java/org/apache/rocketmq/tools/command/consumer/ConsumerSubCommand.java)
- CopyAclsSubCommand.execute (cognitive 21) (tools/src/main/java/org/apache/rocketmq/tools/command/auth/CopyAclsSubCommand.java)
- CopyUsersSubCommand.execute (cognitive 36) (tools/src/main/java/org/apache/rocketmq/tools/command/auth/CopyUsersSubCommand.java)
- DefaultMQAdminExtImpl.consumed (cognitive 16) (tools/src/main/java/org/apache/rocketmq/tools/admin/DefaultMQAdminExtImpl.java)
- …and 101 more
New (455)
- AbstractSendMessageProcessor.msgCheck (cyclomatic 21) (broker/src/main/java/org/apache/rocketmq/broker/processor/AbstractSendMessageProcessor.java)
- Duplicated block (11 lines × 3) (client/src/main/java/org/apache/rocketmq/client/impl/MQClientAPIImpl.java)
- Duplicated block (8 lines × 12) (tools/src/main/java/org/apache/rocketmq/tools/command/auth/CopyAclsSubCommand.java)
- Duplicated block (9 lines × 2) (tools/src/main/java/org/apache/rocketmq/tools/command/auth/CreateUserSubCommand.java)
- Edited copies of one member (3 members, 10+ corresponding lines) (broker/src/main/java/org/apache/rocketmq/broker/transaction/rocksdb/TransactionalMessageRocksDBService.java)
- Edited copies of one member (3 members, 10+ corresponding lines) (client/src/main/java/org/apache/rocketmq/acl/common/AclSigner.java)
- Edited copies of one member (3 members, 10+ corresponding lines) (filter/src/main/java/org/apache/rocketmq/filter/util/BitsArray.java)
- Edited copies of one member (3 members, 10+ corresponding lines) (openmessaging/src/main/java/io/openmessaging/rocketmq/consumer/PullConsumerImpl.java)
- Edited copies of one member (3 members, 10+ corresponding lines) (proxy/src/main/java/org/apache/rocketmq/proxy/service/metadata/ClusterMetadataService.java)
- Edited copies of one member (3 members, 10+ corresponding lines) (store/src/main/java/org/apache/rocketmq/store/stats/BrokerStatsManager.java)
- Edited copies of one member (3 members, 11+ corresponding lines) (broker/src/main/java/org/apache/rocketmq/broker/pagecache/ManyMessageTransfer.java)
- Edited copies of one member (3 members, 11+ corresponding lines) (client/src/main/java/org/apache/rocketmq/client/impl/mqclient/MQClientAPIExt.java)
- Edited copies of one member (3 members, 11+ corresponding lines) (common/src/main/java/org/apache/rocketmq/common/UtilAll.java)
- Edited copies of one member (3 members, 11+ corresponding lines) (store/src/main/java/org/apache/rocketmq/store/timer/TimerMessageStore.java)
- Edited copies of one member (3 members, 12+ corresponding lines) (broker/src/main/java/org/apache/rocketmq/broker/config/v1/RocksDBConsumerOffsetManager.java)
- Edited copies of one member (3 members, 12+ corresponding lines) (broker/src/main/java/org/apache/rocketmq/broker/config/v1/RocksDBSubscriptionGroupManager.java)
- Edited copies of one member (3 members, 12+ corresponding lines) (broker/src/main/java/org/apache/rocketmq/broker/loadbalance/MessageRequestModeManager.java)
- Edited copies of one member (3 members, 12+ corresponding lines) (broker/src/main/java/org/apache/rocketmq/broker/processor/PopBufferMergeService.java)
- Edited copies of one member (3 members, 12+ corresponding lines) (common/src/main/java/org/apache/rocketmq/common/stats/StatsItem.java)
- Edited copies of one member (3 members, 12+ corresponding lines) (store/src/main/java/org/apache/rocketmq/store/dledger/DLedgerCommitLog.java)
- …and 435 more
Changes since last survey
- 8 commits — 4 feature/other, 4 fixes
By area
- broker/src — 3 commits
- client/src — 1 commit
- common/src — 1 commit
- proxy/src — 1 commit
- store/src — 1 commit
- tools/src — 1 commit
Notable commits
- fix: [ISSUE #10966] fix(broker): only set firstTimestamp when mqContext is not empty (#11491)
- fix: [ISSUE #11041] Fix off-by-one in paginated metadata sync dropping the last entry (#11042)
- fix: fix(tools): stop trimming the ACL password in mqadmin create/updateUser (#11447)
- fix: fix: restrict client access to internal timer messages (#11206)
- change: [ISSUE #11466] Extract group max length into a field in TopicValidator (#11467)
- change: [ISSUE #11486] Prevent evicted LiteTopic clients from rejoining dispatch on full sync (#11487)
- change: [ISSUE #11493] Avoid retaining channels for orphan unregister (#11494)
- change: [ISSUE #11495] Disable inplace update for RocksDB consume queue offset column family (#11496)
Architecture
- Unchanged — 0 containers · 10 contexts · 21 edges
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
Survey your own repository
apache/rocketmq was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.
About this page
- The score is its most recent published measurement, taken on 13 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
- Measured at commit 164322b91e26fca2a53e6580003e582ff0621514 — the exact code this score is about.
- Scored under rubric-2026.11.3 — the same rubric and the same method as every other entry in this index.
- Measured by Canine Code Analyzer (cca.canine.dev) using cca measure-2026.11.61+preprod-f3449e6eb431.