qunarcorp/qmq

QMQ is a distributed message queue system that facilitates reliable message production, consumption, and delayed message scheduling. It provides a multi-language client ecosystem (Java, C++, C\#, Erlang) and a meta-server for broker discovery and routing. The architecture includes a backup service for querying message history via HBase and a delay server for managing scheduled tasks, all supported by a comprehensive set of CLI tools and monitoring utilities.

50 changes.

This period marks the initial implementation of the QMQ messaging system, establishing the core architecture across all modules. It introduces the foundational classes for the meta server, broker, delay server, and client, alongside the necessary networking, storage, and configuration infrastructure.

29 changes.

This period focused on expanding the QMQ ecosystem by introducing client libraries for C\#, Erlang, and C++, alongside a comprehensive backup service module. The backup module established a new RESTful API for querying message data from HBase, while the broker layer gained improved cluster and metadata management capabilities.

Introduced the HashedWheelTimer implementation and a WheelLoadCursor utility to support the delay message scheduling mechanism. The new code provides the core timing and cursor tracking logic required for the delay server's wheel-based scheduling, enabling the system to manage and expire delayed tasks.

qmq-delay-server/src/main/java/qunar/tc/qmq/delay/wheel · high confidence

The module now includes a new \PrometheusQmqGauge\ class that implements a Prometheus-compatible gauge metric, and registers the \PrometheusQmqMetricRegistry\ via the Java SPI mechanism. This enables the system to expose QMQ metrics in Prometheus format for HTTP-based monitoring.

qmq-metrics-prometheus · medium confidence

A new application.properties file is introduced for the qmq-demo module, providing default configuration for the QMQ client (appCode, metaServer), thread pool settings (executor core/max/queue size), and Spring data source/JPA settings (MySQL connection, Hibernate dialect, and DDL auto-update).

qmq-demo/src/main/resources · high confidence

A new QMQ demo application has been added to the codebase, providing a complete example of integrating Spring Boot with QMQ for transactional message publishing. The demo includes a Spring Boot entry point, configuration classes for the data source and JPA, and a REST controller to place orders. The core logic uses Spring's @Transactional to ensure the database save and the QMQ message send are handled together, with a consumer listening for 'order.changed' events.

qmq-demo/src/main/java · high confidence

A new command-line interface is introduced in the qmq-tools module, providing a suite of administrative commands for managing QMQ metadata. The Tools class acts as a CLI entry point using picocli, registering subcommands for broker management (AddBroker, ReplaceBroker, ListBrokers, ListBrokerGroups), subject routing (AddNewSubject, ExtendSubjectRoute, AddSubjectBrokerGroup, RemoveSubjectBrokerGroup), and broker state control (MarkReadonly, UnMarkReadonly, ResetOffset). The underlying MetaManagementService handles HTTP POST requests to the meta server using standard Java HttpURLConnection, replacing any previous async HTTP client implementations.

qmq-tools/src/main/java/qunar/tc/qmq/tools · high confidence

The distribution package now includes a comprehensive set of SQL scripts to initialize the database schema. This includes \init.sql\ which creates tables for broker groups, subject information, subject routing, client metadata, offline client states, broker nodes, leader election, datasource configuration, and read-only broker group settings. Additionally, separate scripts are provided for backup operations (\init\_backup.sql\ for a dictionary table, \init\_backup\_hbase.sql\ for HBase tables), and \init\_client.sql\ which creates a table for recording business system messages.

qmq-dist/sql · high confidence

The qmq-backup module introduces new classes to support message and action log backup and synchronization. This includes BackupKeyGenerator for constructing HBase row keys, BatchBackupManager for managing batch backups, and SlaveMetaSupplier for resolving server addresses. The implementation adds DbDicService and DbDicDao for database-backed dictionary lookups, alongside HBaseStoreFactory and related store implementations (AbstractHBaseStore, AbstractHBaseMessageStore) for HBase interactions. Additionally, BackupActionLogSyncProcessor and BackupMessageLogSyncProcessor handle the synchronization of action and message logs, while IndexFileStore and RocksDBStoreImpl provide file and key-value storage capabilities.

qmq-watchdog · high confidence

New classes were added to the qmq-common module to support broker cluster and group metadata, including BrokerCluster, BrokerGroup, BrokerGroupKind, and BrokerState, alongside a MetaServerLocator for discovering the active meta server endpoint. These changes introduce the data structures and discovery logic required for heartbeat and routing extensions.

qmq-common/src/main/java/qunar/tc/qmq/meta · high confidence

Introduced new request and response classes (BrokerAcquireMetaRequest/Response, BrokerRegisterRequest/Response) along with their serializers and the BrokerRegisterService, enabling brokers to register with the meta server, perform periodic heartbeats, and acquire cluster metadata.

qmq-server-common/src/main/java/qunar/tc/qmq/meta · high confidence

A new generic Result class has been introduced in the qmq-store module to encapsulate operation outcomes, pairing a status object with associated data. This provides a standardized way to handle store results, allowing callers to access both the status and the payload data through dedicated getter methods.

qmq-store/src/main/java/qunar/tc/qmq/store/result · high confidence

The delay server receiver now processes incoming messages through a chain of filters that validate message integrity and handle scheduling anomalies. A new \ReceiveFilterChain\ orchestrates three filters: \ValidateFilter\ ensures messages are not null and contain valid IDs and subjects; \OverDelayFilter\ detects messages scheduled more than two years in the future, logs a warning, and adjusts the schedule time to a maximum of two years; \PastDelayFilter\ detects messages that have passed their scheduled time, logs a metric, and updates the schedule time to the current time. This introduces a structured pipeline for pre-processing delay messages before they are invoked.

qmq-delay-server/src/main/java/qunar/tc/qmq/delay/receiver · high confidence

Introduced new data model classes in the metaserver's model package to support broker metadata (BrokerMeta), client information and offline state (ClientDbInfo, ClientMetaInfo, ClientOfflineState), and subject routing/management (SubjectInfo, SubjectRoute, ClientSubjectInfo, ReadonlyBrokerGroupSetting, GroupedConsumer). These classes define the structure for storing and retrieving metadata about brokers, clients, and message subjects within the MQ system.

qmq-metaserver/src/main/java/qunar/tc/qmq/meta/model · high confidence

A new QMon utility class is introduced in the delay server's monitor package to expose a comprehensive set of metrics for the delay server. This includes counters and timers for message processing times (produce, delay, load, send, process, and schedule log append), as well as counters for various failure and retry scenarios (e.g., netty send failures, batch executor failures, and hashed wheel timer errors).

qmq-delay-server/src/main/java/qunar/tc/qmq/delay/monitor · high confidence

A new template for placing orders has been added, providing a form where users can input an Order ID and a name to submit an order request.

qmq-demo/src/main/resources/templates · high confidence

A new Bootstrap class has been added to serve as the main entry point for the delay server. It initializes the ServerWrapper, starts the server, and registers a shutdown hook to ensure the server is properly destroyed when the application terminates.

qmq-delay-server/src/main/java/qunar/tc/qmq/delay/container · high confidence

Added a suite of utility classes in the qmq-common module to support message processing and system configuration. The new code includes Bytes for long-to-byte conversion, Checksums and Crc32 for data integrity checks, NetworkUtils for retrieving local hostnames and IP addresses (with support for environment variable overrides), SubjectUtils for validating message subjects, and RetrySubjectUtils for managing retry and dead-letter queue subject naming conventions. These utilities provide foundational helpers for the messaging system's core logic.

qmq-common/src/main/java/qunar/tc/qmq/utils · high confidence

The C++ client library named 'pitocin' has been added to the codebase. This new component provides a C++ implementation for interacting with the message broker, featuring a main Client class that handles publishing messages, managing HTTP-based metadata requests, and maintaining TCP connections for both meta and broker communication. The library includes supporting classes for connection management, protocol encoding, and request valve logic, along with a comprehensive set of unit tests for the new functionality.

clients/cpp/pitocin · high confidence

The backup service now exposes a set of RESTful HTTP endpoints to manage message backups. The new Bootstrap class initializes a Jetty server that registers servlets for dead messages (/api/message/dead), general messages (/api/message), message details (/api/message/detail), and message records (/api/message/records). This allows external systems to query and retrieve backup data through these specific API paths.

qmq-backup/src/main/java/qunar/tc/qmq/backup/container · high confidence

A new QMon utility class was added to the meta-monitor package, providing static methods to increment counters for broker registration and disconnection events, as well as client registration, subject routing, metadata refresh, and authentication failure metrics.

qmq-metaserver/src/main/java/qunar/tc/qmq/meta/monitor · high confidence

A new service class, ReadonlyBrokerGroupSettingService, has been introduced to handle the persistence of read-only broker group settings. This service provides methods to add and remove settings, wrapping the underlying store operations and handling specific database exceptions like duplicate keys or data access failures.

qmq-metaserver/src/main/java/qunar/tc/qmq/meta/service · high confidence

A new ServerWrapper class has been introduced in the qmq-delay-server module to orchestrate the startup, online/offline states, and graceful shutdown of the delay broker. This component initializes core services including the Netty server, message processors, and synchronization handlers, while also exposing online and offline methods to control the broker's health status and registration with the meta server.

qmq-delay-server/src/main/java/qunar/tc/qmq/delay/startup · high confidence

New utility classes were added to the backup module to support message backup and query operations. BackupMessageKeyBuilder and BackupMessageKeyRegexpBuilder generate HBase row keys and regular expressions for different message types (normal, retry, dead, and record), while HBaseValueDecoder handles the deserialization of backup messages and record queries. Additional helpers include DateTimeUtils for date conversions, Serializer for JSON handling, Tags for tag parsing, and KeyValueList for HBase result processing.

qmq-backup/src/main/java/qunar/tc/qmq/backup/util · high confidence

Introduced new classes for batch processing and high-performance concurrent queues. The \BatchExecutor\ enables processing items in configurable batches using a thread pool, while \MultipleQueueFlusher\ manages multiple queues with load balancing and scheduled flushing. Additionally, \MpscLinkedQueue\ and its supporting classes provide a lock-free, single-consumer multi-producer queue implementation for improved throughput in concurrent scenarios.

qmq-common/src/main/java/qunar/tc/qmq/batch · high confidence

Introduced new domain models for broker topology: BrokerClusterInfo and BrokerGroupInfo, which store cluster and group metadata including master/slave addresses and circuit breaker state. Added the BrokerService interface and its implementation (BrokerServiceImpl) to manage cluster information retrieval and refresh logic. Included a PollBrokerLoadBalance implementation that selects available broker groups using a round-robin and random fallback strategy. These changes provide the foundational components for broker discovery and routing.

qmq-common/src/main/java/qunar/tc/qmq/broker, qmq-remoting/src/main/java/qunar/tc/qmq/broker · high confidence

The qmq-common module now includes several new classes that support client identification and message routing. ClientInfo provides a serializable container for client metadata (app code, logical data center, and extensions). ClientType defines the roles (PRODUCER, CONSUMER, DELAY\_PRODUCER, OTHER) and includes helper methods to identify producer/consumer types. Disposable is a new interface for resource cleanup. MapKeyBuilder offers utility methods for constructing and parsing keys used in subscription and sender routing logic.

qmq-common/src/main/java/qunar/tc/qmq/common · high confidence

The backup service now includes dedicated components for handling dead messages and their content. A new \DeadMsgEventBusListener\ routes dead messages to \DeadMessageBatchBackup\ and \DeadMessageContentBatchBackup\ for storage. Additionally, \IndexEventBusListener\ and \ActionSyncLogIterator\ were added to handle index iteration and action log parsing, ensuring that dead message processing is integrated into the backup workflow.

qmq-backup/src/main/java/qunar/tc/qmq/backup/service/impl · high confidence

New visitor classes (DelayMessageLogVisitor, DispatchLogVisitor, ScheduleIndexVisitor) were added to the qmq-delay-server's store/visitor package. These components implement the LogVisitor interface to parse and iterate over log records from the delay message store and schedule index files, enabling the server to read and process delayed messages and their scheduling metadata.

qmq-delay-server/src/main/java/qunar/tc/qmq/delay/store/visitor · high confidence

Introduced a new set of interfaces in the \qunar.tc.qmq.meta.store\ package to manage broker, client, and subject routing metadata, including \BrokerStore\, \ClientDbConfigurationStore\, \ClientMetaInfoStore\, \ClientOfflineStore\, \ReadonlyBrokerGroupSettingStore\, and \Store\. These interfaces define the contract for storing and querying broker groups, client information, and subject routes. Additionally, a \Serializer\ utility class was added to handle JSON serialization and deserialization using Jackson, configuring the \ObjectMapper\ to support relaxed JSON parsing features such as single quotes and comments.

qmq-metaserver/src/main/java/qunar/tc/qmq/meta/store · high confidence

Added three new utility classes in the qmq-remoting module: ChannelUtil for managing channel attributes, RemoteHelper for parsing and closing remote channels, and RemotingBuilder for constructing request and response datagrams with protocol version 9 headers.

qmq-remoting/src/main/java/qunar/tc/qmq/util · high confidence

Added shell (.sh) and Windows (.cmd) scripts to start, stop, and manage QMQ services including the broker, meta server, delay server, tools, watchdog, and backup components. Each service now has dedicated launch scripts that handle environment setup, Java options, and process management, enabling users to easily start and stop these background services on both Unix and Windows systems.

qmq-dist/bin · high confidence

The backup module now exposes a new set of Java interfaces—DicStore, KvStore, MessageStore, RecordStore, and RocksDBStore—that define how message, record, and dictionary data are persisted and queried. This abstraction decouples the backup logic from specific storage backends, enabling the system to switch or extend storage implementations without changing the core backup workflow.

qmq-backup/src/main/java/qunar/tc/qmq/backup/store · high confidence

The meta server now includes a new client registration authentication mechanism. A \ClientRegisterAuthService\ interface and its default implementation (\DefaultClientRegisterAuthService\) have been added to the SPI layer, along with a \ClientRegisterAuthInfo\ data class. This allows the system to validate or authorize clients during the registration process, moving from an unauthenticated default to a pluggable auth model.

qmq-metaserver/src/main/java/qunar/tc/qmq/meta/spi · medium confidence

The qmq-gateway module now exposes an HTTP interface for sending and pulling messages. A new Bootstrap class starts a Jetty server on port 8080 (configurable via gateway.port) and registers two new servlets: SendServlet handles POST requests at /send/\* to send messages, and PullServlet handles GET requests at /pull/\* to pull messages. This introduces a new HTTP-based entry point for QMQ message operations.

qmq-gateway · high confidence

Added new HBase store implementations for message backup, including HBaseIndexStore, HBaseDeadMessageStore, HBaseDeadMessageContentStore, HBaseRecordStore, and HFile-based bulk load support for index and record data. The FactoryStoreImpl now dynamically creates store factories for HBase and MySQL (stub). Users can now query backed-up messages, dead messages, and action records from HBase, with support for optional consumer group filtering and subject-based indexing.

qmq-backup/src/main/java/qunar/tc/qmq/backup/store/impl · high confidence

Added new foundational classes for the QMQ messaging system, including BaseMessage for message data and attributes, MessageHeader for metadata, RawMessage for network payloads, RemoteMessageQuery for message retrieval, and supporting enums and utilities (ClientRequestType, OnOfflineState, LargeStringUtil). These classes establish the internal data structures and thread management (NamedThreadFactory) that underpin message handling and remote querying.

qmq-common/src/main/java/qunar/tc/qmq/base · high confidence

Added configuration files for various QMQ services, including backup, broker, datasource, delay, gateway, HBase, logback, meta server, valid API tokens, and watchdog. These files define connection endpoints, storage paths, port mappings, and operational parameters for the distributed message queue system.

qmq-dist/conf · high confidence

The C\# client for QMQ is introduced, providing the foundational components for message queue integration. This includes the solution file, core interfaces and classes for encoding/decoding messages, handling cluster connections, managing consumer groups, and defining exception types. The implementation supports netstandard2.0 and includes the necessary infrastructure for publishing and consuming messages within the QMQ ecosystem.

clients/csharp · high confidence

A new Bootstrap class has been added to the qmq-metaserver module to serve as the entry point for the Meta Server. This class configures and starts an embedded Jetty server, binding endpoints for meta address discovery, management, subject consumers, offline status, and slave server address supply. The server port is configurable via the metaserver.properties file, defaulting to 8080.

qmq-metaserver/src/main/java/qunar/tc/qmq/meta/startup · medium confidence

Added the ActorSystem class, which provides a concurrent actor model for processing messages. It manages a pool of actors, each handling a specific message path, and schedules tasks on a configurable thread pool. This introduces a new mechanism for handling asynchronous message processing within the QMQ server common library.

qmq-server-common/src/main/java/qunar/tc/qmq/concurrent · high confidence

Added a new Erlang client library (qmqec) that enables publishing messages and managing broker connections. The library includes core modules for protocol handling, HTTP/TCP meta-server interaction, and broker connection management. It provides a gen\_server-based API for starting connections, publishing messages with key-value pairs, and handling responses. The implementation includes a request table with timeout cleanup mechanisms to prevent memory leaks and supports concurrent reads for broker connection tables to improve efficiency.

clients/erlang · high confidence

Added a new \metainfoclient\ package containing \MetaInfoClient\, \MetaInfoService\, and related classes to handle broker cluster information and consumer state changes. This introduces a dedicated client for fetching and caching metadata about broker clusters and consumer groups, enabling the system to track online/offline states and route messages accordingly.

qmq-remoting/src/main/java/qunar/tc/qmq/metainfoclient · high confidence

Added new Netty-based networking components in the qmq-remoting module, including DecodeHandler and EncodeHandler for serializing and deserializing RemotingCommand messages, along with a NettyClientConfig for client-side settings, an AbstractNettyClient base class for managing connections, and a suite of custom exceptions (e.g., RemoteTimeoutException, BrokerRejectException) to handle remote communication failures.

qmq-remoting/src/main/java/qunar/tc/qmq/netty · high confidence

A new metrics package (qunar.tc.qmq.metrics) has been added to the qmq-common module, providing a unified interface for tracking application metrics. The change introduces a \Metrics\ facade that delegates to a \QmqMetricRegistry\ implementation loaded via Java's \ServiceLoader\, allowing the system to swap out or extend metric collection behavior. The package includes interfaces for counters, meters, and timers, along with a \MockRegistry\ for testing or fallback scenarios. This establishes the foundation for consistent metrics collection across the QMQ system.

qmq-common/src/main/java/qunar/tc/qmq/metrics · high confidence

Added QMon.java, a new utility class in the qmq-server-common module that provides a centralized API for recording message queue metrics. This includes counters and timers for operations such as message production, consumption, acknowledgment, and various error states (e.g., pull timeouts, invalid parameters, lost messages). The class wraps the underlying Metrics library to standardize how these monitoring data points are incremented and timed across the system.

qmq-server-common/src/main/java/qunar/tc/qmq/monitor · high confidence

Added ServerWrapper class to handle the startup sequence for the backup service, including configuration parsing, store initialization, and sync manager setup.

qmq-backup/src/main/java/qunar/tc/qmq/backup/startup · high confidence

A new \BrokerConstants\ class has been added to define configuration keys and default values for broker settings. This includes enabling and managing Sorted Messages Table (SMT) via \smt.enable\ and \smt.retention.hours\, as well as configuring memory table limits (\memtable.max\_reserved\, \memtable.max\_active\) and log retention policies. These constants provide the foundational configuration options for these new capabilities.

qmq-server-common/src/main/java/qunar/tc/qmq/constants · high confidence

Added a new set of Java interfaces to the qmq-api module, establishing the foundational API for message production and consumption. This includes \MessageProducer\ and \MessageConsumer\ for sending and receiving messages, \MessageListener\ for handling incoming messages, and \PullConsumer\ for explicit pull-based consumption. The update also introduces support for transactional messaging via \TransactionListener\ and \TransactionProvider\, enables idempotency checks through \IdempotentChecker\ and \IdempotentAttachable\, and adds filtering capabilities via the \Filter\ interface. Additionally, new types such as \SubscribeParam\ and \TagType\ are introduced to configure subscription behaviors and tag matching logic.

qmq-api · high confidence

Added new configuration and store components for the delay server: a \StoreConfiguration\ interface and its \DefaultStoreConfiguration\ implementation, which define settings for message, schedule, and dispatch logs, as well as checkpoint and segment parameters. Additionally, added an \IterateOffsetManager\ to manage and persist the message log iterate offset via periodic flushing, and a \ScheduleLogValidatorSupport\ to save and load schedule offset checkpoints. These changes provide the underlying storage and configuration infrastructure for the delay server's message processing.

qmq-delay-server/src/main/java/qunar/tc/qmq/delay/config, qmq-delay-server/src/main/java/qunar/tc/qmq/delay/store · high confidence

Added a new dynamic configuration system in the qmq-common module, featuring a \DynamicConfig\ interface and a \DynamicConfigFactory\ for pluggable implementations. The default \LocalDynamicConfig\ implementation loads configuration from local files (using the \qmq.conf\ system property or classpath resources) and supports hot-reloading via a background \ConfigWatcher\ that detects file changes and triggers registered \Listener\ callbacks. The loader uses Java's \ServiceLoader\ to find a \DynamicConfigFactory\ implementation, falling back to the local file-based factory if none is provided.

qmq-common/src/main/java/qunar/tc/qmq/configuration · high confidence

The qmq-metaserver module adds a new cache layer and management infrastructure. This includes \AliveClientManager\ for tracking and cleaning up client metadata, \BrokerMetaManager\ for managing broker group information and slave addresses, and \CachedMetaInfoManager\ for caching subject and broker group data. Additionally, \CacheManager\ and \CacheManagerFactory\ are introduced to coordinate these components, while \CachedOfflineStateManager\ handles client offline states. The change also adds web servlets (\MetaManagementServlet\, \OnOfflineServlet\, \SubjectConsumerServlet\, etc.) to expose these capabilities via HTTP endpoints, and includes supporting classes like \JsonResult\ and exception types.

qmq-metaserver/src/main/java/qunar/tc/qmq/meta/cache · high confidence

The codebase introduces new exception classes, specifically BlockMessageException and DuplicateMessageException, to handle specific message processing states. Additionally, the protocol layer adds SendResult and MessageProducerCode to standardize message sending outcomes and error codes. The change also introduces base request and response classes (ActionLogOffsetRequest/Response, ConsumeManageRequest, SyncRequest) and a ConsumerLag model, alongside utility and management classes (ClientLogUtils, SubjectUtils, MetaManagementAction, etc.) that support the messaging system's internal operations and metadata management.

(repo-wide) · high confidence

Added a new set of Java classes in the \qunar.tc.qmq.delay.store.model\ package to support the delay server's internal storage and dispatch logic. This includes the \RecordResult\ interface and its implementations (\AppendDispatchRecordResult\, \AppendMessageRecordResult\, \AppendScheduleLogRecordResult\, \NopeRecordResult\), as well as data transfer objects like \LogRecord\, \LogRecordHeader\, \MessageLogRecord\, \DispatchLogRecord\, \AppendLogResult\, \AppendRecordResult\, \RawMessageExtend\, and \ScheduleSetSequence\. These classes provide the data structures required for appending and retrieving message and schedule logs within the delay server.

qmq-delay-server/src/main/java/qunar/tc/qmq/delay/store/model · high confidence

The backup module now uses a Service Provider Interface (SPI) to manage dead message handling. A new \DeadMessageHandlerFactory\ loads the active \DeadMessageSpiHandler\ implementation via \ServiceLoader\, falling back to a default \DefaultDeadMessageSpiHandler\ if no provider is found. This allows the dead message backup behavior to be extended or replaced by external implementations.

qmq-backup/src/main/java/qunar/tc/qmq/backup/spi · high confidence

The backup module now includes a set of new base classes and interfaces that support querying message and record data. This includes the \ActionEnum\ and \RecordEnum\ to define action and record types, along with query parameter classes (\BackupQuery\, \RecordQuery\) and result wrappers (\MessageQueryResult\, \RecordQueryResult\). Additionally, the \LogSyncDispatcher\ interface is added to handle log synchronization dispatching. These changes provide the foundational data structures and interfaces required for the new backup query functionality.

qmq-backup/src/main/java/qunar/tc/qmq/backup/base · high confidence

The delay server now includes new base-level components: custom exception classes (AppendException, GroupSendException), a high-performance primitive set (LongHashSet), a result wrapper (ReceivedResult), and an extended segment buffer (SegmentBufferExtend). These additions provide foundational building blocks for message handling, buffering, and error management within the delay server module.

qmq-delay-server/src/main/java/qunar/tc/qmq/delay/base · high confidence

The \qmq-remoting\ module adds a new set of protocol classes to handle message serialization and remoting. This includes \CommandCode\ for defining request and response codes, \RemotingHeader\ for managing header fields like magic code, version, and opaque identifiers, and \RemotingCommand\ to distinguish between request and response commands. Additionally, \QMQSerializer\ provides methods to serialize and deserialize messages, headers, and maps, while \Datagram\ and \PayloadHolder\ support the underlying data transfer and payload handling.

qmq-remoting/src/main/java/qunar/tc/qmq/protocol · high confidence

The backup module adds several new service interfaces and a manager class to support message and record backup operations. Specifically, it introduces the BatchBackup interface for batch processing, the DicService interface for dictionary lookups, the FileStore interface for scheduled flushing, and the MessageService interface for asynchronous message and record queries. Additionally, the ScheduleFlushManager is added to coordinate scheduled flushes across multiple flushable components, and the SyncLogIterator interface is added to handle log iteration. These changes establish the core abstractions for the backup functionality.

qmq-backup/src/main/java/qunar/tc/qmq/backup/service · high confidence

The store module adds a new set of classes to manage checkpoints and action logs, including \Action\, \ActionCheckpoint\, \ActionCheckpointSerde\, \ActionLogVisitor\, \ActionReaderWriter\, \ActionType\, \AppendMessageResult\, \AppendMessageStatus\, \BuildMessageMemTableEventListener\, \CheckpointLoader\, \CheckpointManager\, \CheckpointStore\, \ConsumeFromWhere\, \ConsumeQueue\, \ConsumeQueueManager\, \ConsumerGroupProgress\, \ConsumerLogEntry\, \ConsumerLogFlusher\, \ConsumerLogWroteEvent\, and \ConsumerProgress\. These files implement the logic for serializing and deserializing consumer and action checkpoints, managing consumer group progress, handling message append results, and flushing consumer logs to disk, effectively establishing the foundation for tracking consumer offsets and action states in the storage layer.

qmq-store/src/main/java/qunar/tc/qmq/store · high confidence

The server now includes a new set of classes to manage consumer state and message delivery sequences. This includes \ConsumerGroup\, \ConsumerSequence\, and \ConsumerSequenceManager\ to track pull and acknowledgment progress. Additionally, \Subscriber\, \SubscriberStatusChecker\, \OfflineTask\, and \RetryTask\ are introduced to handle consumer heartbeat monitoring, automatic offline detection, and message retry logic. These components work together to ensure reliable message delivery and proper cleanup of inactive consumers.

qmq-server · high confidence

A new 'Buffer' interface has been added to the qmq-store module, defining a contract for segment buffers with methods to retrieve the underlying ByteBuffer, get the size, and manage reference counting via retain and release operations. This change introduces a new abstraction for buffer management within the store's buffer package.

qmq-store/src/main/java/qunar/tc/qmq/store/buffer · high confidence

Added ActionEvent, PullLogBuilder, and several new action types (PullAction, RangeAckAction) with their corresponding reader/writer implementations (PullActionReaderWriter, RangeAckActionReaderWriter, ForeverOfflineActionReaderWriter). These classes handle the serialization and processing of pull and acknowledgment actions, enabling the system to build and manage pull logs in memory.

qmq-store/src/main/java/qunar/tc/qmq/store/action · high confidence

The qmq-backup module introduces a new set of RESTful API endpoints to support asynchronous querying of message data. This includes an abstract base servlet (AbstractGetServlet) and specific servlets for retrieving dead messages (DeadMessageApiServlet), standard messages (MessageApiServlet), message details (MessageDetailsServlet), and message records (MessageRecordsServlet). These endpoints accept query parameters, process them asynchronously via CompletableFuture, and return JSON responses, enabling users to query backup message data through the QMQ backup service.

qmq-backup/src/main/java/qunar/tc/qmq/backup/api · high confidence

The QMQ client introduces a new \@EnableQmq\ annotation and \@QmqConsumer\ annotation to simplify consumer registration in Spring-based applications. The \ConsumerAnnotationScanner\ automatically discovers methods marked with \@QmqConsumer\ and registers them as message listeners. Additionally, the client now supports environment isolation via a new \EnvProvider\ interface and its implementations, allowing consumers to be bound to specific producer environments. A new \AtomicConfig\ and \PullSubjectsConfig\ system enables dynamic, hot-reloadable configuration for pull settings such as batch size, timeouts, and retry limits. The client also gains a \ClientIdProvider\ for generating unique client identifiers and a \SwitchWaiter\ for managing online/offline states.

qmq-client/src/main · high confidence

The qmq-tools utility now includes a suite of new CLI commands for managing message queue infrastructure, including AddBroker, AddDb, AddNewSubject, AddSubjectBrokerGroup, ExtendSubjectRoute, ListBrokerGroups, ListBrokers, ListSubjectRoutes, MarkReadonly, RemoveSubjectBrokerGroup, ReplaceBroker, ResetOffset, and UnMarkReadonly. These commands allow users to add and remove broker groups and subjects, extend subject routing, list broker and route configurations, reset consumer offsets, and toggle broker read-only status via the MetaManagementService.

qmq-tools/src/main/java/qunar/tc/qmq/tools/command · high confidence

Added new Java classes in the qmq-remoting module to support consumer-side pull requests and metadata exchange. This includes request and response models (PullRequest, AckRequest, MetaInfoRequest, MetaInfoResponse) and a serialization/deserialization handler (PullRequestSerde) that handles network encoding for pull operations. The changes also introduce a PullFilter interface and PullFilterType enum to support filtering criteria such as TAG and SUB\_ENV\_ISOLATION, enabling more granular control over message pulling behavior.

qmq-remoting/src/main/java/qunar/tc/qmq/protocol/consumer · high confidence

The delay server introduces a new log-based storage layer for managing scheduled message dispatches and schedule sets. This change adds a suite of new classes—including AbstractDelayLog, AbstractDelaySegment, and their concrete implementations (DispatchLog, ScheduleSet)—to handle appending, flushing, and cleaning of delay message logs. The implementation utilizes file channels for persistent storage and includes utilities for segment management and offset resolution, enabling the server to store and retrieve scheduled message metadata and dispatch logs more efficiently.

qmq-delay-server/src/main/java/qunar/tc/qmq/delay/store/log · high confidence

The delay server introduces new core interfaces and classes to handle delayed message storage, scheduling, and replication. A new \DelayLogFacade\ interface defines the API for managing message and dispatch logs, including methods for appending, retrieving, and syncing log segments. Supporting types include \ScheduleIndex\ for tracking scheduled message metadata, \EventListener\ for event posting, and \Switchable\ for lifecycle management. Additionally, the \AbstractLogSyncWorker\ and \SyncLogProcessor\ interfaces establish the framework for master/slave log synchronization, enabling the server to replicate delay logs between nodes.

qmq-delay-server/src/main/java/qunar/tc/qmq/delay · high confidence

Added new classes (DelayProcessor, Sender, SenderExecutor, SenderGroup, SenderProcessor) that handle the sending of delayed messages. The implementation includes a batch processing mechanism with configurable thread pools, automatic retry logic for failed sends, and broker group management for routing messages to the correct brokers.

qmq-delay-server/src/main/java/qunar/tc/qmq/delay/sender · high confidence

Added the qmq-sync module containing the foundational classes for message synchronization: the SyncLogProcessor interface and its AbstractSyncLogProcessor implementation for handling log data, the SyncType enumeration for different sync categories, the DelaySyncRequest class for managing delayed sync requests, and the SlaveSyncSender for executing remote sync calls.

qmq-sync · high confidence

The qmq-metaserver module introduces a new set of classes in the \qunar.tc.qmq.meta.route\ package to handle message routing and broker group management. This includes the \ReadonlyBrokerGroupManager\ to filter out read-only brokers for specific client types, and a routing extension framework (\SubjectRouteExtend\, \SubjectRouteExtendFactory\) that allows pluggable route extensions via Java's ServiceLoader mechanism. Additionally, new service interfaces (\SubjectConsumerService\, \SubjectRegisterService\, \ExtendService\) and the main \SubjectRouter\ interface are added to support these routing capabilities.

qmq-metaserver/src/main/java/qunar/tc/qmq/meta/route · high confidence

The backup system now exposes a new configuration interface (BackupConfig) and its default implementation (DefaultBackupConfig) that define constants and dynamic configuration keys for HBase tables (such as qmq\_backup, qmq\_backup\_delay, qmq\_backup\_record, qmq\_backup\_dead, and qmq\_backup\_dead\_content), as well as parameters for message backup (batch size, retry count, flush interval), dead message handling, RocksDB TTL, and store factory type. This allows users to adjust backup behavior, table names, and storage settings through dynamic configuration rather than hard-coded values.

qmq-backup/src/main/java/qunar/tc/qmq/backup/config · high confidence

The message broker's routing logic has been updated to ensure consumers receive default route assignments, addressing a previous issue where routing errors could occur. This change introduces a new \DefaultSubjectRouter\ implementation that handles subject-to-broker group mapping, including logic to assign new brokers or re-assign existing ones based on configuration. Additionally, a \SubjectRouterWrapper\ has been added to provide an extension point for routing, allowing for future customization of the routing behavior through the \SubjectRouteExtendFactory\.

qmq-metaserver/src/main/java/qunar/tc/qmq/meta/route/impl · medium confidence

The qmq-server-common module now provides a Service Provider Interface (SPI) for database access, allowing the default HikariCP-based DataSource implementation to be swapped or extended. The new JdbcTemplateHolder uses ServiceLoader to discover DataSourceFactory implementations, falling back to the default HikariCP configuration if no custom factory is found. This enables users to inject alternative DataSource configurations without modifying core code.

qmq-server-common/src/main/java/qunar/tc/qmq/jdbc · high confidence

Added test utilities and example classes for message production and consumption; Added tests for bulk load and API endpoints; Added unit tests for LongHashSet; Added unit tests for the C++ client library; Added unit tests for utility classes.

The QMQ project's Maven build structure has been established with a parent POM (qmq-parent) defining a unified version (1.1.44-SNAPSHOT) and centralized dependency management for libraries such as Netty 4.0.46, Jackson 2.9.9, Jetty 9.4.17, and Spring 4.2.3. The diff introduces or updates POM files for all modules (client, server, store, etc.) and includes a new C\# client library (qmqclient) targeting .NET Standard 2.0.

(dependencies) · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• CAI 52 → 54 (+2.4)
• Rubric changed (rubric-2026.08.19 → rubric-2026.09.15) — scores are not directly comparable.

• Code Health 76 → 71 (-4.4)
• Architecture 79 → 79 (+0.0)
• Maturity 54 → 54 (+0.0)
• Readiness 38 → 41 (+3.0)
• Security 65 → 78 (+12.5)
• Event Sourcing 100 → 100 (+0.0)

• Coverage not included — suite not readable by the collector
• Dependency hygiene not measured — no packages were read
• Duplicated block (10 lines × 2) (qmq-delay-server/src/main/java/qunar/tc/qmq/delay/store/log/DispatchLogSegmentContainer.java)
• Duplicated block (10 lines × 2) (qmq-store/src/main/java/qunar/tc/qmq/store/DefaultStorage.java)
• Duplicated block (11 lines × 2) (qmq-backup/src/main/java/qunar/tc/qmq/backup/service/impl/DeadMessageBatchBackup.java)
• Duplicated block (11 lines × 2) (qmq-backup/src/main/java/qunar/tc/qmq/backup/service/impl/MessageServiceImpl.java)
• Duplicated block (11 lines × 2) (qmq-backup/src/main/java/qunar/tc/qmq/backup/service/impl/MessageServiceImpl.java)
• Duplicated block (11 lines × 2) (qmq-backup/src/main/java/qunar/tc/qmq/backup/store/impl/AbstractHBaseMessageStore.java)
• Duplicated block (11 lines × 2) (qmq-delay-server/src/main/java/qunar/tc/qmq/delay/receiver/ReceivedDelayMessageProcessor.java)
• Duplicated block (11 lines × 2) (qmq-delay-server/src/main/java/qunar/tc/qmq/delay/store/log/DispatchLogSegmentContainer.java)
• Duplicated block (11 lines × 2) (qmq-delay-server/src/main/java/qunar/tc/qmq/delay/store/log/MessageSegmentContainer.java)
• Duplicated block (11 lines × 2) (qmq-store/src/main/java/qunar/tc/qmq/store/ActionLog.java)
• Duplicated block (11 lines × 2) (qmq-store/src/main/java/qunar/tc/qmq/store/IndexLogVisitor.java)
• Duplicated block (12 lines × 2) (qmq-delay-server/src/main/java/qunar/tc/qmq/delay/store/visitor/DelayMessageLogVisitor.java)
• Duplicated block (12 lines × 2) (qmq-delay-server/src/main/java/qunar/tc/qmq/delay/store/visitor/DelayMessageLogVisitor.java)
• Duplicated block (5 lines × 2) (qmq-backup/src/main/java/qunar/tc/qmq/backup/service/impl/DeadMessageBatchBackup.java)
• Duplicated block (5 lines × 2) (qmq-backup/src/main/java/qunar/tc/qmq/backup/service/impl/RecordBatchBackup.java)
• Duplicated block (5 lines × 2) (qmq-backup/src/main/java/qunar/tc/qmq/backup/store/impl/AbstractHBaseMessageStore.java)
• Duplicated block (5 lines × 2) (qmq-backup/src/main/java/qunar/tc/qmq/backup/store/impl/HFileIndexStore.java)
• Duplicated block (5 lines × 2) (qmq-client/src/main/java/qunar/tc/qmq/consumer/pull/PullService.java)
• …and 44 more

• ClassTooLong: DefaultStorage (qmq-store/src/main/java/qunar/tc/qmq/store/DefaultStorage.java)
• Coverage not measured — no coverage collector is wired up
• Documentation: contradicts the code (README.md)
• Documentation: no architecture or design documentation (docs/cn/exactlyonce.md)
• Documentation: no architecture or design documentation (docs/cn/install.md)
• Documentation: no contributor guidance (README.md)
• Documentation: no installation or build instructions (README.md)
• Documentation: no usage examples (README.md)
• Documentation: written for insiders (docs/cn/local.md)
• Duplicated block (10 lines × 2) (qmq-delay-server/src/main/java/qunar/tc/qmq/delay/store/log/DispatchLogSegmentContainer.java)
• Duplicated block (10 lines × 2) (qmq-gateway/src/main/java/qunar/tc/qmq/gateway/servlet/SendServlet.java)
• Duplicated block (10 lines × 2) (qmq-store/src/main/java/qunar/tc/qmq/store/ActionLogVisitor.java)
• Duplicated block (10 lines × 2) (qmq-store/src/main/java/qunar/tc/qmq/store/ConsumerLogFlusher.java)
• Duplicated block (10 lines × 2) (qmq-store/src/main/java/qunar/tc/qmq/store/ConsumerLogFlusher.java)
• Duplicated block (11 lines × 2) (qmq-client/src/main/java/qunar/tc/qmq/consumer/pull/DefaultPullEntry.java)
• Duplicated block (11 lines × 2) (qmq-delay-server/src/main/java/qunar/tc/qmq/delay/store/log/DispatchLogSegmentContainer.java)
• Duplicated block (11 lines × 2) (qmq-delay-server/src/main/java/qunar/tc/qmq/delay/store/visitor/DelayMessageLogVisitor.java)
• Duplicated block (11 lines × 2) (qmq-store/src/main/java/qunar/tc/qmq/store/IndexLogVisitor.java)
• Duplicated block (11 lines × 2) (qmq-tools/src/main/java/qunar/tc/qmq/tools/command/AddBrokerCommand.java)
• Duplicated block (11 lines × 2) (qmq-tools/src/main/java/qunar/tc/qmq/tools/command/ListBrokerGroupsCommand.java)
• …and 88 more

• Containers 0 added · 0 removed · contexts 12 added · 0 removed · edges 31 added · 0 removed

• qmq-backup
• qmq-client
• qmq-common
• qmq-delay-server
• qmq-demo
• qmq-metaserver
• qmq-metrics-prometheus
• qmq-remoting
• qmq-server
• qmq-server-common
• qmq-store
• qmq-sync

• qmq-backup → qmq-common
• qmq-backup → qmq-remoting
• qmq-backup → qmq-server-common
• qmq-backup → qmq-store
• qmq-backup → qmq-sync
• qmq-client → qmq-common
• qmq-client → qmq-remoting
• qmq-delay-server → qmq-common
• qmq-delay-server → qmq-remoting
• qmq-delay-server → qmq-server-common
• qmq-delay-server → qmq-store
• qmq-delay-server → qmq-sync
• qmq-metaserver → qmq-common
• qmq-metaserver → qmq-remoting
• qmq-metaserver → qmq-server-common
• qmq-metrics-prometheus → qmq-common
• qmq-remoting → qmq-common
• qmq-server → qmq-common
• qmq-server → qmq-remoting
• qmq-server → qmq-server-common
• …and 11 more

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

qunarcorp/qmq was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.