kafka4beam/kafka_protocol
63.6
Adequate · 15 November 2026
4.8k
lines of production code
Erlang
primary language
5
measurements over time
What this system is
Features
Added local Kafka test environment and benchmarking scripts
The scripts directory now includes Docker Compose configurations for running local Kafka clusters, supporting both Zookeeper-based setups (for versions 0.9 through 2.x) and KRaft mode (for versions 3.x and 4.x). A new setup script automates the provisioning of these environments, including topic creation and SASL/SSL credential configuration. Additionally, a new Erlang benchmark module and shell script have been added to measure produce request encoding performance, and a helper script for running EUnit tests has been introduced.
scripts · high confidence
Initial release of Kafka protocol library for Erlang/Elixir
This entry introduces the \kafka\_protocol\ library, providing basic Kafka connection management, wire format encode/decode functions, and RPC primitives for Erlang and Elixir applications. The library now requires users to explicitly add compression dependencies (such as \snappyer\, \lz4b\, or \ezstd\) to their own project configurations, as these are no longer bundled as hard dependencies. It includes code generation from Kafka protocol schemas, support for SASL authentication, and compatibility with Kafka versions up to 4.0.0.
(repo-wide) · high confidence
Initial release of kafka\_protocol library
This change introduces the kafka\_protocol library for Erlang/Elixir, providing a complete implementation of the Kafka protocol. It includes core modules for connection management (kpro\_connection), broker discovery (kpro\_brokers), and batch encoding/decoding (kpro\_batch, kpro\_batch\_v01). The library supports multiple compression codecs (gzip, snappy, lz4, zstd) via a pluggable backend (kpro\_compress), handles SASL authentication (kpro\_auth\_backend), and exposes APIs for transactional operations, consumer group metadata decoding, and API version negotiation (kpro\_api\_vsn). The application structure is defined in kafka\_protocol.app.src, specifying dependencies on kernel, stdlib, ssl, and crc32cer.
src · high confidence
Behavioural changes
Generated Kafka protocol schema and code generator
The library now includes a code generator (priv/kpro\_gen.escript) that reads the Kafka protocol definition (priv/kafka.bnf) to produce the schema module (src/kpro\_schema.erl) and associated API key and error code mappings. This update introduces support for the full range of Kafka API keys and error codes, including newer features like transactional operations, ACLs, and delegation tokens, ensuring the client can correctly encode and decode requests and responses for these protocols.
priv · high confidence
Kafka protocol BNF generation tooling updated
The code generator for Kafka protocol BNF files has been replaced with a new Java application (KafkaProtocolBnf.java) that writes directly to the output file. This tool now explicitly handles Kafka's compact arrays by using curly brackets in the generated BNF syntax, aligning with the Kafka documentation, and includes logic to generate API keys while discarding broker-only APIs.
_priv/kafka\_protocol\bnf · high confidence
Refactored Erlang header files for Kafka protocol support
The library's include directory has been restructured into four distinct header files to better organize the Kafka client definitions. kpro.hrl now defines the core request and response records, including a new no\_ack field for fire-and-forget requests. kpro\_error\_codes.hrl provides a comprehensive, generated list of Kafka error codes. kpro\_private.hrl consolidates internal constants, adding support for zstd compression, SCRAM-SHA-256/512 SASL authentication mechanisms, and timestamp type handling. kpro\_public.hrl defines the kafka\_message record, extending it with timestamp and headers fields to support newer Kafka message formats.
include · high confidence
Test coverage
Added comprehensive test coverage for Kafka protocol features
Added new test suites covering API version negotiation, batch encoding (including large batches and multiple compression algorithms like zstd), connection handling (SSL, SASL, IPv6), fetch operations (including incremental fetch), consumer group lifecycle, produce requests (including pre-encoded batches and async send), list offsets (with leader epoch), and topic management. These tests ensure correct behavior across various Kafka versions and configuration scenarios.
test · high confidence
Dependencies
Add Kafka BNF generation tool with Kafka 2.4.0 dependency
A new Gradle-based Java application has been added to generate Backus-Naur Form (BNF) specifications for the Kafka protocol. This tool, located in priv/kafka\_protocol\_bnf, introduces a dependency on the Apache Kafka clients library version 2.4.0 and is configured to output the generated BNF file directly.
(dependencies) · high confidence
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
How this codebase got here
Score
- CAI 63 → 64 (+0.8)
- Rubric changed (rubric-2026.10.5 → rubric-2026.11.4) — scores are not directly comparable.
Lenses
- Code Health 69 → 68 (-0.6)
- Architecture 100 → 100 (-0.0)
- Maturity 53 → 53 (-0.7)
- Readiness 60 → 65 (+4.9)
- Security 87 → 87 (+0.0)
- Event Sourcing 100 → 100 (+0.0)
Resolved (2)
- No ADRs found
- Off-boarding risk: anonymized user #1
New (7)
- Edited copies of one member (5 members, 9+ corresponding lines) (src/kpro_txn_lib.erl)
- Edited copy of a member (10 corresponding lines) (src/kpro_connection.erl)
- Edited copy of a member (8 corresponding lines) (src/kpro_req_lib.erl)
- Further sole-owners (lower concentration)
- No ADRs found (priv/kafka_protocol_bnf/build.gradle)
- Off-boarding risk: anonymized user #1
- Repeated repair: src/kpro_connection.erl (src/kpro_connection.erl)
Changes since last survey
- 4 commits — 3 feature/other, 1 fixes
By area
- (repo) — 2 commits
- (root) — 2 commits
Notable commits
- fix: fix: restore request pipelining after SASL reauthentication
- change: Merge pull request #154 from terry-xiaoyu/codex/jmq-sasl-compatibility
- change: Merge pull request #155 from zmstone/261101-reauthentication-pipelining
- change: docs: prepare changelog for 4.3.8
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
Survey your own repository
kafka4beam/kafka_protocol was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.
About this page
- The score is its most recent published measurement, taken on 15 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
- Measured at commit f3d251665552c07434325e2c494bfc617fface99 — the exact code this score is about.
- Scored under rubric-2026.11.4 — the same rubric and the same method as every other entry in this index.
- Measured by Canine Code Analyzer (cca.canine.dev) using cca measure-2026.11.65+preprod-3c33c1882dc1.