zio/zio-kafka
51.1
Weak · 6 November 2026
8.8k
lines of production code
Scala
primary language
4
measurements over time
What this system is
Features
Add Kafka client and ZIO-Kafka comparison benchmarks
Added new JMH benchmarks in the \comparison\ package to measure and compare the performance of the native Apache Kafka Java client against ZIO-Kafka. The suite includes \KafkaClientBenchmarks\ for testing both auto-subscription and manual partition assignment with the native client, and \ZioKafkaBenchmarks\ for testing ZIO-Kafka's automatic and manual subscription modes, providing a baseline for evaluating consumer throughput and latency differences.
zio-kafka-bench/src/main/scala/zio/kafka/bench/comparison · high confidence
Add minimal example application for manual testing
A new example application has been added to the zio-kafka-example module, providing a minimal, runnable consumer setup (Main.scala) and additional examples for producer/consumer interactions (ReadmeExample.scala) and transactions (Transactional.scala). This includes an embedded Kafka service layer (MyKafka.scala) and a Logback configuration (logback.xml) to facilitate manual testing and verification of library features.
zio-kafka-example/src/main · high confidence
Added ACL data model types for Kafka AdminClient
The zio-kafka admin module now includes Scala case classes and companion objects for AccessControlEntry, AccessControlEntryFilter, ResourcePattern, and ResourcePatternFilter. These types provide bidirectional conversion to and from the corresponding Java Kafka client classes, establishing the foundational data structures required for managing Access Control Lists (ACLs) within the AdminClient.
zio-kafka/src/main/scala/zio/kafka/admin · high confidence
Added ZIO-based JMH benchmarks for Kafka producer and consumer performance
New JMH benchmark suites have been added to measure the performance of zio-kafka's producer and consumer components. The consumer benchmarks (\ZioKafkaConsumerBenchmark\) evaluate throughput with and without offset commits, while the producer benchmarks (\ZioKafkaProducerBenchmark\ and \ZioKafkaSeqProducerBenchmark\) assess single-record and chunk-based sending in both sequential and parallel modes. These benchmarks use an embedded Kafka instance and report average execution times to help users understand and optimize throughput characteristics.
zio-kafka-bench/src/main/scala/zio/kafka/bench · high confidence
Configurable producer metrics collection
Users can now customize how producer metrics are collected by providing a custom implementation of the new \ProducerMetricsObserver\ trait via \ProducerSettings.withMetricsObserver\. The library includes a default \ZioMetricsProducerMetricsObserver\ that exposes metrics such as produce latency, batch size, send queue size, and authentication errors using zio-metrics, allowing for flexible integration with existing monitoring systems.
zio-kafka/src/main/scala/zio/kafka/producer · high confidence
Consumer metrics are now exposed via zio-metrics
A new \ZioMetricsConsumerMetricsObserver\ implementation has been added to the consumer metrics module, enabling the collection of consumer-side observations (such as poll, commit, rebalance, and runloop metrics) using the zio-metrics library. This change makes consumer metric data available for monitoring and observability, complementing the existing producer metric configuration.
zio-kafka/src/main/scala/zio/kafka/consumer/metrics · high confidence
Initial project setup and documentation
The repository has been initialized with the ZIO Kafka library (version 3.6.0), including a comprehensive README with usage examples, a Contributor License Agreement (CLA), and configuration files for the build system (SBT), code formatting (Scalafmt 3.9.4), and static analysis (Scalafix).
(repo-wide) · high confidence
Initial release of zio-kafka-testkit module
The \zio-kafka-testkit\ module (renamed from \zio-kafka-test-utils\ with base package \zio.kafka.testkit\) is now available, providing utilities for testing Kafka applications. It includes \KafkaTestUtils\ for constructing producers and transactional producers, helper methods for producing messages, and the \ZIOSpecWithKafka\ trait to simplify running tests against an embedded Kafka cluster. The module also ships with generated SSL keystore and truststore files for secure testing.
zio-kafka-testkit · high confidence
Introduction of a new diagnostics callback interface
A new \Diagnostics\ trait and a \NoOp\ implementation have been added to the library, providing a callback interface for diagnostic events. Additionally, an internal \ConcurrentDiagnostics\ helper is introduced to run these diagnostics in a separate fiber using an unbounded queue, ensuring that diagnostic emissions do not block the main execution flow and that a final event is emitted upon cleanup.
zio-kafka/src/main/scala/zio/kafka/diagnostics · high confidence
New consumer configuration and security APIs
The consumer module introduces a comprehensive \ConsumerSettings\ case class to centralize configuration, offering builder methods for tuning throughput or latency, managing commit and rebalance timeouts, and configuring offset retrieval strategies. A new \RebalanceListener\ API provides ZIO-native hooks for partition assignment, revocation, and loss events, while \StreamControl\ exposes a standardized interface to manage subscription lifecycles. Additionally, \KafkaCredentialStore\ simplifies SSL/TLS setup by allowing users to configure security protocols directly from PEM string inputs.
zio-kafka/src/main/scala/zio/kafka/consumer · high confidence
New testkit and admin client APIs for testing and cluster management
This release introduces the \zio-kafka-testkit\ module, providing \EmbeddedKafka\ and \SaslEmbedded\ layers to spin up in-memory Kafka brokers for testing, including support for custom broker properties and SASL authentication. It also adds a comprehensive \AdminClient\ API for managing Kafka clusters, including topic creation/deletion, consumer group management, and full ACL support (create, describe, delete ACLs, and resource pattern filters). The consumer API is updated with \CommittableRecord\ for easier offset committing and new \FetchStrategy\ implementations (\QueueSizeBasedFetchStrategy\, \ManyPartitionsQueueSizeBasedFetchStrategy\) to control prefetching and backpressure.
repository · high confidence
Behavioural changes
Consumer runloop execution model refactored for isolation and fairness
The internal consumer architecture has been restructured to improve stability and performance. The runloop now executes on a dedicated single-threaded executor with unique thread naming, isolating it from other ZIO effects. Access to the underlying Java Kafka consumer is now gated by a semaphore to ensure fair and thread-safe access, particularly during rebalances. Additionally, configuration for max poll intervals and rebalance durations is now explicitly managed via a new RunloopConfig, and a same-thread runtime layer is provided for specific internal operations.
zio-kafka/src/main/scala/zio/kafka/consumer/internal · high confidence
Introduces ZIO-native Serde, Serializer, and Deserializer abstractions
The \zio-kafka\ serde module now provides ZIO-based \Serializer\, \Deserializer\, and \Serde\ traits that replace the previous Kafka-native interfaces. These new abstractions allow serialization and deserialization logic to be expressed as ZIO effects (\RIO\), enabling seamless integration with ZIO's environment and error handling. The module includes built-in support for common types (String, Int, Long, etc.) via the \Serdes\ trait, offers utility methods for mapping and blocking execution, and provides factory methods to wrap existing Kafka serializers and deserializers.
zio-kafka/src/main/scala/zio/kafka/serde · high confidence
Resilient SSL endpoint validation against partial bootstrap server failures
The \SslHelper.validateEndpoint\ utility now tolerates individual bootstrap server connection failures during SSL configuration checks. Instead of failing the entire validation if some servers are unreachable, it proceeds to check the remaining available servers and only fails if no servers respond or if a specific configuration error is detected, preventing application crashes when only a subset of Kafka bootstrap servers are down.
zio-kafka/src/main/scala/zio/kafka/utils · high confidence
Test coverage
Added benchmark suite and comprehensive producer tests; Added comprehensive admin client test suite; Added integration tests for example consumer, producer, and service components; Added producer unit tests and test infrastructure; Added test coverage for Serde, Serializer, Deserializer, and KafkaCredentialStore; Added tests for SSL endpoint validation resilience; Added unit tests for internal consumer components; Added unit tests for queue size-based fetch strategies; Added validation tests for ConsumerSettings.
Dependencies
Build infrastructure updated to SBT 1.13.0 with ZIO SBT Ecosystem and MIMA checks
The project's build system has been upgraded to SBT version 1.13.0. The build now utilizes the ZIO SBT Ecosystem plugins (version 0.8.5) for CI, website generation, and ecosystem management. Additionally, binary compatibility checks are enabled via the MIMA plugin (version 1.2.0), configured to compare against a specified previous artifact version while filtering out internal package changes in the consumer and diagnostics modules.
project · high confidence
Update ZIO Kafka dependencies to latest versions
This change updates the project's build configuration to use ZIO 2.1.26, Scala 2.13.18 and 3.9.0, Kafka clients 4.3.1, and Logback 1.6.4. It also removes support for Scala 2.12, ensuring the library now only targets Scala 2.13+ and 3+.
(dependencies) · high confidence
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
How this codebase got here
Score
- CAI 60 → 51 (-8.7)
- Rubric changed (rubric-2026.10.5 → rubric-2026.11.2) — scores are not directly comparable.
Lenses
- Code Health 61 → 61 (+0.0)
- Architecture 90 → 89 (-1.0)
- Maturity 52 → 61 (+8.5)
- Readiness 76 → 59 (-16.5)
- Security 62 → 37 (-25.0)
- Performance 81 → 81 (+0.0)
Resolved (5)
- Change coupling: Offset.scala ↔ Runloop.scala (zio-kafka/src/main/scala/zio/kafka/consumer/Offset.scala)
- Documentation: no installation or build instructions (docs/index.md)
- Documentation: no project overview (docs/writing-tests.md)
- Off-boarding risk: anonymized user #1
- Outdated: org.apache.kafka:kafka-clients
New (2)
- Dependency hygiene PARTLY measured — sbt declarations read, dependency currency not (Maven Central unreachable)
- Off-boarding risk: anonymized user #1
Changes since last survey
- 3 commits — 3 feature/other, 0 fixes
By area
- (root) — 2 commits
- project/build.properties — 1 commit
Notable commits
- change: Update README.md (#1803)
- change: Update logback-classic to 1.6.6 (#1804)
- change: Update sbt, scripted-plugin to 1.13.1 (#1805)
Architecture
- Unchanged — 0 containers · 1 contexts · 0 edges
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
Survey your own repository
zio/zio-kafka was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.
About this page
- The score is its most recent published measurement, taken on 6 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
- Measured at commit 4d2fde684dafc27a20eac8cef974585d0238685d — the exact code this score is about.
- Scored under rubric-2026.11.2 — the same rubric and the same method as every other entry in this index.
- Measured by Canine Code Analyzer (cca.canine.dev) using cca measure-2026.11.3+preprod-60d1b390edf0.