embeddedkafka/embedded-kafka

This is a Scala library that provides embedded instances of Apache Kafka, Kafka Connect, and Kafka Streams for use in testing environments. It enables developers to spin up and manage these components within their test suites, supporting both Kafka 4.x KRaft mode and modern Scala versions. The system facilitates end-to-end testing of message producers, consumers, connectors, and stream processing logic without requiring external infrastructure.

5 changes.

The repository was initialized with comprehensive tooling, including SBT 2.x, Scala 3 support, and automated dependency management. Legacy embedded Kafka traits and their associated tests were removed to streamline the codebase, which was then restructured into a multi-module architecture supporting Scala 2.13 and 3.

8 changes.

The project migrated the embedded Kafka broker from ZooKeeper to KRaft mode, restructuring the codebase into a modular trait-based architecture. It simultaneously introduced new embedded components for Kafka Connect and Kafka Streams, enabling comprehensive end-to-end testing of these services within the library.

Users can now start and interact with an embedded Kafka Connect server within tests using the new \EmbeddedKafkaConnect\ trait. This addition introduces the \startConnect\ method, which initializes a standalone Connect worker with configurable REST API ports, offset storage paths, and extra configuration options, allowing for end-to-end testing of Connect connectors alongside the existing embedded Kafka broker.

kafka-connect/src/main · high confidence

The library now includes a new \EmbeddedKafkaStreams\ trait and associated configuration classes that allow users to start and manage Kafka Streams instances within an embedded Kafka environment. This enables testing of stream processing logic by providing a \runStreams\ method that handles topic creation, stream initialization, and cleanup, while automatically detecting available ports and configuring state directories in temporary folders.

kafka-streams/src/main · high confidence

The repository has been initialized with a new MIT license, a comprehensive README documenting usage for embedded Kafka (including configuration for broker/controller ports and custom properties), and a \.scalafmt.conf\ file setting the formatter version to 3.11.5 with Scala 2.13 dialect. Additionally, a \.git-blame-ignore-revs\ file was added to ignore specific formatting commits, and a \renovate.json\ file was created to enable automated dependency updates for the master and v3.x branches.

(repo-wide) · high confidence

The legacy \EmbeddedKafka\ trait in \src/main/scala/net/manub/embeddedkafka/EmbeddedKafka.scala\ has been removed. This trait previously provided a \withEmbeddedKafka\ method that manually instantiated ZooKeeper and Kafka servers with hardcoded ports (6000 and 6001) and basic configuration. Its removal indicates a shift toward newer, more robust embedded Kafka implementations (such as the \withRunningKafka\ or \EmbeddedKafka\ objects mentioned in commit history) that offer better lifecycle management, configurable ports, and AVRO support.

src/main · high confidence

The embedded Kafka implementation has been migrated from ZooKeeper-based startup to the new KRaft (Kafka Raft) mode, requiring explicit configuration of both broker and controller ports (defaulting to 6001 and 6002 respectively) and introducing a \controllerPort\ parameter in \EmbeddedKafkaConfig\. The library has moved to the \io.github.embeddedkafka\ package, and the codebase has been restructured into a modular trait-based architecture (\AdminOps\, \ConsumerOps\, \ProducerOps\, \KafkaOps\) to improve maintainability and separation of concerns.

embedded-kafka/src/main · high confidence

Added comprehensive test suite for EmbeddedKafka; Added test logging configuration for embedded Kafka; Added test utilities for JSON serialization; Added tests for Kafka Streams integration; Added tests for the embedded Kafka Connect server; Removal of legacy EmbeddedKafkaSpec test suite.

The project has been restructured from a single module into three sub-modules: embedded-kafka, kafka-streams, and kafka-connect, with dependencies now managed via a centralized Dependencies.scala file. The build targets Scala 2.13 and 3, replacing the previous Scala 2.11 target, and includes updated settings for compilation, testing, coverage, and release processes.

(dependencies) · high confidence

The build system has been upgraded from SBT 0.13.8 to SBT 2.0.9, requiring users to update their local SBT installation. This change is accompanied by significant dependency updates: Kafka is upgraded to version 4.3.1, Scala 3 to 3.3.8, Scala 2.13 to 2.13.18, Jackson to 2.21.2, and SLF4J to 2.0.19. Additionally, the build plugins have been refreshed, including sbt-release (1.5.0), sbt-scalafmt (2.6.2), sbt-ci-release (1.12.1), sbt-scoverage (2.4.4), and sbt-tpolecat (0.5.7).

project · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• First survey — no prior run to compare against. CAI 61.

• Code Health 100
• Architecture 100
• Maturity 56
• Readiness 53
• Security 65

• 300 commits — 294 feature/other, 6 fixes

• (root) — 96 commits
• project/plugins.sbt — 65 commits
• project/Dependencies.scala — 58 commits
• project/build.properties — 52 commits
• .github/workflows — 13 commits
• embedded-kafka/src — 9 commits
• .github/.scala-steward.conf — 3 commits
• kafka-connect/src — 2 commits
• .github/dependabot.yml — 1 commit
• .github/mergify.yml — 1 commit

• fix: Add doc generation CI step and fix scaladoc link error in EmbeddedKafkaStreams
• fix: Revert "Update slf4j-reload4j to 2.0.9 (#466)" (#467)
• fix: Revert release.yml changes (#461)
• fix: add setup-sbt step to fix workflows on ubuntu-24.04 (#544)
• fix: bump kafka to v3.8.0 and fix compilation errors (#514)
• fix: fix badges (#417)
• change: Add 'Reformat with scalafmt 3.9.7' to .git-blame-ignore-revs
• change: Add renovate.json
• change: Bump Kafka to v3.4.0 (#424)
• change: Bump actions/checkout from 2 to 3 (#360)
• change: Bump actions/checkout from 3 to 4 (#469)
• change: Bump actions/setup-java from 3 to 4 (#481)
• change: Bump actions/setup-java from 5 to 6
• change: Cross-compile project to Scala 3 (#421)
• change: Downgrade scala 3 version to version 3.3.3. This is the LTS version of Scala 3. (#501)
• change: Enable Renovate on v3.x branch in addition of master
• change: Reformat with scalafmt 3.9.7
• change: Setting version to 3.1.0
• change: Setting version to 3.1.1-SNAPSHOT
• change: Setting version to 3.2.0
• …and 280 more

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

embeddedkafka/embedded-kafka was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.