devsisters/shardcake

Shardcake is a distributed sharding library for ZIO applications that manages entity lifecycle and message routing across multiple pods. It provides core capabilities for inter-pod communication, including fire-and-forget, request-response, and bidirectional streaming, while coordinating pod assignments via a configurable Shard Manager. The system supports pluggable backends for serialization and storage, with built-in implementations for Redis and Kryo, and includes comprehensive health checking and metrics for monitoring cluster state.

The storage-redisson module now provides a new Storage implementation backed by Redis via the Redisson client. Users can configure the storage keys for shard assignments and pod registrations through RedisConfig, and the implementation exposes a ZIO layer (StorageRedis.live) that integrates with RedissonClient to persist and retrieve shard assignments and pod states, including a stream for listening to assignment changes.

storage-redisson · high confidence

A new benchmarks module has been introduced to allow performance testing of the sharding system. This includes a JMH-based SendBenchmark that measures throughput by sending ping messages, utilizing a local Server implementation with in-memory storage and a mock ShardManagerClient to isolate sharding behavior without external dependencies.

benchmarks · high confidence

This change introduces the Shardcake library, a distributed sharding system for ZIO applications. It provides core components including a \Sharding\ service for managing entity lifecycle and message routing, a \ShardManagerClient\ for coordinating pod assignments via a GraphQL API, and a \Config\ object for tuning parameters like timeouts and shard counts. The library supports various communication patterns through the \Messenger\ trait, including fire-and-forget, request-response, and streaming (both sending and receiving streams), with automatic stream restart capabilities during rebalances. It also includes a \Broadcaster\ for topic-based messaging, metrics for monitoring shards and entities, and a \LocalSharding\ layer for simplified local testing.

entities/src/main/scala/com/devsisters/shardcake · high confidence

Added a new Kryo serialization module that provides a ZIO layer for serializing and deserializing messages using the \scala-kryo-serialization\ library. This implementation supports standard encode/decode operations as well as batched chunk processing, allowing users to switch to Kryo for potentially improved performance. A corresponding test suite verifies that objects can be serialized and deserialized correctly.

serialization-kryo · high confidence

The examples directory now includes a new 'complex' package containing a complete demonstration of the Shardcake library. This example features a Shard Manager service (ShardManagerApp) and a Guild application (GuildApp) that manages entity lifecycles using Redis for state storage. The GuildBehavior implementation showcases specific sharding patterns, including handling Join/Leave messages, enforcing a maximum guild size of five members, and supporting entity termination via explicit messages. The example also updates the underlying Redis connection configuration to use a password-protected endpoint.

examples/src/main/scala/example/complex · high confidence

This change introduces a set of new trait-based interfaces in the \com.devsisters.shardcake.interfaces\ package to decouple core shardcake functionality from specific implementations. The \Pods\ interface defines methods for inter-pod communication, including support for streaming replies and sending message streams, alongside a \noop\ layer for testing. The \Serialization\ interface provides a contract for encoding and decoding messages, with a default \javaSerialization\ layer. The \Storage\ interface abstracts the persistence of shard assignments and pod states, offering an in-memory \memory\ layer for single-pod testing. Additionally, the \PodsHealth\ interface (renamed from the previous \sharding\ package) standardizes pod liveness checks, providing \noop\ and \local\ layers for development and testing. These interfaces collectively enable easier testing and implementation of custom backends for shardcake.

core/src/main/scala/com/devsisters/shardcake/interfaces · high confidence

Added new example files (GuildApp, GuildBehavior, ShardManagerApp) that demonstrate how to use shardcake for entity sharding. The example shows registering a 'guild' entity type, handling messages like Join/Leave, and supporting streaming replies via StreamReplier. It also includes a ShardManagerApp for managing the shard topology.

examples/src/main/scala/example/simple · high confidence

The legacy sharding entity implementation has been removed from the \entities\ module. This change deletes the core files responsible for entity lifecycle management and inter-pod communication, including \EntityManager\, \Sharding\, \ShardClient\, \ShardingService\, and supporting types like \Config\, \EntityType\, and \Message\. Users relying on this specific entity-based sharding API will no longer have access to these classes.

entities/src/main/scala/com/devsisters/sharding · high confidence

The \Pods\, \Serialization\, and \Storage\ trait interfaces, along with the \ShardId\ type alias defined in the package object, have been removed from the core sharding module. This cleanup eliminates the previous abstraction layer for pod assignment, data serialization, and storage management, as well as the specific integer-based shard identifier type.

core/src/main/scala/com/devsisters/sharding · high confidence

The gRPC protocol layer now exposes a \GrpcConfig\ case class that allows users to configure the gRPC client and server behavior. Users can provide a custom \Executor\ for handling gRPC requests, set a \shutdownTimeout\ for graceful server termination, and define \clientInterceptors\ and \serverInterceptors\ (such as for tracing or logging) to be applied to outgoing and incoming calls respectively. These configuration options are integrated into the \GrpcPods\ client implementation and the \GrpcShardingService\ server layer, replacing previous hardcoded defaults.

protocol-grpc/src/main/scala · high confidence

The internal entity management logic has been restructured to introduce dedicated channel abstractions for handling replies and outgoing messages. EntityManager now explicitly manages entity lifecycles, including idle-time expiration and termination signals, while ReplyChannel provides distinct implementations for single-value promises and streaming queues to support both immediate and stream-based responses. Additionally, SendChannel abstracts the serialization and transmission of both single messages and streams to remote pods, and the GraphQL client has been moved to the shardcake package namespace.

entities/src/main/scala/com/devsisters/shardcake/internal · high confidence

The shardcake library now exposes distinct exception classes to provide clearer diagnostics for common runtime issues. Users will encounter \EntityNotManagedByThisPod\ when a message targets a pod not currently responsible for the entity (typically during rebalancing), \InvalidShardId\ for malformed shard identifiers, \PodUnavailable\ when a target pod is unresponsive, and \SendTimeoutException\ if a message send exceeds the configured timeout. Additionally, \StreamCancelled\ is introduced to explicitly signal when a stream connection is interrupted due to rebalancing, entity inactivity, or pod unavailability.

entities/src/main/scala/com/devsisters/shardcake/errors · high confidence

The Kubernetes pod health check implementation has been refactored to allow filtering pods by a custom \labelSelector\ in addition to the existing namespace and IP-based filtering, giving users finer control over which pods are monitored. The configuration model has been updated to include this new field, and the underlying health logic now converts Kubernetes API failures into more descriptive, user-friendly exception messages to aid in debugging connectivity or permission issues.

health-k8s · high confidence

The Redis storage implementation no longer uses hardcoded keys for shard assignments and pod registration. Users can now provide a \RedisConfig\ specifying custom keys (defaulting to "shard\_assignments" and "pods"), allowing for flexible deployment scenarios where key names must differ from the defaults. Additionally, the \savePods\ operation now conditionally writes to Redis only when the pod map is non-empty, reducing unnecessary write operations.

storage-redis/src/main · high confidence

The core library package has been renamed from \com.devsisters.sharding\ to \com.devsisters.shardcake\, affecting internal types such as \Pod\ and \PodAddress\. Additionally, a new \package.scala\ file introduces type aliases for \ShardId\ and \EpochMillis\, along with a private helper method to render shard IDs as a sorted string for logging and API purposes.

core/src/main/scala/com/devsisters/shardcake · high confidence

The Shard Manager now exposes a comprehensive configuration model (ManagerConfig) allowing users to tune rebalancing behavior (interval, retry interval, and max rebalance rate), pod health checking (interval, ping timeout), and persistence reliability (retry interval and count). Operationally, the manager enforces that only healthy pods can register, performs regular health checks on all pods, and exposes new metrics (pods, assigned/unassigned shards, rebalances, pod health checks) via a new ManagerMetrics object. The GraphQL API is updated to include a checkAllPodsHealth mutation and returns sorted shard assignments, while the underlying HTTP server implementation is modernized to use zio-http.

manager · high confidence

The sharding gRPC service now supports streaming requests and responses, allowing clients to send streams of messages and receive streamed replies via the new SendStream, SendAndReceiveStream, and SendStreamAndReceiveStream RPCs. Additionally, the SendRequest message includes a new optional reply\_id field to facilitate reply correlation, and the Java package for generated code has been updated from com.devsisters.sharding.protobuf to com.devsisters.shardcake.protobuf.

protocol-grpc/src/main/protobuf · high confidence

Added end-to-end and authentication example tests; Added integration tests for Redis storage implementation; Added test for Java serialization round-trip; Added tests for broadcasting, sharding, and streaming capabilities.

This update upgrades the project's core Scala versions (2.12.21, 2.13.18, 3.3.7) and major libraries, including ZIO (2.1.24), Caliban (3.0.0), sttp (4.0.13), and zio-grpc (0.6.3). It introduces new submodules for Redisson-based storage (\storageRedisson\) and Kryo serialization (\serializationKryo\), while renaming existing modules to the \shardcake\ prefix and removing the legacy \examples\ and \protobuf\ build configurations.

(dependencies) · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• First survey — no prior run to compare against. CAI 66.

• Code Health 95
• Architecture 99
• Maturity 68
• Readiness 53
• Security 77

• 225 commits — 202 feature/other, 23 fixes

• entities/src — 77 commits
• (root) — 58 commits
• vuepress/docs — 35 commits
• manager/src — 14 commits
• examples/src — 12 commits
• .github/workflows — 8 commits
• protocol-grpc/src — 7 commits
• (repo) — 3 commits
• docs/assets — 3 commits
• .github/worflows — 2 commits
• health-k8s/src — 2 commits
• benchmarks/src — 1 commit
• core/src — 1 commit
• project/plugins.sbt — 1 commit
• storage-redisson/src — 1 commit

• fix: Docs fix
• fix: Fix CI
• fix: Fix Scala 3 compile error
• fix: Fix assignedShards metric (#124)
• fix: Fix formula in docs (#11)
• fix: Fix images
• fix: Fix log
• fix: Fix promise leak when using streaming with long-running actors (#136)
• fix: Fix recruiting url (#176)
• fix: Fix replyStream interruption (#111)
• fix: Fix scala 3
• fix: Fix slow memory leak while an actor stays active (#109)
• fix: Fix streaming response interruption (#77)
• fix: Fix test
• fix: Fix test
• fix: Fix unassignedShards metric and rename singletons metric tag (#123)
• fix: Fix version in docs
• fix: Fix version in docs
• fix: Fixes
• fix: Revert "Revert "update to caliban 2.5 and zio-http-rc4 (#110)""
• …and 205 more

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

devsisters/shardcake was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.