databricks/dicer
57.8
Adequate · 6 November 2026
57.4k
lines of production code
Scala
primary language
4
measurements over time
What this system is
Features
Add caching utility libraries and test infrastructure
This change introduces a new set of utility libraries for the caching system, including an intrusive min-heap, a state machine framework, a consistent hash ring, deterministic sampling utilities, and HyperLogLog support via protocol buffers. It also adds Java API test utilities for checking equality and loading test data, along with the necessary Bazel build files, protocol buffer definitions, and test data files to support these components.
(repo-wide), dicer/common, dicer/external · high confidence
Added Bazel support for compiling Protobufs to Rust using buffa
This change introduces a new set of Bazel rules and build files in the bazel directory to enable compiling Protocol Buffer definitions into Rust crates using the buffa code generator. It includes the addition of a Cargo.Bazel.lock file for Rust dependencies and a maven\_install.json file for Java dependencies, alongside new Bazel build rules (BUILD.bazel, .bzl files) that define how to invoke the buffa codegen tool, handle well-known types, and manage cross-crate dependencies for generated Rust code.
bazel · high confidence
Added HTTP client wrapper and internal header support in RPC wrappers
The RPC wrapper layer now includes a new SimpleWebClient for synchronous HTTP GET requests, primarily to support readiness probe testing. Additionally, the Headers object has been extended with a new constant for Dicer two-level sharding (x-databricks-internal-dicer-secondary-slice-key), and HttpRequestInfo now exposes a getRemoteAddr method. These changes enhance the internal compatibility and testing capabilities of the RPC wrapper without altering external API contracts.
wrappers/rpc · high confidence
Added Java API support for Dicer client configuration
New files have been added to the \dicer/client/javaapi\ module to provide concrete configuration implementations (\ClerkConfImpl\ and \SliceletConfImpl\) that bridge the gap between Java and Scala code. These classes allow Java users to pass configuration and TLS options to the underlying Scala Clerk and Slicelet components, bypassing the Scala-specific cake pattern. Additionally, a \TestUtils\ utility class was added to \caching/util/javaapi\ to support Java-based testing by providing methods to load test data from textproto files and verify list equality and sorting.
caching/util/javaapi, dicer/client/javaapi · high confidence
Assigner OSS build stubs internal-only features and adds core migration and probe logic
The Assigner component now includes the core implementation for the consistent-hashing preferred assigner driver, a migration driver that wraps old and new drivers to enable staged migration between them, and a probe source that exposes Kubernetes readiness and liveness status based on connection health. To support the open-source build, internal-only capabilities such as DPage debug endpoints, Dicer Simulator event logging, Dicer Tee event emission, remote membership checking, and HTTP-layer request rate limiting are provided as no-op stubs. Additionally, the diff introduces the gossip protocol implementation for exchanging target migration configuration between peer Assigners, utilities for slice key arithmetic, and a load distribution aggregator for combining replica load metrics.
dicer/assigner · high confidence
Introduce Dicer client feature rollout configuration and flag evaluation
Added the \DicerClientFeatureRolloutConfig\ data model, the \DicerClientFeatureRolloutFlag\ trait, and the \DicerClientFeatureRolloutFlagImpl\ production implementation to the \dicer/client/feature-rollouts\ module. This change enables the Dicer client to evaluate feature flags based on region-specific rollout rules defined in \.textproto\ configuration files, supporting deterministic sampling and target-specific overrides while degrading safely to disabled if configuration or environment details are unavailable.
dicer/client/feature-rollouts · high confidence
New Java API for Dicer external components
Added a new Java API in \dicer/external/javaapi\ that provides Java wrappers for the existing Scala Dicer implementation. This includes classes for configuring and managing Clerks (\Clerk\, \ClerkConfig\), Slicelets (\Slicelet\, \SliceletConfig\), and test environments (\DicerTestEnvironment\). The API also exposes core data types such as \Slice\, \SliceKey\, \Target\, and \ResourceAddress\, along with helper classes like \SliceKeyBuilder\ and \ImplFriend\ for internal conversions, enabling Java applications to interact with Dicer's sharding and assignment logic.
dicer/external/javaapi · high confidence
New friend APIs for two-level sharding and SliceKey deserialization
Added three new accessor objects in the \dicer/friend/external\ package to expose restricted capabilities for internal or allowlisted consumers. \TwoLevelShardingClerkAccessor\ provides a method to retrieve resource stubs affinitized to a primary and secondary key pair, supporting two-level sharding routing. \TwoLevelShardingHeaders\ offers helpers to serialize and deserialize the secondary slice key into and from HTTP headers. \SliceKeyAccessor\ exposes a \fromRawBytes\ method for deserializing previously serialized \SliceKey\ instances without validation, intended for unmarshaling data from storage or protobufs.
dicer/friend/external · high confidence
Behavioural changes
1 commit (0 fixes) modifying (repo-wide)
A change to existing behaviour in (repo-wide) — 1 commit, 89 files: “Project import generated by Copybara”. Also in this area: New Rust metrics facade for Caching and Dicer clients.
_wrappers/rust/infra, wrappers/rust/throttled\logger, wrappers/rust/metrics · low confidence · unverified
Demo server key type changed to Long and logging improved
The Dicer demo server now accepts Long keys instead of Int keys for cache operations, updating the internal cache and request handling to use Long types. Additionally, the demo's key generation utility was simplified to use standard fingerprinting, and the assignment algorithm's logging was enhanced with deterministic, truncated string representations for Resources, MutableSliceAssignment, and ResourceState to improve debuggability.
dicer/assigner/algorithm, dicer/demo · high confidence
Dicer Assigner config layer: new authorization, migration, and metrics scaffolding with behavioral restrictions
This change introduces the Dicer Assigner configuration layer with several new capabilities and behavioral constraints. It adds an authorization framework (Authorizer, AuthorizerHelper, UnauthorizedException) that currently acts as a no-op fallback, accepting all watch requests, while AuthorizerMetrics tracks rejection reasons (e.g., missing headers, name mismatches) for observability. It introduces TargetMigrationConfig and StaticTargetMigrationConfigProvider to define migration roles (Source/Destination) and types (e.g., GeneralToSmk), but the OSS provider is hardcoded to NO\_MIGRATION, meaning no target migration occurs. It also renames DicerConfigProvider to DicerSafeConfigProvider, restricting it to cluster-scoped overrides (rejecting instance-scoped ones) and adding validation. Additionally, it removes the ZERO\_PENALTY churn configuration and exposes more metadata (size, targetNames) from InternalTargetConfigMap.
dicer/assigner/config, wrappers · high confidence
OSS configuration and feature-flag wrappers updated for compatibility and static config support
The wrappers/conf and wrappers/feature-flag modules have been updated to better mirror internal APIs and support static configuration values. Configuration now reads from the DB\_CONF environment variable via a new RawConfigSingleton, and DbConfSingletonImpl uses this singleton instead of an empty base config. New files (CurrentProject, CurrentProjectInfo, Environment, KamClientConfig, KamEndpoint) and updates to KubernetesLocation (adding region, legacy context, and environment fields) and ProjectConf (adding string-named constructors) improve API compatibility. In feature flags, BaseDynamicConf now respects static configuration values rather than always returning defaults, and new FlagValueProvider/NamedFlag traits are added for OSS compatibility.
wrappers/conf · high confidence
Refactored debugging server and added infrastructure resource resolution
The local debugging HTTP server has been refactored: the previous singleton \DebugHttpServer\ is removed and replaced by a new \InfoService\ that manages the server lifecycle, registers ZPages and debug endpoints, and supports an optional readiness probe endpoint at \/ready\. The debugging dashboard root handler is now a dedicated \DebuggingRootPageHandler\, and the admin debug URL path has changed from \/debug\ to \/admin/debug\. Additionally, the infrastructure data model now supports regions alongside Kubernetes clusters, introducing \InfraResource\ types and a \ResourcePath\ utility to resolve URIs to specific infrastructure resources.
wrappers/deployment, wrappers/instrumentation · high confidence
Removal of threading utilities and updates to DatabricksMain configuration and hooks
This change removes the \AdvancedInstrumentedScheduledThreadPoolExecutor\, \InstrumentedScheduledThreadPoolExecutor\, and \NamedExecutor\ files from the \wrappers/util\ package, eliminating the open-source wrappers for instrumented thread pools and named executors. Additionally, \DatabricksMain\ is updated to accept an optional \rawConfig\ parameter for configuration overrides and introduces \newReadinessSource\ and \newLivenessSource\ hooks to mirror internal readiness and liveness status sources, allowing OSS services to compile against these interfaces even though they currently return \None\.
wrappers/util · high confidence
Test coverage
1 commit adding/updating tests in (repo-wide); 1 commit adding/updating tests in (repo-wide); Added test infrastructure for Rust metrics and refactored Scala test utilities.
Dependencies
Initial Cargo.lock generated for Rust dependencies
A Cargo.lock file has been added to the project, pinning the versions of Rust dependencies such as clap, buffa, and bytes. This ensures reproducible builds by locking the dependency tree to specific versions and checksums.
(dependencies) · high confidence
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
How this codebase got here
Score
- CAI 58 → 58 (-0.2)
- Rubric changed (rubric-2026.10.5 → rubric-2026.11.2) — scores are not directly comparable.
Lenses
- Code Health 64 → 54 (-9.4)
- Architecture 97 → 97 (+0.1)
- Maturity 59 → 70 (+11.1)
- Readiness 52 → 54 (+1.7)
- Security 60 → 58 (-2.0)
- Performance 78 → 78 (+0.0)
Resolved (4)
- Documentation: no usage examples (docs/UserGuide.md)
- TodoComment (dicer/assigner/src/LoadWatcher.scala)
- Unstable project //dicer/assigner:assigner_probe_source
- Unstable project //dicer/assigner:dicer_tee_event
New (6)
- HackComment (caching/util/test/WatchCellSuite.scala)
- Hotspot: dicer/common/src/SubscriberHandler.scala (dicer/common/src/SubscriberHandler.scala)
- Medium advisory (unsound): RUSTSEC-2026-0386 (Cargo.lock)
- SubscriberHandler.replyFromCell (cyclomatic 17) (dicer/common/src/SubscriberHandler.scala)
- TodoComment (dicer/client/test/ClerkRetrySuiteBase.scala)
- TodoComment (dicer/client/test/ClerkRetrySuiteBase.scala)
Changes since last survey
- 1 commits — 1 feature/other, 0 fixes
By area
- dicer/assigner — 1 commit
Notable commits
- change: Project import generated by Copybara.
Architecture
- Unchanged — 0 containers · 1 contexts · 0 edges
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
Survey your own repository
databricks/dicer was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.
About this page
- The score is its most recent published measurement, taken on 6 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
- Measured at commit 4c19a6c723e118cd27ead35119daca0a99e47f10 — the exact code this score is about.
- Scored under rubric-2026.11.2 — the same rubric and the same method as every other entry in this index.
- Measured by Canine Code Analyzer (cca.canine.dev) using cca measure-2026.11.3+preprod-60d1b390edf0.