apache/cassandra-spark-connector

This release marks the Spark Cassandra Connector's adoption by the Apache Software Foundation, introducing ASF governance structures and a complete migration to the SBT build system. A significant addition is a new idiomatic Java API wrapper for Spark and Streaming contexts, alongside a refactored driver layer featuring a local-node-first load balancing policy. The release is heavily focused on test coverage, adding extensive integration and unit tests for RDDs, SQL operations, data types, and authentication mechanisms, while also providing new CCM cluster management infrastructure for testing.

A new executable script named 'sbt' has been added to the project root. This script automatically downloads and manages the sbt (Scala Build Tool) launcher JAR, allowing users to build the project without manually installing sbt. The script attempts to download sbt version 1.10.0 from Maven Central if it is not already present, falling back to a manual installation instruction if the download fails or if curl/wget are unavailable.

sbt · high confidence

The project has been adopted by the Apache Software Foundation, introducing ASF governance structures. This includes the addition of \.asf.yaml\ for project management, a \NOTICE\ file for copyright attribution, and a \scalastyle-config.xml\ for code style enforcement. The \CHANGES.txt\ file has been restructured to document the history of releases and features under the new governance.

(repo-wide) · high confidence

The test-support module introduces a new \com.datastax.spark.connector.ccm.mode\ package containing several new executor classes—\ClusterModeExecutor\, \DebugModeExecutor\, \DeveloperModeExecutor\, \ExistingModeExecutor\, and \StandardModeExecutor\. These classes provide different strategies for managing Cassandra Cluster Manager (CCM) instances during integration tests, including a developer mode that preserves clusters and a standard mode that handles version-specific configurations like enabling materialized views for Cassandra 4.0+.

test-support/src/main/scala/com/datastax/spark/connector/ccm/mode · high confidence

Added new test-support components (CcmBridge, CcmConfig, ClusterMode) to manage local Cassandra clusters for integration testing. The CcmBridge provides a Scala wrapper around the CCM (Cassandra Cluster Manager) to start, stop, and interact with local Cassandra nodes. CcmConfig allows configuration of cluster parameters such as node count, IP prefixes, and SSL settings. ClusterMode introduces different operational modes (Standard, Debug, Developer, Existing) that control how the test cluster is created, persisted, and cleaned up, with the mode selectable via the CCM\_CLUSTER\_MODE environment variable.

test-support/src/main/scala/com/datastax/spark/connector/ccm · high confidence

Added a new set of Java API wrapper classes in the \com.datastax.spark.connector.japi\ package, including \CassandraJavaUtil\, \CassandraJavaRDD\, \CassandraJavaPairRDD\, and context-specific wrappers like \SparkContextJavaFunctions\ and \StreamingContextJavaFunctions\. These classes provide a more idiomatic Java interface for interacting with Cassandra tables and RDDs, enabling Java users to easily convert between Scala RDDs and Java RDDs, and access connector functionality through static factory methods on the utility classes.

connector/src/main · high confidence

The test-support module now includes a client certificate (client.crt) and its corresponding private key (client.key) in the resources directory. These files are new and appear to be used for client-side authentication or SSL/TLS configuration within the test environment, likely enabling integration tests to connect to services requiring mutual TLS or client certificate validation.

test-support/src/main/resources · high confidence

New resource files have been added to the integration test suite to support test execution and debugging. This includes logging configurations for Log4j, Log4j2, and Logback, which control the verbosity and format of test output. Additionally, a HOCON-based test configuration file (test.conf) and a metrics properties file have been introduced to define default driver settings and metrics sinks for the integration environment.

connector/src/it/resources · high confidence

The project's build configuration has been reorganized into new SBT files: Dependencies.scala, Publishing.scala, Testing.scala, Versions.scala, build.properties, plugins.sbt, and updates.sbt. These files define the project's dependencies (including Apache Spark 3.5.0, Cassandra Java Driver 4.18.1, and various test libraries), publishing settings, and test execution strategies. This change updates the build system to use SBT 1.10.0 and modernizes the dependency management structure.

project · high confidence

The driver's row data handling has been refactored to use new \CassandraRow\, \GettableData\, and \ScalaGettableData\ traits, which provide a cleaner API for accessing column values by name or index. Additionally, a new \LocalNodeFirstLoadBalancingPolicy\ has been introduced to prioritize local datacenter nodes and support node filtering, improving query routing efficiency.

driver/src/main · high confidence

Added Java test beans for mapper unit tests; Added RowMock test utility for CQL row handling; Added comprehensive test coverage for core connector components; Added embedded test infrastructure for the Spark connector; Added integration test utilities for multi-threaded Spark operations; Added integration tests for Cassandra connector authentication, SSL, and schema handling; Added integration tests for Cassandra data types; Added integration tests for Cassandra direct join, source options, and streaming; Added integration tests for DataFrame date handling, metadata, UDTs, and streaming sinks; Added integration tests for Java bean persistence and test base classes; Added integration tests for PrefetchingResultSetIterator; Added integration tests for Search Analytics and Solr query optimization; Added integration tests for Spark Streaming operations; Added integration tests for Storage-Attached Index (SAI) query pushdown; Added integration tests for all supported Cassandra data types; Added integration tests for partitioner components; Added integration tests for the Cassandra catalog namespace and table operations; Added integration tests for the Spark Cassandra Connector RDDs; Added integration tests for the Spark-Cassandra connector writer module; Added macOS localhost aliasing script for testing; Added test coverage for CassandraRDD mocking and CQL where clause stringification; Added test resource configuration files for logging; Added test suites for DSE authentication configuration factories; Added tests for Java utility classes and sample JavaBeans; Added tests for the partitioner module; Added unit tests for Cassandra connector configuration and connection factory; Added unit tests for Cassandra connector predicate and settings logic; Added unit tests for Cassandra connector writer components; Added unit tests for DHT token and range partitioning logic; Added unit tests for column selection and row reading; Added unit tests for connector utility classes; Added unit tests for input and output metrics updaters; Added unit tests for the Java API connector classes; New integration test cluster management infrastructure; Re-enable documentation validation tests.

The project is now built using SBT (Scala Build Tool) instead of the previous build system. This change introduces a new \build.sbt\ file that defines the project structure, including Scala versions 2.12 and 2.13, publishing credentials, and dependency management for the connector, driver, and test support modules. This establishes the foundation for future dependency updates and releases.

(dependencies) · medium confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• CAI 27 → 59 (+31.7)
• Rubric changed (rubric-2026.08.15 → rubric-2026.09.15) — scores are not directly comparable.

• Code Health 97 → 93 (-3.8)
• Architecture 96 → 99 (+3.8)
• Maturity 64 → 50 (-13.9)
• Readiness 23 → 67 (+44.0)
• Security 1 → 54 (+53.5)

• Context/problem (which version to prefer) and consequences/trade-offs are not stated in the visible text (doc/12_building_and_artifacts.md)
• Coverage not measured — test suite did not build
• Dimension evaluation failed
• Duplicated block (11 lines × 2) (connector/src/main/java/com/datastax/spark/connector/japi/rdd/CassandraTableScanJavaRDD.java)
• Duplicated block (19 lines × 2) (connector/src/main/java/com/datastax/spark/connector/japi/RDDAndDStreamCommonJavaFunctions.java)
• Duplicated block (20 lines × 2) (connector/src/main/java/com/datastax/spark/connector/japi/CassandraJavaUtil.java)
• Duplicated block (211 lines × 4) (connector/src/main/java/com/datastax/spark/connector/japi/RDDAndDStreamCommonJavaFunctions.java)
• Duplicated block (212 lines × 2) (connector/src/main/java/com/datastax/spark/connector/japi/RDDAndDStreamCommonJavaFunctions.java)
• Duplicated block (37 lines × 2) (connector/src/main/java/com/datastax/spark/connector/japi/CassandraJavaUtil.java)
• Duplicated block (48 lines × 2) (connector/src/main/java/com/datastax/spark/connector/japi/CassandraJavaUtil.java)
• Duplicated block (5 lines × 3) (connector/src/main/java/com/datastax/spark/connector/japi/RDDAndDStreamCommonJavaFunctions.java)
• Duplicated block (52 lines × 2) (connector/src/main/java/com/datastax/spark/connector/japi/CassandraJavaUtil.java)
• Duplicated block (54 lines × 15) (connector/src/main/java/com/datastax/spark/connector/japi/CassandraJavaUtil.java)
• Duplicated block (54 lines × 2) (connector/src/main/java/com/datastax/spark/connector/japi/CassandraJavaUtil.java)
• Duplicated block (608 lines × 2) (connector/src/main/java/com/datastax/spark/connector/japi/CassandraJavaUtil.java)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• …and 17 more

• CassandraSourceUtil.sparkSqlToJavaDriverType (cyclomatic 18) (connector/src/main/scala/com/datastax/spark/connector/datasource/CassandraSourceUtil.scala)
• ClassTooLong: CassandraJavaUtil (connector/src/main/java/com/datastax/spark/connector/japi/CassandraJavaUtil.java)
• ColumnType.fromScalaType (cognitive 55) (driver/src/main/scala/com/datastax/spark/connector/types/ColumnType.scala)
• ColumnType.fromScalaType (cyclomatic 38) (driver/src/main/scala/com/datastax/spark/connector/types/ColumnType.scala)
• Dormant codebase
• Duplicated block (10 lines × 2) (connector/src/main/scala/com/datastax/spark/connector/rdd/CassandraCoGroupedRDD.scala)
• Duplicated block (10 lines × 2) (connector/src/main/scala/com/datastax/spark/connector/rdd/CassandraCoGroupedRDD.scala)
• Duplicated block (10 lines × 3) (connector/src/main/scala/com/datastax/spark/connector/datasource/ScanHelper.scala)
• Duplicated block (11 lines × 2) (connector/src/main/scala/com/datastax/spark/connector/rdd/CassandraTableScanRDD.scala)
• Duplicated block (14 lines × 2) (connector/src/main/scala/org/apache/spark/sql/cassandra/execution/CassandraDirectJoinExec.scala)
• Duplicated block (14–15 lines × 2) (connector/src/main/scala/com/datastax/spark/connector/rdd/CassandraJoinRDD.scala)
• Duplicated block (7 lines × 2) (connector/src/main/scala/com/datastax/spark/connector/rdd/CassandraJoinRDD.scala)
• Duplicated block (9 lines × 2) (connector/src/main/scala/com/datastax/spark/connector/rdd/CassandraCoGroupedRDD.scala)
• Duplicated block (9 lines × 2) (connector/src/main/scala/com/datastax/spark/connector/rdd/CassandraCoGroupedRDD.scala)
• FileTooLong: japi/CassandraJavaUtil.java (connector/src/main/java/com/datastax/spark/connector/japi/CassandraJavaUtil.java)
• FileTooLong: types/TypeConverter.scala (driver/src/main/scala/com/datastax/spark/connector/types/TypeConverter.scala)
• FixmeComment (driver/src/main/scala/com/datastax/spark/connector/util/AnyObjectFactory.scala)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• …and 47 more

• Containers 0 added · 0 removed · contexts 1 added · 0 removed · edges 0 added · 0 removed

• repository

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

apache/cassandra-spark-connector was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.