Skip to content
CAI
Software that uses CAICheck a score

snowflakedb/spark-snowflake

54.2

Adequate · 7 November 2026

11.9k

lines of production code

Scala

primary language

4

measurements over time

CAI band scale
CAI trend line
CAI lens gauges

What this system is

Features

Initial release of the Snowflake Spark Connector

This change introduces the initial version of the Snowflake Spark Connector, providing the core infrastructure for reading from and writing to Snowflake tables via Spark. It establishes the foundational components including parameter management (Parameters.scala), JDBC interaction (SnowflakeJDBCWrapper.scala), and the Spark Data Source API implementation (SnowflakeRelation.scala). The connector supports data transfer through both direct SELECT queries and cloud storage stages (S3/Azure), handling encryption, compression, and various data formats (CSV, JSON, Parquet). It also includes basic authentication support (OAuth, key pairs) and telemetry logging.

repository · high confidence

Initial repository setup with CI/CD, build, and documentation

The repository is initialized with essential infrastructure files, including a GitHub Actions workflow for integration and cluster testing, a deployment script for publishing to Maven Central, and an SBT-based build system supporting cross-compilation for Spark 3.5 and 4.0. The project adds a Codecov configuration for coverage reporting, a pre-commit hook for secret scanning, and a Scalastyle configuration to enforce code standards. Documentation is provided via a main README, developer notes, and test instructions, alongside example configuration files for Snowflake connections and a standard Apache 2.0 license.

(repo-wide) · high confidence

Spark 3.5 and 4.x compatibility with native Variant support and credential redaction

The connector now includes dedicated source implementations for Spark 3.5 and Spark 4.x. In Spark 4.x, the connector enables native support for the Spark VariantType, allowing VARIANT columns to be read and written directly via JSON staging. For both versions, the connector hardens security by registering sensitive option keys (such as private keys and OAuth secrets) with Spark's plan redaction, ensuring these credentials are masked in logs and the History Server UI.

src/main · high confidence

Behavioural changes

Integration test logging configuration updated

Added new log4j and log4j2 configuration files for the integration test suite to control log verbosity and output. The default test logging is now set to WARN to reduce log volume, while specific configurations are provided for console output, file logging, and compatibility with Spark 3.3, ensuring cleaner test runs and better separation of master/worker logs for security reviews.

src/it/resources · high confidence

Legacy spark-redshift codebase archived and migrated to Snowflake package structure

The legacy \spark-redshift\ codebase has been moved into the \legacy\ directory and restructured under the \net.snowflake.spark.snowflake\ package namespace. This change includes the addition of \Utils.SNOWFLAKE\_SOURCE\_NAME\ to enable compile-time validation of connection strings, and updates to configuration files and documentation (including a new \OLD-README.md\ and tutorial files) to reflect the transition from the Databricks \com.databricks.spark.redshift\ format to the Snowflake-specific implementation.

legacy · high confidence

Test coverage

Add Spark 3.5 integration test infrastructure; Added Mockito compatibility helper for test builds; Added Python test for Spark UDF integration; Added Snowflake-specific integration test infrastructure; Added Spark 4.x unit tests for Variant type handling; Added integration test suites for Snowflake Spark Connector; Added integration tests for SnowflakeSparkUtils; Added integration tests for internal stage file operations; Added placeholder directories for Spark 3.5, 4.0, and 4.1 test suites; Added tests for IO module security, GCP support, and upload abort handling; Added tests for SnowflakeFallbackCatalog error handling and V1Table fallback logic; Added unit tests for Snowflake Spark Connector; Integration tests for Spark 4.x compatibility and native VariantType support.

Dependencies

Upgrade to Spark 4.x support and Snowflake JDBC 4.x

The connector now supports Apache Spark 4.x (including 4.0 and 4.1) and Scala 2.13, requiring Java 17 for compilation and runtime. The Snowflake JDBC driver has been upgraded to version 4.0.2, and the connector version is set to 3.2.2. This update includes specific source directory handling for Spark 4.x, adjusted JVM options for Java 17 module opens, and updated dependency versions for Parquet Avro (1.15.2 for Spark 4.x, 1.13.1 for earlier versions) and Jackson (2.15.2).

(dependencies) · high confidence

Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

How this codebase got here

Score

  • CAI 55 → 54 (-1.1)
  • Rubric changed (rubric-2026.10.5 → rubric-2026.11.2) — scores are not directly comparable.

Lenses

  • Code Health 54 → 54 (+0.0)
  • Architecture 86 → 84 (-1.9)
  • Maturity 51 → 57 (+6.0)
  • Readiness 59 → 55 (-3.3)
  • Security 65 → 51 (-13.9)

Resolved (6)

  • Off-boarding risk: anonymized user #1
  • Outdated: com.amazonaws:aws-java-sdk-s3
  • Outdated: com.fasterxml.jackson.core:jackson-annotations
  • Outdated: com.fasterxml.jackson.core:jackson-core
  • Outdated: com.fasterxml.jackson.core:jackson-databind
  • Outdated: net.snowflake:snowflake-jdbc

New (2)

  • Dependency hygiene PARTLY measured — sbt declarations read, dependency currency not (Maven Central unreachable)
  • Off-boarding risk: anonymized user #1

Changes since last survey

  • 2 commits — 2 feature/other, 0 fixes

By area

  • (root) — 2 commits

Notable commits

  • change: SNOW-4355138: Generate Utils.VERSION from sparkConnectorVersion (#680)
  • change: bump version to 3.2.3 (#678)

Architecture

  • Unchanged — 0 containers · 1 contexts · 0 edges

Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

Survey your own repository

snowflakedb/spark-snowflake was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.

About this page

  • The score is its most recent published measurement, taken on 7 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
  • Measured at commit 775f5928be41d28c536b0457a89cb0132e631ed7 — the exact code this score is about.
  • Scored under rubric-2026.11.2 — the same rubric and the same method as every other entry in this index.
  • Measured by Canine Code Analyzer (cca.canine.dev) using cca measure-2026.11.3+preprod-5f8314471808.