projectglow/glow
67.2
Adequate · 6 November 2026
21.6k
lines of production code
Scala
with Python
4
measurements over time
What this system is
Features
Added HLS event logging capability
A new logging module has been introduced to record High-Level Service (HLS) events. This feature allows users to log specific events by providing a tag and optional arguments, which are then forwarded to the underlying JVM-based event recorder via PySpark.
python/glow/logging · high confidence
Added Spark submit script
A new shell script named spark-submit has been added to the bin directory. This script launches the Apache Spark submitter class using the configured classpath and Java options, providing a simple entry point for submitting Spark applications.
bin · high confidence
Added VCF processing utility scripts
New shell and Python scripts have been added to the test-data/vcf/scripts directory to support variant data processing workflows. These include gwas.sh for simulating a GWAS interface from VCF to CSV, gwas-region.py for filtering association results based on gene regions, and helper utilities (prepend-chr.sh, remove-info.sh, remove-rows.sh) for manipulating VCF format fields, along with a group\_file.txt for defining gene-region mappings.
test-data/vcf/scripts · high confidence
Initial Python package structure and build tooling
This change introduces the foundational structure for the Python package, including \setup.py\ for distribution, \environment.yml\ for development dependencies (such as PySpark 3.5.1, pandas, and scikit-learn), and a \render\_template.py\ script that generates language-specific client code from YAML definitions. It also adds configuration files like \.style.yapf\ for code formatting, \MANIFEST.in\ for packaging assets, and tests for the template rendering logic.
python · high confidence
Initial project scaffolding and configuration
The repository is initialized with foundational configuration and documentation files. This includes the Apache 2.0 license, a Contributor Covenant Code of Conduct, and contributor guidelines (CONTRIBUTING.md, GIT-PROCESS.md). Build and development tooling is established via \.scalafmt.conf\ (v2.7.5), \scalastyle-config.xml\, and a \functions.yml\ manifest that defines the Spark SQL functions for code generation. Testing infrastructure is added through \pytest.ini\, \conftest.py\ (handling Spark session fixtures and version-specific test skipping), and \codecov.yml\. Documentation hosting is configured via \.readthedocs.yml\, and release workflows are documented in \RELEASE.md\.
(repo-wide) · high confidence
Introduce Glow Python package with interval overlap joins and NumPy support
The \python/glow\ package is now available, providing a Python API for genomic data processing on Spark. This release adds \left\_overlap\_join\ and \left\_semi\_overlap\_join\ functions in \glow.sql.functions\ to perform interval-based joins optimized with range-join logic, and introduces \glow.register\ to initialize SQL extensions and Py4J converters. It also enables direct use of 1D and 2D NumPy double arrays in Spark expressions via automatic conversion to Java arrays and \DenseMatrix\ literals.
python/glow · high confidence
Introduce Weighted Genomic Regression (WGR) module for ridge and logistic regression
The \python/glow/wgr\ package is added, providing a new set of tools for genomic analysis including \RidgeReduction\ for feature space reduction, \RidgeRegression\ for quantitative trait modeling, and \LogisticRidgeRegression\ for binary classification. These classes leverage Spark DataFrames and Pandas for efficient computation of ridge models with cross-validation, supporting covariates and intercepts. The module also includes utility functions in \wgr\_functions.py\ for handling sample IDs, blocking variants, and reshaping labels for GWAS workflows.
python/glow/wgr · high confidence
Introduce service-based transformer registration and BGEN file support
The core module now registers genomic data transformers (LiftOverVariants, BlockVariantsAndSamples, NormalizeVariants, SplitMultiallelics, Pipe, and CleanupPipe) via Java ServiceLoader, allowing them to be discovered and applied dynamically through the Glow API. Additionally, this change adds complete support for reading and writing BGEN files, including schema inference, genotype reading for uncompressed and compressed (zlib, zstd) data, and record writing with configurable probability bit depths and ploidy handling.
core/src/main · high confidence
Behavioural changes
1 commit (0 fixes) modifying test-data/tabix-test-vcf
A change to existing behaviour in test-data/tabix-test-vcf — 1 commit, 9 files: “Filter Parsing and Tabix Index Push Down and Test Data (\#10)” (refs \#10).
test-data/tabix-test-vcf · medium confidence · unverified
14 commits (5 fixes) modifying glow.py
A change to existing behaviour in glow.py — 14 commits (5 fixs), 3 files: “Hail related enhancements (\#377)”; “Extending sample masking functionality in gwas linear regression (\#416)”; “Prepare phenotypes (\#328)”; and 11 more (refs \#377, \#416, \#328, \#390, \#323, …).
glow.py · medium confidence · unverified
14 commits (7 fixes) modifying python/glow/gwas
A change to existing behaviour in python/glow/gwas — 14 commits (7 fixs), 2 files: “Extending sample masking functionality in gwas linear regression (\#416)”; “Fix broken validation (\#343)”; “Prepare phenotypes (\#328)”; and 11 more (refs \#416, \#343, \#328, \#710, \#323, …).
python/glow/gwas · medium confidence · unverified
62 commits (7 fixes) modifying repository
A change to existing behaviour in repository — 62 commits (7 fixs), 26 files: “Simplified linear regression (\#140)”; “GlowGR (\#231)”; “More changes for logging and bgen write options (\#65)”; and 59 more (refs \#140, \#231, \#65, \#501, \#248, …).
repository · medium confidence · unverified
Automated generation of Scala SQL functions from YAML definitions
The core module now generates its Scala SQL function definitions (\core/functions.scala.TEMPLATE\) from a \functions.yml\ configuration file instead of maintaining them as static source code. This change introduces a template-based approach where function signatures, documentation, and argument handling (including optional arguments) are derived directly from the YAML definitions, simplifying the addition and maintenance of new SQL functions.
core · high confidence
Test coverage
Add PLINK test data fixtures; Added GFF3 test fixtures for schema inference and embedded FASTA; Added Picard-compatible liftover test fixtures; Added REGENIE test data for binary phenotype analysis; Added VCF test fixtures for edge cases and annotation formats; Added comprehensive test coverage for GWAS regression functions; Added liftover test data and documentation; Added test coverage for Glow Python integration; Added test data for BEDTools intersect operation; Added test data for binary logistic regression and variant sample block generation; Added test fixtures for variant splitter and normalizer; Added test infrastructure and suites for Glow data sources; Added thresholded VCF test data for BGEN validation; Expanded test data for VCF and CSV parsing; Initial test suite for Glow WGR module.
Dependencies
105 commits updating dependencies (1 manifest)
A dependency / build maintenance change in (dependencies) — 105 commits (10 fixs), 1 file.
(dependencies) · high confidence · unverified
Build infrastructure and dependency updates
The build system has been upgraded to SBT 1.10.4, and several SBT plugins have been updated or added, including sbt-assembly (2.3.0), sbt-pgp (2.3.0), sbt-sonatype (3.12.2), sbt-scalafmt (2.5.2), sbt-scoverage (2.2.2), and sbt-header (5.10.0). The ScalaTest dependency has been upgraded from version 3.0.5 to 3.2.18, and a library dependency scheme for scala-xml has been configured to allow always-compatible updates.
project · high confidence
Housekeeping
Added documentation for VCF merge test data
A README file has been added to the test-data/vcf-merge directory to document the source of the test VCF files, specifying that they contain the first 1000 sites of chromosome 22 from 1000 Genomes samples HG000096 and HG00097.
test-data/vcf-merge · high confidence
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
How this codebase got here
Score
- CAI 68 → 67 (-0.6)
- Rubric changed (rubric-2026.10.5 → rubric-2026.11.2) — scores are not directly comparable.
Lenses
- Code Health 72 → 72 (-0.0)
- Architecture 91 → 90 (-1.4)
- Maturity 67 → 71 (+3.4)
- Readiness 63 → 62 (-0.9)
- Security 78 → 69 (-8.5)
Resolved (8)
- Documentation: no installation or build instructions (docs/source/customizing/index.rst)
- Documentation: no usage examples (docs/source/customizing/index.rst)
- Orphaned file (core/src/main/scala/io/projectglow/gff/GffDataSource.scala)
- Outdated: com.github.broadinstitute:picard
- Outdated: com.github.samtools:htsjdk
- Outdated: org.apache.commons:commons-lang3
- Outdated: org.slf4j:slf4j-api
- Outdated: org.yaml:snakeyaml
New (2)
- Dependency hygiene PARTLY measured — sbt declarations read, dependency currency not (Maven Central unreachable)
- Off-boarding risk: anonymized user #1
Changes since last survey
- 1 commits — 0 feature/other, 1 fixes
By area
- core/src — 1 commit
Notable commits
- fix: Fix GFF reader failing under ANSI mode (Spark 4 default) (#823)
Architecture
- Unchanged — 0 containers · 2 contexts · 0 edges
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
Survey your own repository
projectglow/glow was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.
About this page
- The score is its most recent published measurement, taken on 6 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
- Measured at commit cc351dbefd47d9a9254f8ee7164d453554da4924 — the exact code this score is about.
- Scored under rubric-2026.11.2 — the same rubric and the same method as every other entry in this index.
- Measured by Canine Code Analyzer (cca.canine.dev) using cca measure-2026.11.3+preprod-60d1b390edf0.