medialab/xan
62.9
Weak · 14 November 2026
49.2k
lines of production code
Rust
primary language
5
measurements over time
What this system is
Features
Initial release of xan with Zenodo metadata and documentation
This commit introduces the initial version of the \xan\ CLI tool (renamed from \xcsv\/\xsv\), providing a high-performance, SIMD-accelerated CSV data manipulation command line interface written in Rust. It establishes the project's foundational documentation, including a comprehensive README with installation instructions for various package managers (Cargo, Homebrew, Nix, etc.), a CHANGELOG detailing features and breaking changes from version 0.62.0 onwards, and an AUTHORS file. Additionally, it adds \.zenodo.json\ to enable formal citation of the software via Zenodo, listing the project's creators and affiliations.
(repo-wide) · high confidence
New aggregation functions and statistical capabilities
The moonblade aggregation engine now supports a wider range of statistical computations, including geometric mean, harmonic mean, and root-mean-square (RMS) via Welford's algorithm, as well as approximate cardinality and quantiles using HyperLogLogPlus and TDigest. Users can also access new aggregators for all/any boolean checks, first/last values, and numeric extents with argmin/argmax tracking. The system introduces sparkline generation for number distributions and temporal granularity inference, alongside improved precision for sums using the Kahan-Babuska algorithm.
src/moonblade/agg/aggregators · high confidence
New automation scripts for documentation, releases, and segmentation testing
Three new shell scripts have been added to the repository to streamline development workflows. The \docs.sh\ script automates the generation of command-line help documentation and Moonblade reference materials by invoking the built binary. The \release.sh\ script simplifies the publishing process by handling version retrieval, cargo publishing, and git tagging in a single step. Additionally, \test\_segmentation.sh\ provides a functional test suite for file segmentation logic, iterating through various chunk configurations to verify data integrity.
scripts · high confidence
New graph, parquet, and processing infrastructure
The application now includes core infrastructure for graph analysis, Parquet data support, and pipeline processing. A new \graph.rs\ module implements graph data structures, degree mapping, and GEXF serialization (supporting both 1.2 and 1.3 namespaces). Parquet file reading is enabled via a new \parquet.rs\ module that handles schema projection and type mapping. Additionally, a \processing.rs\ module introduces a \Children\ abstraction for managing child processes in pipelines, including a \check\ method for error handling, and a \ratatui.rs\ module provides utilities for rendering TUI frames to stdout. Also in this area: Fix histogram domain-max calculation for nonlinear scales; Renamed utility modules and reorganized TUI code structure; Add --add-missing flag to xan select to create empty columns for non-existent names; Heatmap command gains scale options and label hiding.
src · high confidence
New optimized collection primitives and hash map backend
The \src/collections\ module now provides several new data structures to improve performance and functionality: \ClusteredInsertHashmap\ for faster lookups on clustered keys, \ContextBuffer\ for managing before/after context windows, \TopKHeap\ for efficient top-k selection, and \IncrementalId\ for generating unique IDs. Additionally, the module's default \HashMap\ and \HashSet\ types have been switched to use the \ahash\ crate (via \AHashMap\/\AHashSet\) for improved hashing performance.
src/collections · high confidence
New xan agg command for custom aggregation expressions
The \xan agg\ command is introduced to allow users to aggregate CSV data using custom aggregation expressions (e.g., \sum(count1) as sum1\). Unlike \xan stats\ for typical statistics or \xan groupby\ for grouped aggregation, this command produces a single row of CSV containing the results of aggregating the whole file. The implementation leverages the \AggregationProgram\ from the moonblade module and supports dynamic expressions to mangle data before aggregation.
src/cmd · high confidence
Architecture
Moonblade functions refactored into modular submodules
The moonblade function library has been reorganized from a single monolithic file into distinct submodules (fmt, fuzzy, io, maps, ops, sequences, strings, temporal, utils, web). This structural change improves code maintainability and allows for clearer separation of concerns across formatting, file I/O, data manipulation, and temporal operations, while preserving the existing public API surface for users.
src/moonblade/functions · high confidence
New modular aggregation and windowing engine for Moonblade
The \src/moonblade/agg\ module has been restructured into a new, modular architecture that separates aggregation logic into distinct components: \program.rs\ for defining aggregation programs (including group-by, pivot, and along-columns variants), \stats.rs\ for descriptive statistics (mean, variance, quantiles, frequencies), and \window.rs\ for window functions (rolling sums/averages, cumulative min/max, ranking, lead/lag). This change introduces a unified \Aggregator\ enum and \AggregationProgram\ interface, enabling more maintainable and extensible aggregation pipelines while supporting parallel execution and memory-optimized layouts for existing and new functions like covariance, correlation, and geometric/harmonic means.
src/moonblade/agg · high confidence
Behavioural changes
Moonblade expression engine rewritten with Pest parser and new program types
The Moonblade expression engine has been completely rewritten, replacing the previous implementation with a new Pest-based parser (grammar.pest) and a fresh interpreter architecture. This change introduces new program types including \ChooseProgram\ (for handling current/new record comparisons), \SelectionProgram\ (for named and plural column selections), and \ScrapingProgram\ (for HTML scraping with CSS selectors and text extraction). The parser now supports a richer syntax including list and map literals, lambda expressions, named arguments, pipeline operators, and improved string/regex handling. Error reporting has been restructured with dedicated \ConcretizationError\ and \EvaluationError\ types for clearer diagnostics.
src/moonblade · high confidence
Moonblade type system refactored with new argument handling and temporal support
The moonblade types module has been restructured to introduce a new \BoundArgument\ system for more efficient argument passing and a \FunctionArguments\ API that supports named arguments, variadic parameters, and arity validation. \DynamicValue\ now includes dedicated temporal variants (\Zoned\, \DateTime\, \Date\, \Time\, \Span\) powered by the \jiff\ library, alongside a new \DynamicNumber\ type for numeric operations. The \HeadersIndex\ has been updated to support negative indexing and duplicate header names, while \DynamicValue\ serialization and deserialization have been modernized to use \TryFrom\ and improved visitor patterns.
src/moonblade/types · high confidence
Test coverage
Added test resources for text scraping and HTML parsing; Expanded test coverage for core CSV manipulation commands.
Dependencies
Major dependency overhaul and project rebrand to xan
The project has been renamed from xcsv to xan, updated to Rust edition 2024 with a minimum supported Rust version of 1.88.0, and its dependency tree has been significantly expanded and modernized. Key updates include upgrading simd-csv to 0.14.0, paltoquet to 0.12.0, and calamine to 0.36.1, while adding support for new formats and features via crates like bgzip, npyz, and parquet. The Cargo.lock file has been regenerated to reflect these changes, introducing a comprehensive list of new dependencies such as ahash, bstr, and fast-float2, and removing older git-based dependencies in favor of versioned crates.io packages.
(dependencies) · high confidence
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
How this codebase got here
Score
- CAI 68 → 63 (-5.5)
- Rubric changed (rubric-2026.10.5 → rubric-2026.11.4) — scores are not directly comparable.
Lenses
- Code Health 79 → 70 (-9.9)
- Architecture 87 → 99 (+12.0)
- Maturity 74 → 76 (+1.7)
- Readiness 61 → 53 (-8.1)
- Security 71 → 67 (-4.7)
Resolved (30)
- Args::adj_or_count (cognitive 18) (src/cmd/matrix.rs)
- Args::bivar (cognitive 17) (src/cmd/matrix.rs)
- Args::correlation (cyclomatic 17) (src/cmd/matrix.rs)
- Duplicated block (11 lines × 2) (src/moonblade/agg/program.rs)
- Duplicated block (12 lines × 2) (src/cmd/slice.rs)
- Duplicated block (12 lines × 2) (src/cmd/vocab.rs)
- Duplicated block (14 lines × 2) (src/moonblade/functions/temporal.rs)
- Duplicated block (15 lines × 2) (src/cmd/matrix.rs)
- Duplicated block (17 lines × 2) (src/cmd/explode.rs)
- Duplicated block (19 lines × 2) (src/cmd/join.rs)
- Duplicated block (6 lines × 2) (src/config.rs)
- Duplicated block (7 lines × 3) (src/cmd/explode.rs)
- Duplicated block (8 lines × 3) (src/cmd/join.rs)
- Duplicated block (8–9 lines × 2) (src/cmd/search.rs)
- Flaky test: xan::tests.test_rename::rename_slugify
- Flaky test: xan::tests.test_select::select_all_then_some::headers
- Flaky test: xan::tests.test_select::select_reverse::headers
- Flaky test: xan::tests.test_select::select_reverse::no_headers
- Hotspot: src/cmd/cat.rs (src/cmd/cat.rs)
- Hotspot: src/main.rs (src/main.rs)
- …and 10 more
New (241)
- Args::inputs (cognitive 28) (src/cmd/scrape.rs)
- ClassTooLong: Config (src/config.rs)
- Critical CVE: [GHSA redacted] (Cargo.lock)
- Duplicated block (11 lines × 2) (src/config.rs)
- Duplicated block (11 lines × 2) (src/moonblade/agg/program.rs)
- Duplicated block (11 lines × 2) (src/moonblade/interpreter.rs)
- Duplicated block (12 lines × 2) (src/cmd/slice.rs)
- Duplicated block (12 lines × 2) (src/cmd/vocab.rs)
- Duplicated block (12 lines × 2) (src/config.rs)
- Duplicated block (13 lines × 2) (src/cmd/search.rs)
- Duplicated block (13–14 lines × 2) (src/cmd/plot.rs)
- Duplicated block (13–14 lines × 2) (src/cmd/search.rs)
- Duplicated block (14 lines × 2) (src/cmd/search.rs)
- Duplicated block (14 lines × 2) (src/cmd/search.rs)
- Duplicated block (14 lines × 2) (src/moonblade/agg/program.rs)
- Duplicated block (14–15 lines × 2) (src/moonblade/agg/program.rs)
- Duplicated block (16 lines × 2) (src/cmd/matrix.rs)
- Duplicated block (16 lines × 2) (src/moonblade/agg/program.rs)
- Duplicated block (18 lines × 2) (src/cmd/search.rs)
- Duplicated block (19 lines × 2) (src/cmd/join.rs)
- …and 221 more
Changes since last survey
- 44 commits — 35 feature/other, 9 fixes
By area
- (root) — 23 commits
- src/cmd — 12 commits
- src/moonblade — 3 commits
- docs/cmd — 2 commits
- (repo) — 1 commit
- .github/workflows — 1 commit
- docs/moonblade — 1 commit
- tests/test_scrape.rs — 1 commit
Notable commits
- fix: Adding xan enum --raw Fix #1165
- fix: Adding xan from -D/--output-delimiter Fix #1228
- fix: Adding xan search -H/--highlight Fix #1097
- fix: Fixing quadratic perf issue wrt variadic args Fix #1212
- fix: Fixing scrape streaming input Fix #1139
- fix: JSON serialization for plural-yielding aggregation functions Fix #1216
- fix: Proper refactoring of xan explode Fix #1230
- fix: Renaming xan search --left to -k/--keep-unmatched Fix #1219
- fix: xan search -R & --replacement-column to abide by -k Fix #1225
- change: Adding .svg to .gitignore
- change: Adding missing documentation for data scraping extractor
- change: Adding xan behead --raw Related to #1165
- change: Adding xan count --row-groups Related to #1143
- change: Adding xan flatten -r/--highlight-regex & --highlight-tag
- change: Adding xan from -f=parquet --row-group-index Related to #1143
- change: Adding xan rename --raw Related to #1165
- change: Amortize IO allocations Related to #1173
- change: AmortizedSerializer
- change: Better scraper DSL wrt post-processing Related to #1173
- change: Changelog
- …and 24 more
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
Survey your own repository
medialab/xan was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.
About this page
- The score is its most recent published measurement, taken on 14 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
- Measured at commit f4e85feea93d8c76244efa00c84091a425300893 — the exact code this score is about.
- Scored under rubric-2026.11.4 — the same rubric and the same method as every other entry in this index.
- Measured by Canine Code Analyzer (cca.canine.dev) using cca measure-2026.11.65+preprod-3c33c1882dc1.