tilo/smarter_csv

SmarterCSV is a high-performance Ruby library for parsing CSV files, featuring a modular architecture that supports both pure-Ruby and C-accelerated processing paths. It provides robust handling of diverse CSV formats, including auto-detection of separators and line endings, along with advanced features like chunked reading, header transformation, and custom value conversion. The system ensures data integrity through comprehensive edge-case testing and thread-safe error tracking, while offering detailed migration guides and configuration options for performance tuning.

5 changes.

The project underwent a significant structural refactoring, splitting the library into modular components and introducing a new Reader class to support thread-safe error tracking and improved parsing logic. This architectural shift was accompanied by a comprehensive overhaul of project configuration, including the addition of RuboCop, RSpec, and Travis CI setups, alongside expanded documentation and modernized gem dependencies.

5 changes.

The project underwent a major rewrite of its C extension to significantly improve CSV parsing performance through optimized algorithms and build system improvements. This period also focused on expanding test coverage to ensure correctness across diverse edge cases, encoding scenarios, and compatibility between the C and Ruby code paths.

8 changes.

This period focused on expanding test coverage for the C-accelerated CSV parsing engine, ensuring parity with the Ruby fallback across complex edge cases like quoting, encoding, and header handling. Additionally, an automated script was introduced to generate migration documentation, keeping upgrade guides synchronized with release notes.

A new Ruby script, \bin/generate-upgrading-md\, has been added to automatically generate the \UPGRADING.md\ documentation. This tool reads the \CHANGELOG.md\ and \docs/upgrade\_path.json\ to produce a structured migration guide that lists version series, highlights in-series cautions, and details specific migration actions required for upgrades, ensuring the documentation stays synchronized with release notes.

bin · high confidence

The custom \Hash.zip\ class method, which previously allowed creating a hash from parallel arrays of keys and values, has been removed from the \Hash\ class. Users relying on this extension will no longer have access to this convenience method and should use standard Ruby alternatives or their own implementations.

lib/extensions · high confidence

The C extension has been completely rewritten to deliver significantly faster CSV parsing. The new implementation introduces a \ParseContext\ structure to pre-compute parse options once per file, eliminating repeated Ruby hash lookups in the hot path. It also vendors the Eisel-Lemire algorithm from \fast\_float\ for correctly-rounded, high-performance decimal-to-double conversion. To ensure compatibility across different build environments, the build system now probes the compiler for supported optimization flags (such as \-march=native\ or \-mtune=native\) rather than relying on platform heuristics, allowing users to tune performance via the \SMARTER\_CSV\_PERFORMANCE\ environment variable.

ext · high confidence

The repository now includes a \.rspec\ file to standardize test execution (requiring \spec\_helper\, enabling color, and using documentation format), a \.rubocop.yml\ file to configure the RuboCop linter (targeting Ruby 2.5 and disabling numerous style/metrics rules), and a \.travis.yml\ file to define the CI build matrix across multiple Ruby and JRuby versions. Additionally, the project documentation has been significantly expanded with a new \CHANGELOG.md\, an \UPGRADING.md\ guide, a \CONTRIBUTORS.md\ file, and a rewritten \README.md\ that highlights performance improvements and new features. The license file has been updated to \LICENSE.txt\ (MIT), and the \Rakefile\ has been enhanced to support RSpec, RuboCop, and C-extension compilation tasks.

(repo-wide) · high confidence

The library's internal architecture has been restructured into distinct modules (AutoDetection, FileIO, Parser, Reader, etc.) to improve maintainability and performance. This change introduces a new PeekableIO wrapper that enables robust auto-detection of row and column separators on non-seekable streams (like pipes or STDIN) by buffering input, and enhances the auto-detection logic for line endings to handle quoted fields and partial reads more accurately. Additionally, header and hash transformation logic has been extracted into separate modules, allowing for more precise control over header disambiguation, key mapping, and value conversion (including numeric and zero-value removal).

_lib/smarter\csv · high confidence

The library has been restructured from a single-file implementation into a modular architecture with separate files for file I/O, auto-detection, parsing, and writing. A new SmarterCSV::Reader class is now the primary interface for processing, allowing access to internal state and original headers. Class-level methods like process, parse, each, and each\_chunk now delegate to this Reader and expose thread-safe error and warning tracking via SmarterCSV.errors and SmarterCSV.warnings, which are cleared at the start of each call and stored per-thread for safety in multi-threaded environments.

lib · high confidence

Added comprehensive test coverage for quoted field parsing edge cases; Added comprehensive test coverage for the CSV parsing engine; Added feature tests for C-accelerated CSV parsing and hash transformations; Added feature tests for CSV formatting edge cases; Added feature tests for CSV parsing behavior; Added feature tests for chunked reading and header handling; Added feature tests for special CSV parsing edge cases; Added test suite for backslash and quote escaping edge cases; Added tests for numeric conversion validation and value converter support; Expanded test coverage for SmarterCSV core features and edge cases; Expanded test fixtures for CSV parsing edge cases.

The smarter\_csv gemspec has been updated to require Ruby 2.6.0 or higher and now explicitly declares a dependency on the bigdecimal gem to ensure compatibility with Ruby 3.4+. The gem's metadata has been expanded with links for documentation, changelogs, and an upgrade wizard, and the license is now explicitly set to MIT. Additionally, the Gemfile has been restructured to include development dependencies such as rake, rubocop, and pry, as well as test dependencies like rspec and simplecov.

(dependencies) · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• First survey — no prior run to compare against. CAI 70.

• Code Health 72
• Architecture 100
• Maturity 59
• Readiness 90
• Security 82

• 300 commits — 266 feature/other, 34 fixes

• (root) — 142 commits
• lib/smarter_csv — 50 commits
• .github/workflows — 16 commits
• spec/smarter_csv — 16 commits
• spec/fixtures — 11 commits
• docs/ruby_csv_pitfalls.md — 10 commits
• spec/features — 8 commits
• docs/_introduction.md — 6 commits
• docs/upgrade_path.json — 6 commits
• docs/upgrade_wizard.html — 5 commits
• ext/smarter_csv — 5 commits
• lib/smarter_csv.rb — 5 commits
• (repo) — 4 commits
• docs/basic_read_api.md — 3 commits
• docs/header_transformations.md — 3 commits
• .github/FUNDING.yml — 2 commits
• .github/dependabot.yml — 2 commits
• docs/data_transformations.md — 2 commits
• docs/.nojekyll — 1 commit
• docs/basic_api.md — 1 commit

• fix: Bug fixes for 1.16.2 (#327)
• fix: Fix Ruby Toolbox badge link in README
• fix: Fix clang: error: unsupported argument 'native' to option '-march=' Error on ARM macOS (#313)
• fix: Fix invalid byte sequence handling in headers (#104)
• fix: Fix issue #139; Version 1.9.0 (#242)
• fix: Fix issue #229 / windows one-column files (#231)
• fix: SmarterCSV::Writer bugfix for quote_headers:true
• fix: Update changelog formatting and fix section titles
• fix: Version 1.17.1 : bug fix
• fix: adding tests for #136; fix error class hierarchy
• fix: bugfix issue #195
• fix: bugfix; release 1.2.1
• fix: fix
• fix: fix #161
• fix: fix README
• fix: fix README
• fix: fix Rakefile
• fix: fix comment_regex default value / https://github.com/tilo/smarter_csv/issues/136
• fix: fix doc bug
• fix: fix doc bug
• …and 280 more

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

tilo/smarter_csv was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.