sharkdp/hyperfine

Added a suite of Python scripts for analyzing and visualizing \hyperfine\ benchmark results exported as JSON. This includes \advanced\_statistics.py\ for computing detailed statistics (mean, median, percentiles, min/max), \plot\_whisker.py\ for box-and-whisker plots, \plot\_histogram.py\ for distribution histograms, \plot\_progression.py\ for sequential run plots with moving averages, \plot\_parametrized.py\ for errorbar plots of parametrized benchmarks, \plot\_benchmark\_comparison.py\ for grouped bar plots, and \welch\_ttest.py\ for statistical significance testing. The \scripts/README.md\ documents prerequisites (numpy, matplotlib, scipy) and usage examples.

scripts · high confidence

Users can now access the manual page for hyperfine via the standard man command, and documentation includes a new diagram illustrating the execution order of benchmark phases.

doc · high confidence

The build process now generates shell completion scripts for Bash, Fish, Zsh, PowerShell, and Elvish. This allows users to enable command-line auto-completion for hyperfine in their respective shells, improving usability and reducing typing errors.

(repo-wide) · high confidence

The command-line interface has been completely rewritten using the clap crate, introducing new options for parameterized benchmarking via --parameter-scan and --parameter-list, as well as --parameter-step-size. The internal architecture has been refactored into separate modules for commands, options, and scheduling, and the tool now supports running multiple commands with comparative output. Additionally, statistical outlier detection has been implemented to identify and handle anomalous benchmark results.

src · high confidence

A new \src/output\ module has been introduced, providing dedicated functionality for formatting and warnings. This includes \format.rs\ for duration formatting with microsecond precision, \progress\_bar.rs\ for configuring the \indicatif\ progress bar, and \warnings.rs\ for generating outlier and execution-time warnings.

src/output · high confidence

The codebase now includes a new \src/util\ module containing several extracted utilities: \exit\_code.rs\ provides platform-specific exit code extraction; \min\_max.rs\ offers safe min/max functions for f64; \number.rs\ defines a \Number\ enum supporting integer and decimal types; \randomized\_environment\_offset.rs\ generates random strings for testing offset effects; and \units.rs\ provides time unit formatting for seconds, milliseconds, and microseconds.

src/util · high confidence

The \src/benchmark\ directory has been restructured into a new \benchmark\ module containing dedicated files for benchmark results (\benchmark\_result.rs\), execution logic (\executor.rs\), timing data (\timing\_result.rs\), relative speed comparison (\relative\_speed.rs\), and scheduling (\scheduler.rs\). This refactoring introduces a \Benchmark\ struct to encapsulate the state and logic for running a single benchmark, while the \Executor\ trait and its implementations (\RawExecutor\, \ShellExecutor\) are now responsible for command execution and measurement. The \Scheduler\ now orchestrates the entire benchmarking process, including setup, warmup, benchmark, and cleanup commands, and handles the computation and display of relative speed comparisons. This change improves code organization and separates concerns within the benchmarking workflow.

src/benchmark · high confidence

The export module has been refactored to use a generic \MarkupExporter\ trait for markup-based formats (Markdown, Asciidoc, and Emacs org-mode), replacing previous per-format implementations. This change introduces a shared \MarkupFormatter\ interface that handles table header, row, and divider generation, allowing for more consistent and maintainable code across different markup formats. The CSV and JSON exporters remain separate but are now part of a unified \Exporter\ trait system. This refactoring simplifies the codebase and makes it easier to add new markup-based export formats in the future.

src/export · high confidence

The \src/timer\ module has been restructured into a dedicated module with platform-specific implementations for Unix and Windows. This change introduces a unified \TimerResult\ struct that exposes real, user, and system time alongside peak memory usage in bytes. On Windows, the implementation now uses job objects to accurately track CPU time and memory, while Unix systems use \getrusage\ for similar metrics. The \execute\_and\_measure\ function now returns a \TimerResult\ containing these metrics, providing users with more detailed performance data about their processes.

src/timer · high confidence

Added new integration tests in the \tests\ directory to verify the execution order of setup, prepare, benchmark, and conclude commands, as well as validation for command-line arguments like \--shell=none\, \--runs\, and \--conclude\. These tests ensure that the tool correctly sequences pre/post-benchmark hooks and handles invalid argument combinations.

tests · high confidence

The project's Rust dependencies have been updated to their latest compatible versions, including major bumps for clap (2→4), libc, serde, and serde\_json. The Cargo.toml has been updated to specify Rust edition 2018, set the MSRV to 1.88.0, and configure release profile optimizations (LTO, strip, single codegen unit). Additionally, the build system now uses clap and clap\_complete as build dependencies, and dev dependencies include insta for snapshot testing.

(dependencies) · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• CAI 40 → 69 (+28.7)
• Rubric changed (rubric-2026.08.15 → rubric-2026.09.15) — scores are not directly comparable.

• Code Health 100 → 94 (-5.6)
• Architecture 69 → 100 (+31.0)
• Maturity 59 → 59 (-0.3)
• Readiness 26 → 78 (+51.7)
• Security 36 → 69 (+33.0)

• Coverage not measured — test suite did not build
• Dimension evaluation failed
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• No exposed public API
• No tests found
• Test reliability not included

• Commands::from_cli_arguments (cognitive 24) (src/command.rs)
• Dependency hygiene PARTLY measured — Cargo dependencies read, dependency currency not (crates.io unreachable)
• Documentation: no project overview (README.md)
• FunctionTooLong: hyperfine::cli::build_command (src/cli.rs)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• Medium CVE: [GHSA redacted] (Cargo.lock)
• Medium advisory (unmaintained): RUSTSEC-2024-0384 (Cargo.lock)
• …and 12 more

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

sharkdp/hyperfine was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.