kucherenko/jscpd

This system is a high-performance, Rust-based code analysis engine that detects copy-paste duplication, semantic clones, and dead code across a wide range of programming languages. It provides comprehensive project health scoring, complexity metrics, and trend tracking, while offering extensive reporting formats and CI integration capabilities. The tool also supports cross-codebase comparison and integrates with AI models via a built-in MCP server for enhanced developer workflows.

The nested-paths-demo fixture now includes core application logic for financial calculations and reporting. A new money module provides utilities for converting string amounts to integer cents, formatting those cents into human-readable currency strings, and splitting amounts evenly. Additionally, a report module has been added to generate monthly summaries of ledger entries, grouping them by category and displaying totals.

fixtures/nested-paths-demo · high confidence

A new Rust example (\examples/rust-cpd-finder\) has been added to demonstrate how to integrate and run the jscpd v5 copy-paste detection engine from a Rust application. The example provides a minimal CLI tool that accepts file paths as arguments, configures the detection sensitivity (defaulting to 50 minimum tokens), and outputs the number of detected clones along with details of the first ten matches.

examples/rust-cpd-finder · high confidence

Added sample code fixtures for ActionScript, Ada, ANTLR4, Apex, APL, Astro, AWK, BASIC, Bicep, Brainfuck, CFML, CFScript, and C-like languages (C, C++, C\#, etc.) to enable testing and validation of duplication detection for these specific file formats and syntaxes.

fixtures · high confidence

This change introduces the complete codebase for the semantic-demo fixture, providing a functional full-stack application. The backend (Rust/Axum) implements a REST API for listing articles, fetching individual articles by slug, and calculating cart quotes with tax and shipping logic, backed by a PostgreSQL database with automatic migrations. The frontend (Svelte/TypeScript) provides the user interface, including components for article cards, pagination, a shopping cart summary, a signup form with client-side validation, a payment form with Luhn card validation, and a theme toggle. It also includes shared utilities for API communication with retry logic and relative time formatting.

fixtures/semantic-demo/backend, fixtures/semantic-demo/frontend · high confidence

Added a new benchmark suite in the \benchmark/\ directory that evaluates six copy/paste detection tools (jscpd@5, jscpd-rs, Duplo, Fallow, Simian, PMD CPD) against a multi-language fixture set. The benchmark measures execution speed, clone detection counts, and duplicate line coverage, while also assessing how well each tool handles cross-format detection in component files (.vue, .svelte, .astro) and embedded code in Markdown. Additionally, it compares the AI token efficiency of each tool's output format, highlighting jscpd@5's new 'ai' reporter as the most efficient for LLM consumption.

benchmark · high confidence

The \basta\ tool now includes a new engine for detecting dead code, allowing users to find unused files, exports, declarations, and imports. This feature adds a \--dead-code\ flag (accessible via \jscpd --dead-code\ or the \basta\ binary) that builds an import graph from project entry points and reports unreachable code. The engine supports multiple languages, integrates with framework definitions to recognize entry points, and provides confidence scores for findings to help users prioritize cleanup.

rust/crates/basta/src · high confidence

The cpd-tokenizer crate has been introduced to replace the previous tokenization logic, providing a Rust-native implementation that supports 223 language formats via a new formats registry. It adds specialized tokenizers for embedded languages in container files, including Razor (.razor), Vue Single-File Components (SFC), Svelte, and Astro, ensuring that code blocks within HTML are correctly isolated and tokenized. The crate also introduces cross-format clone detection for TypeScript by stripping type-only syntax to match JavaScript, and adds function-level similarity extraction for JavaScript and TypeScript using the OXC parser. Additionally, it implements a generic whitespace-and-punctuation tokenizer for non-JS/TS languages with correct comment handling for various syntaxes (C-style, hash, semicolon, etc.) and supports ignore regions via \jscpd:ignore\ markers and code-level regex patterns.

rust/crates/cpd-tokenizer/src · high confidence

The \rust/crates/cpd\ crate introduces a new Rust-based implementation of the \jscpd\/\cpd\ CLI, replacing the previous TypeScript version. This change adds several new analysis modes: \--compare\ for side-by-side function comparison between two folders using semantic embeddings; \--complexity\ for scanning code complexity without clone detection; \--dashboard\ and \--health\ for generating project health scores and badges; and \--dead-code\ for identifying unused code via the \basta\ engine. It also introduces \--history\ to track duplication trends over git commits, \--baseline-from-ref\ for stateless CI gating, and \--mcp\ to expose the tool as a Model Context Protocol server for LLM integration. The CLI now supports cross-format clone detection (\--cross-formats\), isolated folder skipping (\--skip-isolated\), and detailed summary reports (\--summary\).

rust/crates/cpd/src · high confidence

The \cpd-semantic\ crate now includes a \--compare\ mode that pairs functions between two distinct codebases (such as a project and its port, or iOS and Android versions) rather than just finding clones within a single project. This feature introduces a two-step pairing process: first, it matches functions based on semantic similarity using code embeddings, requiring mutual near-best matches that exceed specific thresholds; second, it pairs remaining unpaired functions by name similarity (ignoring case and underscores) if they reside in linked modules and meet a medium similarity level. The implementation includes new modules for comparison logic (\compare.rs\), a \UnitReader\ pass to collect functions for comparison, and updated thresholding rules that distinguish between same-language and cross-language pairs.

rust/crates/cpd-semantic/src · high confidence

The cpd-core crate now includes dedicated modules for new analysis capabilities: deadcode.rs defines the data model for unused code findings (files, exports, symbols, imports, members); health.rs implements a 0–100 project health score based on duplication, dead code, and complexity; history.rs provides the data structures and sparkline helpers for duplication trends over git history; and similarity.rs introduces function-level similarity detection using AST shingles and MinHash for JS/TS. These modules support the corresponding CLI flags (--rust-diagnostics, --health, --history, --similarity) by providing the underlying data models and calculation logic.

rust/crates/cpd-core/src · high confidence

The \basta\ tool now includes a new engine for detecting dead code, starting with support for JavaScript, TypeScript, JSX, TSX, and Python. The JavaScript analyzer leverages the \oxc\ parser to perform semantic analysis, correctly resolving bindings through shadowing and hoisting, and handles CommonJS patterns (\require\, \module.exports\) that standard ESM parsers miss. It also supports single-file components (Vue, Svelte, Astro) by parsing the script portion while scanning markup for component usage. The Python analyzer uses the \ruff\ parser to infer module boundaries via \\_\all\\_\ and handles dynamic runtime references (e.g., \getattr\, string-based imports) by lowering confidence rather than ignoring them. Both analyzers define entry points via convention-based globs (e.g., \index.ts\, \main.py\) and read manifest files (\package.json\, \pyproject.toml\) to identify project roots and dependencies.

rust/crates/basta/src/lang · high confidence

The repository now includes a suite of scripts in rust/scripts to automate the build, packaging, and publishing of the Rust-based jscpd binary. build-pypi-wheels.py repacks release tarballs into PyPI platform wheels for macOS, Linux, and Windows, exposing both jscpd and cpd commands. publish-npm.sh and publish-crates.sh handle publishing to npm and crates.io respectively, managing platform-specific packages, dependency ordering, and idempotency checks. sync-version.mjs synchronizes version numbers across the Rust workspace, npm wrappers, and PyPI metadata, while npm-prebuilt-package.mjs constructs the individual platform packages. gen-formats-md.mjs keeps the supported formats documentation in sync with the tokenizer source.

rust/scripts · high confidence

The \cpd-semantic\ crate introduces a new \--semantic\ mode that detects code clones using semantic similarity. By default, it runs the CodeRankEmbed model locally on the CPU using the Candle library, eliminating the need for external API calls. The system includes a persistent on-disk vector cache to speed up repeated scans, configurable similarity thresholds per model, and support for external OpenAI-compatible APIs via the \http\ provider. Security is enforced by preventing config files from sending code or API keys to remote hosts unless explicitly allowed via the command line.

rust/crates/cpd-semantic/src/embed · high confidence

The Rust cpd-reporter crate has been restructured into a multi-crate workspace, bringing a comprehensive set of new reporting capabilities and CI integration features. Users can now generate SVG badges for duplication and health scores, output reports in CodeClimate/GitLab Code Quality and OpenMetrics formats, and view a unified project dashboard covering health, complexity, and dead code. A new baseline system allows CI pipelines to gate on new duplication only by comparing clone fingerprints against a committed baseline file. Console output has been enhanced with verbose blame comparisons, and the reporter now supports opt-in codebase summaries and duplication history trends.

rust/crates/cpd-reporter/src · high confidence

The Rust-based jscpd engine (v5) introduces experimental semantic clone detection via \--semantic\, which uses code embedding models to find functionally similar code beyond token matching. It also adds a new dead-code detection engine (basta) accessible via \--dead-code\, which analyzes import graphs to find unused code. The release includes a comprehensive dashboard (\--dashboard\) and health scoring (\--health\), improved complexity metrics, and fixes for nested scan paths, CRLF line endings, and Svelte store usage. The toolchain is pinned to Rust 1.97.1, and dependencies like \sha2\ and \paste\ (replaced by \pastey\ for security) have been updated.

rust · high confidence

The \rust/basta\ location now ships a standalone npm package (version 0.1.1) that includes a \README.md\, a \platform-map.js\ defining supported OS/arch/libc combinations, and a \run-basta.js\ entry point that resolves and executes the corresponding prebuilt Rust binary. This enables users to install and run \basta\ via npm without manual compilation. The tool also introduces framework detection, allowing it to automatically identify and keep alive files loaded by specific frameworks (e.g., Next.js, Django) based on a built-in list and optional custom configuration, ensuring that framework-managed entry points are not reported as dead code.

rust/basta · high confidence

The project has transitioned to a Rust-based engine (v5) that ships as a self-contained binary with no runtime dependencies, replacing the previous Node.js/TypeScript implementation. This new version supports 224 programming languages and introduces advanced detection features including Type-2 clones (ignoring identifiers, literals, and annotations), Type-3 near-miss clones via syntax-tree similarity and gap merging, and experimental Type-4 semantic clones using code embeddings. The release also adds a built-in MCP server for AI integration, a GitHub Action, and a pre-commit hook that uses a PyPI wheel instead of Node.js. Documentation has been restructured to separate v4 and v5 details, and the project now includes a CITATION file, Code of Conduct, and security policy.

(repo-wide) · high confidence

The \paste\ crate has been replaced with \pastey\ to resolve RUSTSEC-2024-0436, addressing the unmaintained status of the original library. This change ensures that dependencies like \gemm\, \pulp\, and \tokenizers\ continue to function correctly by providing the \paste!\ macro through the maintained \pastey\ successor, while keeping the workspace patch in place until \paste\ is no longer required.

rust/patches/paste · high confidence

The Rust cpd-finder crate has been reorganized into distinct modules: blame.rs now handles git blame enrichment for clone fragments, orchestrate.rs manages the full detection pipeline configuration and execution, pass.rs defines the trait for whole-file clone passes, statistics.rs computes duplication metrics, and walker.rs handles file discovery with improved symlink and pattern handling. This restructuring supports new features like cross-format clone detection, function-level similarity analysis, and improved path filtering while maintaining compatibility with existing CLI options.

rust/crates/cpd-finder/src · high confidence

The jscpd tool now offers a v5 release built on a Rust engine, providing a self-contained binary that runs without a Node.js runtime. This update adds native support for Windows on ARM, alongside existing support for macOS and Linux (x64 and arm64, glibc and musl). The npm package now installs a prebuilt binary for the detected platform, and the CLI behavior (including exit codes) has been aligned with v4 standards. Users should note that persistent stores (LevelDB/Redis) and the Node.js programming API are no longer supported in this version.

rust/jscpd · high confidence

Added demo fixtures for ignore mechanisms; Added integration tests for cpd-finder; Added integration tests for reporter blame data and output generation; Added integration tests for the Rust cpd binary.

This change adds the foundational dependency manifests and lockfiles for the new Rust-based jscpd v5 engine (jscpd 5.3.3) and the new dead-code detection tool (basta 0.3.0). It establishes the Rust workspace structure in \rust/Cargo.toml\ with crates for core logic, tokenization, finding, reporting, and semantic analysis, alongside platform-specific npm packages for the prebuilt binaries. It also introduces the \basta\ package for detecting unused code across JavaScript, TypeScript, Python, and Rust, and adds a benchmark tool configuration using \fallow\ and \jscpd-rs\. As part of this architectural shift, the legacy root \package.json\ is removed.

(dependencies) · high confidence

Added sample application code in the \fixtures/dead-code-demo\ directory to serve as test cases for dead-code detection tools. This includes a Vite-based bundler demo with internationalization, routing, and legacy API stubs, as well as a monorepo structure containing a shop app and a shared UI package with unused components.

fixtures/dead-code-demo/bundler, fixtures/dead-code-demo/monorepo/apps/shop, fixtures/dead-code-demo/monorepo/packages/ui · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• CAI 64 → 64 (-0.2)
• Rubric changed (rubric-2026.09.9 → rubric-2026.09.18) — scores are not directly comparable.

• Code Health 57 → 61 (+4.4)
• Architecture 100 → 99 (-1.2)
• Maturity 74 → 75 (+0.4)
• Readiness 63 → 56 (-6.6)
• Security 80 → 85 (+5.0)
• Domain Modelling 100 → 100 (+0.0)
• Accessibility 70 → 70 (+0.0)
• Performance 100 (new)

• Documentation: no installation or build instructions (docs/packages.md)
• Documentation: no usage examples (docs/packages.md)
• FunctionTooLong: cpd_tokenizer::generic::tokenize_line_content (rust/crates/cpd-tokenizer/src/generic.rs)
• FunctionTooLong: jscpd::main (rust/crates/cpd/src/main.rs)
• Hotspot: rust/crates/cpd-tokenizer/src/generic.rs (rust/crates/cpd-tokenizer/src/generic.rs)
• Off-boarding risk: anonymized user #1
• Repeated repair: rust/crates/cpd-core/src/paths.rs (rust/crates/cpd-core/src/paths.rs)
• cpd_tokenizer::generic::tokenize_line_content (cyclomatic 46) (rust/crates/cpd-tokenizer/src/generic.rs)
• jscpd::main (cognitive 112) (rust/crates/cpd/src/main.rs)
• jscpd::main (cyclomatic 81) (rust/crates/cpd/src/main.rs)

• Cached::embed (cognitive 22) (rust/crates/cpd-semantic/src/embed/mod.rs)
• Cached::embed (cyclomatic 16) (rust/crates/cpd-semantic/src/embed/mod.rs)
• Coverage not measured — JavaScript/TypeScript suite
• Documentation: no architecture or design documentation (rust/jscpd/README.md)
• Duplicated block (10 lines × 2) (fixtures/dart/file1.dart)
• Duplicated block (10 lines × 2) (rust/crates/cpd-reporter/src/dashboard.rs)
• Duplicated block (11 lines × 2) (rust/crates/cpd-reporter/src/dashboard.rs)
• Duplicated block (13 lines × 2) (fixtures/clike/file1.kt)
• Duplicated block (14 lines × 2) (fixtures/type2-demo/annotations/AlphaService.java)
• Duplicated block (16 lines × 2) (fixtures/dart/file1.dart)
• Duplicated block (16 lines × 2) (fixtures/swift/file1.swift)
• Duplicated block (19 lines × 2) (fixtures/dart/file1.dart)
• Duplicated block (20 lines × 2) (fixtures/python/file1.py)
• Duplicated block (220 lines × 2) (fixtures/clike/file1.java)
• Duplicated block (23–27 lines × 2) (rust/crates/basta/src/analyze.rs)
• Duplicated block (26 lines × 2) (fixtures/crlf-demo/checkout_eu.py)
• Duplicated block (27 lines × 2) (fixtures/dart/file1.dart)
• Duplicated block (5 lines × 2) (fixtures/clike/file1.kt)
• Duplicated block (5 lines × 2) (rust/crates/cpd/src/cli.rs)
• Duplicated block (6 lines × 2) (fixtures/dart/file1.dart)
• …and 80 more

• 178 commits — 125 feature/other, 53 fixes

• rust/crates — 73 commits
• (repo) — 45 commits
• (root) — 16 commits
• .github/workflows — 11 commits
• docs/rust.md — 6 commits
• fixtures/dead-code-demo — 4 commits
• rust/CHANGELOG.md — 4 commits
• fixtures/semantic-demo — 3 commits
• skills/code-migration — 3 commits
• rust/basta — 2 commits
• docs/ai-ready.md — 1 commit
• fixtures/compare-demo — 1 commit
• fixtures/crlf-demo — 1 commit
• fixtures/dashboard-demo — 1 commit
• fixtures/health-markup-demo — 1 commit
• fixtures/nested-paths-demo — 1 commit
• fixtures/summary-demo — 1 commit
• rust/Cargo.lock — 1 commit
• rust/Cargo.toml — 1 commit
• rust/patches — 1 commit

• fix: Merge pull request #1060 from kucherenko/fix/1059-follow-symlinks-paths
• fix: Merge pull request #1076 from kucherenko/fix/health-dashboard-review-findings
• fix: Merge pull request #1081 from kucherenko/fix/markup-complexity
• fix: Merge pull request #1083 from kucherenko/fix/wxt-entrypoints
• fix: Merge pull request #1084 from kucherenko/fix/health-markup-label
• fix: Merge pull request #1088 from kucherenko/fix/fixture-vitest-version
• fix: Merge pull request #1091 from kucherenko/fix/embedded-block-statistics
• fix: Merge pull request #1102 from kucherenko/fix/basta-svelte-stores
• fix: Merge pull request #1106 from kucherenko/fix/nested-scan-paths
• fix: Merge pull request #1109 from kucherenko/fix/audit-ignore-paste
• fix: Merge pull request #1116 from kucherenko/fix/crates-publish-fail-loudly
• fix: Merge remote-tracking branch 'origin/master' into fix/wxt-entrypoints
• fix: fix(basta): Rust test code is test code, and workspace paths resolve
• fix: fix(basta): Svelte store reads and template literals in markup count as uses
• fix: fix(basta): measure Rust item extents the same on a CRLF checkout
• fix: fix(basta): read Python names hidden in quoted annotations and src layouts
• fix: fix(basta): read WXT extensions from their own config (#1082)
• fix: fix(basta): scope config keys to real keys, honor WXT imports: false
• fix: fix(basta): what a run beside knip and fallow on 55 repositories found
• fix: fix(build): optimize gemm-f16 in test builds so arm64 tests compile
• …and 158 more

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

kucherenko/jscpd was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.