pest-parser/pest
56.0
Adequate · 14 November 2026
19.7k
lines of production code
Rust
primary language
5
measurements over time
What this system is
Features
Added HTTP, JSON, SQL, and TOML grammars
New Pest grammar definitions have been added for HTTP, JSON, SQL, and TOML, enabling the parser to recognize and structure these data formats. The HTTP grammar handles request lines, headers, and delimiters; the JSON grammar supports objects, arrays, strings, numbers, booleans, and null values while rejecting unescaped control characters; the SQL grammar covers queries, DDL, ACL commands, and expressions; and the TOML grammar supports tables, arrays, strings, literals, and date-time values.
grammars/src/grammars · high confidence
Added fuzzing dictionaries for HTTP, JSON, SQL, and TOML grammars
New fuzzing dictionaries have been added to the \grammars/fuzz/dict\ directory to improve coverage for specific data formats. The commit introduces \http.dict\ with common HTTP methods, headers, and status codes; \json.dict\ containing valid and edge-case JSON structures, escape sequences, and numeric literals; \sql.dict\ with standard SQL keywords, operators, and syntax patterns; and \toml.dict\ covering TOML syntax, data types, and special characters. These files provide seed data for fuzzing tools to better exercise the parsers for these formats.
grammars/fuzz/dict · high confidence
Added fuzzing infrastructure for pest\_meta parser
Added a new fuzzing target for the pest\_meta parser that includes a call limit of 5,000 to prevent crashes or timeouts during execution. The change introduces the necessary fuzzing setup files, including a .gitignore for build artifacts and a README linking to the main fuzzing documentation.
meta/fuzz · high confidence
Added parentheses parsing example
A new example demonstrating how to parse nested parentheses structures using the low-level parser state API. The code defines a \ParenParser\ that recursively handles \(\ and \)\ delimiters, and includes conditional support for \miette\ error reporting via the \miette-error\ feature flag.
pest/examples · high confidence
Generator crate restructured with doc comment support and no\_std capability
The generator module has been reorganized into a new crate structure, introducing dedicated source files for documentation parsing (\docs.rs\), code generation (\generator.rs\), and derive input parsing (\parse\_derive.rs\). This change adds support for extracting and propagating \///\ and \//!\ doc comments from grammar files into the generated Rust code, allowing users to see documentation for rules in their IDE. It also enables \no\_std\ support for the generator via feature flags and introduces the \\#\[grammar\_inline\]\ attribute to allow defining grammars directly in source code without external files. Additionally, the generator now correctly handles the \\#\[non\_exhaustive\]\ attribute on the derived struct to apply it to the generated \Rule\ enum.
generator · high confidence
Initial release of the pest\_derive crate
The \derive/src\ location now contains the \pest\_derive\ crate, providing the \\#\[derive(Parser)\]\ macro that allows users to generate parsers from \.pest\ grammar files or inline grammar definitions. This entry establishes the foundational API for declarative parser generation in the project.
derive/src · high confidence
Introduce on-the-fly AST execution via the VM crate
The new \vm\ crate provides a virtual machine that executes optimized grammar ASTs at runtime rather than relying on generated code. This enables dynamic parsing capabilities, powering tools like the fiddle and debugger by allowing rules to be run on-the-fly. The implementation includes core parsing logic, built-in atomic rules (such as \ANY\, \EOI\, and character classes), and helper macros (\parses\_to\, \fails\_with\) for testing the VM's behavior.
vm/src · high confidence
Introduce pest\_debugger crate and CLI debugger
Adds a new \pest\_debugger\ library crate and a CLI-based debugger (\main.rs\) for stepping through pest grammar parsing. The debugger allows users to load grammars and inputs, set breakpoints on specific rules, and step through execution via commands like \run\, \continue\, and \breakpoint\. It exposes a \DebuggerContext\ API for programmatic integration and supports long command names and prefixes for easier CLI usage.
debugger · high confidence
Meta-grammar AST and parser implementation
The \meta/src\ module now defines the Abstract Syntax Tree (AST) for the pest meta-grammar, introducing \Rule\ and \Expr\ types to represent grammar structures and expressions. This includes support for rule modifiers (silent, atomic, compound atomic, non-atomic), expression operators (sequence, choice, predicates, repetitions), and new features like \PUSH\_LITERAL\, \PEEK\ slices, and node tagging. The module also contains the generated parser (\grammar.rs\) and the source grammar definition (\grammar.pest\) that parses these meta-grammar files, enabling the tool to interpret and validate pest grammar definitions.
meta/src · high confidence
New SQL grammar with Pratt parser and HTTP/TOML/JSON parsers added
The library now exposes four distinct grammars: a new SQL parser that uses a Pratt parser for correct operator precedence, an HTTP request parser, and sample parsers for JSON and TOML. The SQL grammar is adapted from the Picodata sbroad project to simulate SQLite-style queries. Additionally, call limits are enforced on the JSON and TOML fuzzing tests to prevent hangs on deeply nested inputs, and the JSON grammar rejects unescaped control characters.
grammars/src · high confidence
New derive examples for calculator and help-menu grammars
Added new example files in the derive/examples directory demonstrating how to use pest\derive with multiple grammar files and Pratt parsers. The calc example shows a calculator grammar (base.pest and calc.pest) supporting infix operators (+, -, \, /, ^), prefix negation, and postfix factorial, parsed using a Pratt parser to handle precedence. The help-menu example (help-menu.pest and help-menu.rs) demonstrates parsing CLI help text structures with optional, required, and choice arguments.
derive/examples · high confidence
Removals
Removal of StringInput implementation
The \StringInput\ struct and its module have been removed from the \src/inputs\ directory. This eliminates the previous in-memory string matching capability that allowed users to create inputs from \&str\ and perform case-sensitive character comparisons via the \matches\ method.
src/inputs · high confidence
Removal of the legacy RDP parser implementation
The \src/parsers\ module, which contained the \Rdp\ trait and the \impl\_rdp!\ macro for recursive-descent parsing, has been removed. This change eliminates the legacy parser interface, meaning users can no longer implement or use the \Rdp\ trait for parsing logic in this location.
src/parsers · high confidence
Removed legacy RDP-based parser implementation
The legacy recursive-descent parser (RDP) implementation has been removed from the library. This change deletes the \grammar!\ macro, the \Input\ and \Parser\ traits, and the \Rdp\ trait, along with their associated modules (\src/grammar.rs\, \src/input.rs\, \src/parser.rs\, and related exports in \src/lib.rs\). Users relying on the \impl\_rdp!\ macro and the manual rule-definition style provided by this legacy system will no longer have access to these APIs.
src · high confidence
Behavioural changes
Bootstrap process now generates grammar.rs from pest source
The bootstrap tool has been updated to explicitly generate the \grammar.rs\ file from the \grammar.pest\ source during the build process. This change ensures the generated parser code is created locally, addressing previous issues with \pest\_derive\ under workspaces on Windows and keeping the generated source file in the repository tree rather than relying solely on compile-time derivation.
bootstrap · high confidence
Include license files in the package
The package now includes Apache and MIT license files via symlinks, ensuring that licensing information is distributed with the package contents.
meta · medium confidence
Major internal refactoring of parser state, error handling, and stack management
The core parsing engine has been significantly restructured. The \ParserState\ now manages parsing logic directly, introducing a new \Stack\ type with snapshot/restore capabilities for backtracking and a \set\_call\_limit\ function to prevent stack overflows. Error reporting has been enhanced with a new \Error\ structure that includes detailed call stacks, parse attempts, and line/column location information. Additionally, a new \ConstPrattParser\ has been added alongside the existing runtime version to allow for compile-time operator precedence configuration, and benchmarks have been added to measure the performance of these new features.
pest/src · high confidence
New AST optimizer pipeline with dedicated optimization passes
The meta/optimizer module now implements a structured optimization pipeline for pest's ASTs, introducing a new OptimizedExpr type and a sequence of dedicated passes: concatenation of adjacent string literals, factorization of common prefixes in choices, list pattern optimization to reduce redundant matches, rotation of nested sequences/choices, inlining of skip patterns, unrolling of repetition quantifiers, and error-state restoration for stack-modifying expressions. This refactors the previous monolithic optimizer into modular components, improving maintainability and enabling targeted performance improvements for grammar parsing.
meta/src/optimizer · high confidence
Refactored iterator internals and added PairsBuilder for testing
The iterator subsystem in \pest/src/iterators\ has been restructured to improve performance and testability. A new \LineIndex\ component replaces the previous cursor-based approach for line/column calculations, using a binary-searchable offset table to speed up \Pair::line\_col\ and \Pairs::next\. The internal token representation has been replaced by a smaller, more efficient \QueueableToken\ enum that stores direct indices to paired tokens, reducing memory usage and lookup time. Additionally, a new \PairsBuilder\ utility has been added, allowing users to manually construct \Pairs\ from rules and byte offsets without running a parser, which simplifies unit testing for code that consumes parser output.
pest/src/iterators · high confidence
Repository restructured with new configuration, documentation, and licensing files
The repository has been reorganized to improve developer experience and project governance. A new .gitpod.yml file enables ready-to-go cloud development workspaces, while CONTRIBUTING.md and SECURITY.md provide clear guidelines for community involvement and vulnerability reporting. The project license has been updated from MPL-2.0 to the dual MIT/Apache-2.0 model, reflected in the new LICENSE-MIT and LICENSE-APACHE files. Additionally, standard tooling configurations (rustfmt.toml, typos.toml, codecov.yml) and automation scripts (release.sh, semvercheck.sh, update\_unicode.sh) have been added to streamline maintenance, testing, and Unicode data updates.
(repo-wide) · high confidence
Symlinked license and readme files in derive and grammars directories
The \derive\ and \grammars\ directories now use symbolic links for their license files (\LICENSE-APACHE\, \LICENSE-MIT\) and the readme file (\\_README.md\). These links point to the corresponding files in the parent directory, ensuring that license and documentation information is consistent across these subprojects without duplicating content.
derive, grammars · high confidence
Unicode data updated to version 18.0.0
The built-in Unicode property rules in pest have been regenerated to align with Unicode Standard version 18.0.0. This update refreshes the underlying trie data for binary properties (such as Emoji and Alphabetic), general categories, and scripts, ensuring that pattern matching against character classes reflects the latest official Unicode definitions.
pest/src/unicode · high confidence
Fixes
Add license and README symlinks to pest crate
The pest crate now includes symlinks to the project's LICENSE-APACHE, LICENSE-MIT, and README files. This ensures that the license information and documentation are accessible within the crate directory, resolving issues with broken or missing references in sub-crates.
pest · high confidence
Test coverage
Added HTTP grammar benchmark and test data fixtures; Added VM test coverage for grammar rules, lists, reporting, and surrounding; Added comprehensive test suite for pest\_derive grammar parsing and built-in rules; Added fuzzing targets for HTTP, SQL, JSON, and TOML grammars; Added fuzzing test fixtures for grammar parsing; Added test coverage for HTTP, JSON, SQL, and TOML grammars; Added test coverage for calculator and JSON parsing.
Dependencies
Pest 2.9.2 workspace and dependency updates
The pest parser library has been updated to version 2.9.2 across its workspace crates (pest, pest\_derive, pest\_generator, pest\_meta, pest\_vm, pest\_debugger, pest\_grammars). This release includes a MSRV bump to Rust 1.83, updates to the \syn\ dependency (v3.0 in the generator), and the addition of \miette\ error support via the \miette-error\ feature. The \pest\_debugger\ crate now depends on \rustyline\ 13 for improved terminal interaction and \thiserror\ 2 for error handling. Fuzzing targets for JSON, HTTP, TOML, and SQL grammars have been added to the \pest\_grammars\ crate.
(dependencies) · high confidence
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
How this codebase got here
This is the PUBLIC form of this artifact. Findings are listed in full, but the details of SECURITY findings — which rule fired, in which file, on which line, and how to fix it — are deliberately withheld, and any secret-scanner results are excluded entirely. Where detail is absent here it was REMOVED FOR PUBLICATION; it is not missing from the analysis. The complete artifact is available from the repository owner.
Score
- CAI 56 → 56 (-0.5)
- Rubric changed (rubric-2026.10.5 → rubric-2026.11.4) — scores are not directly comparable.
Lenses
- Code Health 94 → 90 (-4.6)
- Architecture 98 → 98 (+0.0)
- Maturity 58 → 58 (+0.6)
- Readiness 41 → 44 (+3.3)
- Security 71 → 57 (-13.9)
- Performance 85 → 85 (+0.0)
Resolved (24)
- Further sole-owners (lower concentration)
- High: security finding (details withheld)
- High: security finding (details withheld)
- High: security finding (details withheld)
- High: security finding (details withheld)
- High: security finding (details withheld)
- High: security finding (details withheld)
- High: security finding (details withheld)
- High: security finding (details withheld)
- High: security finding (details withheld)
- High: security finding (details withheld)
- High: security finding (details withheld)
- High: security finding (details withheld)
- High: security finding (details withheld)
- High: security finding (details withheld)
- High: security finding (details withheld)
- High: security finding (details withheld)
- Medium: security finding (details withheld)
- No ADRs found
- Off-boarding risk: anonymized user #1
- …and 4 more
New (56)
- Duplicated block (13 lines × 5) (meta/src/parser.rs)
- Duplicated block (23 lines × 3) (meta/src/parser.rs)
- Duplicated block (6 lines × 3) (meta/src/optimizer/unroller.rs)
- Duplicated block (7 lines × 2) (meta/src/optimizer/unroller.rs)
- Duplicated block (8–9 lines × 2) (vm/src/lib.rs)
- Edited copies of one member (3 members, 15+ corresponding lines) (pest/src/iterators/flat_pairs.rs)
- Edited copies of one member (3 members, 18+ corresponding lines) (meta/src/validator.rs)
- Edited copies of one member (3 members, 9+ corresponding lines) (pest/src/parser_state.rs)
- Edited copy of a member (12 corresponding lines) (pest/src/pratt_parser.rs)
- Edited copy of a member (15 corresponding lines) (meta/src/validator/stack_loops.rs)
- Edited copy of a member (18 corresponding lines) (pest/src/position.rs)
- Edited copy of a member (180 corresponding lines) (generator/src/generator.rs)
- Edited copy of a member (21 corresponding lines) (debugger/src/main.rs)
- Edited copy of a member (3 corresponding lines) (meta/src/optimizer/mod.rs)
- Edited copy of a member (36 corresponding lines) (meta/src/validator.rs)
- Edited copy of a member (44 corresponding lines) (meta/src/optimizer/mod.rs)
- Edited copy of a member (5 corresponding lines) (pest/src/iterators/pair.rs)
- Edited copy of a member (5 corresponding lines) (pest/src/iterators/pairs.rs)
- Edited copy of a member (6 corresponding lines) (debugger/src/lib.rs)
- Edited copy of a member (6 corresponding lines) (pest/src/iterators/pair.rs)
- …and 36 more
Changes since last survey
- 2 commits — 2 feature/other, 0 fixes
By area
- .github/workflows — 1 commit
- pest/src — 1 commit
Notable commits
- change: Bump MSRV to 1.88 and migrate to edition 2024 (#1237)
- change: ci: add gh actions lock (#1239)
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
Survey your own repository
pest-parser/pest was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.
About this page
- The score is its most recent published measurement, taken on 14 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
- Measured at commit 7e14252f727e25e492749035ac299dbd860ebff2 — the exact code this score is about.
- Scored under rubric-2026.11.4 — the same rubric and the same method as every other entry in this index.
- Measured by Canine Code Analyzer (cca.canine.dev) using cca measure-2026.11.65+preprod-e1a3b69fc8ee.