antlr/antlr4

This system is the ANTLR 4 parser generator and its associated multi-language runtime libraries. It provides the core tooling to parse and transform grammars, alongside comprehensive runtime implementations for Java, C++, C\#, Dart, Go, JavaScript, Python, Swift, and TypeScript. The codebase includes utilities for tree traversal, pattern matching, and error handling, ensuring consistent behavior across all supported targets.

32 changes.

This period marks the comprehensive development of the ANTLR v4 tooling and Java runtime, introducing a new parsing infrastructure, code generation pipeline, and parse tree API. The work includes the removal of legacy v3 components, the implementation of left-recursion handling, and the addition of multi-language support for C\#, Swift, and others.

43 changes.

This period focused on significantly expanding ANTLR 4's language support by introducing new code generation templates and complete runtime implementations for Python 3, JavaScript, C++, Swift, and TypeScript. The work involved building out the full stack for these targets, including error handling, tree traversal, and build system integration, while also modernizing the existing Java and Go runtimes with parallel testing and ES6 refactoring.

38 changes.

This period focused on completing the ANTLR4 runtime implementations for Swift, Dart, and JavaScript, while significantly expanding test coverage and infrastructure for C++, C\#, Go, PHP, and Python. The work also included refactoring Unicode data generation and establishing automated release and deployment workflows.

The ANTLR v4 Tool is introduced, providing the core command-line interface for grammar processing. This includes support for specifying output directories, grammar file encodings, message formatting styles, and options for generating parse tree listeners, visitors, and dependency files. The tool also supports advanced debugging features such as launching a StringTemplate visualizer and enabling verbose logging.

tool/src/org/antlr/v4 · high confidence

The C++ runtime now includes a complete tree pattern matching implementation. Users can compile string-based tree patterns (e.g., \\<ID\> = \<expr\>;\) into \ParseTreePattern\ objects and use \ParseTreePatternMatcher\ to match them against \ParseTree\ structures. The API provides methods to retrieve matched nodes by label (\get\, \getAll\), access mismatched nodes, and check for successful matches. This enables programmatic querying of parse trees using symbolic patterns.

runtime/Cpp/runtime/src/tree/pattern · high confidence

The C++ runtime now includes a new synchronization abstraction (Synchronization.h/cpp) that provides Mutex, SharedMutex, and OnceFlag wrappers. These wrappers allow the runtime to use either the standard C++ library primitives or the Abseil library for thread synchronization, controlled by the ANTLR4CPP\_USING\_ABSEIL macro. This change enables users to integrate custom or alternative synchronization implementations.

runtime/Cpp/runtime/src/internal · high confidence

The C++ runtime now provides official build support for both CMake and Microsoft Visual Studio (VS2019 and VS2022). Users can now build the ANTLR C++ runtime as a shared or static library using CMake, with options to enable or disable specific build types. For Windows users, Visual Studio project files are provided, allowing direct compilation of the runtime within the Visual Studio IDE. This change simplifies the build process across different platforms and IDEs, ensuring consistent compilation and linking of the C++ runtime components.

runtime/Cpp/runtime · high confidence

The CMake build system now provides two installable packages: \antlr4-runtime\, which exports the C++ runtime library targets (\antlr4\_shared\ and \antlr4\_static\), and \antlr4-generator\, which provides the \antlr4\_generate\ function for code generation. Additionally, the \ExternalAntlr4Cpp\ module allows the C++ runtime to be built as an external project, supporting both static and shared library builds across Visual Studio, Xcode, and Makefile generators.

runtime/Cpp/cmake · high confidence

Added a new script and data files to parse the Unicode ExtendedPictographic property. The \parse.py\ script converts the Unicode \ExtendedPictographic.txt\ into a \ExtendedPictographic-Parsed.txt\ file containing \set.add\ calls for specific code points and ranges. This data is intended to be consumed by \UnicodeDataTemplateController\ to improve grapheme cluster, word, and line segmentation by ensuring ZWJ sequences do not break grapheme clusters.

scripts/parse-extended-pictographic · high confidence

A new C++ demo application for Linux has been added to the runtime, providing a complete example of how to use the ANTLR C++ runtime to lex and parse a sample input string containing Unicode characters. The demo includes the necessary headers, token stream processing, and parse tree output, serving as a practical reference for users building C++ projects with ANTLR.

runtime/Cpp/demo/Linux, runtime/Cpp/demo/Mac/antlr4-cpp-demo · high confidence

The C++ runtime is now distributed as three separate NuGet packages: a noarch package containing headers, a shared DLL package for x86/x64 Debug/Release builds, and a static library package for the same architectures. A new \pack.cmd\ script automates building the DLL and static variants and generates the \.nupkg\ files, enabling users to install the ANTLR4 C++ runtime directly via NuGet.

runtime/Cpp/runtime/nuget · high confidence

The Python 3 runtime now includes the core error handling infrastructure: the \ErrorListener\ interface and its \ConsoleErrorListener\ and \ProxyErrorListener\ implementations, the \ErrorStrategy\ base class with the \DefaultErrorStrategy\ for recovery and reporting, and specific exception classes (\RecognitionException\, \NoViableAltException\, \InputMismatchException\, etc.) for parser errors. This provides the necessary components for reporting and recovering from syntax errors in generated parsers.

runtime/Python3/src/antlr4/error · high confidence

The Python 3 runtime now includes a complete tree-matching and traversal API. Users can compile tree patterns (e.g., \\<ID\> = \<expr\>;\) and match them against parse trees using \ParseTreePatternMatcher\. The \ParseTreeMatch\ class provides methods to retrieve matched nodes by label (\get\, \getAll\) and check for success. Additionally, the \Trees\ utility class offers helper methods for tree traversal, including finding all nodes of a specific type or rule index, retrieving ancestors, and printing tree structures in LISP-like format. This brings feature parity with other ANTLR targets for tree pattern matching and inspection.

runtime/Python3/src/antlr4/tree · high confidence

The Swift runtime now includes a \boot.py\ script that enables developers to generate an Xcode project and a Swift Package Manager (SPM) module for the ANTLR4 Swift runtime. This allows users to easily integrate the runtime into their Swift projects using standard Swift tooling, with generated parsers and test suites automatically configured.

runtime/Swift · high confidence

The Swift runtime now includes the full set of error handling components, including \ANTLRErrorListener\, \ANTLRErrorStrategy\, \BailErrorStrategy\, \DefaultErrorStrategy\, \BaseErrorListener\, and \ConsoleErrorListener\. Additionally, core input stream classes like \ANTLRInputStream\ and \ANTLRFileStream\ are implemented to handle character and file-based input. These additions provide the necessary infrastructure for parsing, error reporting, and input management in Swift applications using ANTLR.

runtime/Swift/Sources · high confidence

Introduced a new TypeScript code generation target for ANTLR, providing templates to generate TypeScript parser, listener, and visitor classes. The implementation includes specific TypeScript features such as explicit 'override' specifiers for static members, 'null \| string' union types for optional string fields, and proper ES6 import statements for the antlr4 library.

tool/resources/org/antlr/v4/tool/templates/codegen/PHP, tool/resources/org/antlr/v4/tool/templates/codegen/Swift, tool/resources/org/antlr/v4/tool/templates/codegen/TypeScript · high confidence

The ANTLR tool now supports generating code for a wide range of programming languages. This change introduces new target implementations for C\#, C++, Dart, Go, JavaScript, Java, PHP, Python 3, Swift, and TypeScript. Each target defines language-specific reserved words, character escape sequences, and file naming conventions, enabling users to generate parsers and visitors for these platforms.

tool/src/org/antlr/v4/codegen/target · high confidence

The Dart runtime now includes support for matching parse trees against tree patterns, enabling users to extract specific nodes by label or rule/token type. This change introduces the \ParseTreePattern\ and \ParseTreeMatch\ classes, along with \Chunk\ and \TagChunk\ classes for parsing tree pattern syntax. Additionally, the \Trees\ utility class provides methods for traversing and querying parse trees, such as finding all nodes of a specific type or retrieving all descendants. These additions allow developers to perform complex tree-based queries and pattern matching on generated parse trees.

runtime/Dart/lib/src/tree · high confidence

The runtime adds new utility components to support multi-platform builds and improve code organization. This includes a \BitSet\ implementation with optimized bit-counting and scan-forward operations that use conditional imports to provide efficient 64-bit logic for native targets and fallback 32-bit logic for HTML targets. Additionally, a \MurmurHash\ utility is introduced for computing hash codes, and platform-specific I/O wrappers (\platform\_io.dart\, \platform\_html.dart\, \platform\_stub.dart\) are added to abstract file reading and console output across different Dart environments.

runtime/Dart/lib/src/util · high confidence

Added DOT source files for ATN state diagrams, including Block, Closure (Greedy/NonGreedy), Optional, PositiveClosure, Rule, and various merge scenarios (ArrayMerge, FullMerge, LocalMerge, SingletonMerge). These diagrams illustrate the structure and relationships of ANTLR4 runtime states, supporting the documentation of the Abstract Syntax Tree (AST) and ATN state relations.

runtime/Java/src/main · high confidence

The C++ runtime now includes a new \tree/xpath\ module that enables querying parse trees using a subset of XPath syntax. This feature allows users to select nodes in the parse tree by rule name, token name, or wildcard, supporting operators for root (\/\), anywhere (\//\), and inversion (\!\). The implementation consists of an ANTLR-generated lexer (\XPathLexer\) and a set of C++ classes (\XPath\, \XPathElement\, and specific element types like \XPathRuleElement\ and \XPathTokenElement\) that parse and evaluate these paths against a given \ParseTree\.

runtime/Cpp/runtime/src/tree/xpath · high confidence

The Dart runtime now includes a complete implementation of the Abstract Syntax Tree (ATN) infrastructure, including core classes such as \ATN\, \ATNConfig\, \ATNConfigSet\, and \ATNState\. This addition brings the Dart runtime in line with other language targets by providing the necessary components for parsing and lexing, including support for lexer actions, simulation, and deserialization.

runtime/Dart/lib/src/atn · high confidence

The JavaScript runtime now includes new context classes to support parsing and prediction logic. This adds \RuleContext\ and \ParserRuleContext\ to represent parse tree nodes, along with \PredictionContext\, \ArrayPredictionContext\, \SingletonPredictionContext\, and \EmptyPredictionContext\ to manage parser state and lookahead information. These classes provide the necessary structure for the ANTLR JavaScript runtime to build and traverse parse trees and manage prediction contexts during parsing.

runtime/JavaScript/src/antlr4/context · high confidence

The JavaScript runtime now includes new utility classes: AltDict, BitSet, HashCode, HashMap, HashSet, Interval, and IntervalSet. These provide native JavaScript implementations for common data structures and bit manipulation, supporting features like error listener state tracking and interval management within the ANTLR runtime.

runtime/JavaScript/src/antlr4/misc · high confidence

Developers using the NetBeans IDE will now find pre-configured project settings in the new nb-configuration.xml file, which defines code formatting rules (such as 4-space tabs and no space after type casts) and a custom spell-checker wordlist. Additionally, a doxyfile has been added to configure the generation of Java API documentation using Doxygen, enabling consistent documentation output for the ANTLR v4 API.

runtime/Java · high confidence

New Xcode project scheme files have been added for the ANTLR C++ runtime, providing build configurations for the dynamic library (libantlr4-runtime.dylib), the iOS framework (antlr4\_ios.framework), and the static library (libantlr4-runtime.a). These changes ensure that developers using Xcode can correctly build, test, and archive the respective ANTLR runtime variants.

runtime/Cpp/runtime/antlrcpp.xcodeproj/xcshareddata · high confidence

The Dart runtime library has been expanded with new files to support the ANTLR parser generator's output. This includes the \DFA\ class and its serializer in \runtime/Dart/lib/src/dfa/\, which handle deterministic finite automaton states and serialization. Additionally, new files \runtime/Dart/lib/src/input\_stream.dart\, \runtime/Dart/lib/src/interval\_set.dart\, \runtime/Dart/lib/src/lexer.dart\, \runtime/Dart/lib/src/ll1\_analyzer.dart\, \runtime/Dart/lib/src/parser.dart\, \runtime/Dart/lib/src/parser\_interpreter.dart\, and \runtime/Dart/lib/src/parser\_rule\_context.dart\ have been added, providing the core parsing, lexing, and context management structures required for the Dart target.

runtime/Dart/lib/src · high confidence

The JavaScript runtime now includes several new utility modules in the \runtime/JavaScript/src/antlr4/utils\ directory. These additions include \DoubleDict\ for two-dimensional key-value storage, \arrayToString\ and \valueToString\ for serialization, \equalArrays\ and \standardEqualsFunction\ for deep array and object comparison, \escapeWhitespace\ for string escaping, \titleCase\ for text formatting, and \stringToCharArray\/\stringHashCode\ for string processing. Type definitions (\.d.ts\) are also provided for the new utilities.

runtime/JavaScript/src/antlr4/utils · high confidence

The tool-testsuite now runs tests concurrently using JUnit 5's parallel execution engine. This is enabled by a new \junit-platform.properties\ configuration file that sets \junit.jupiter.execution.parallel.enabled\ to true, allowing the test suite to leverage multi-core processors for faster test execution.

tool-testsuite · high confidence

The tool now includes new utility classes in the org.antlr.v4.misc package to support grammar processing. CharSupport handles character literal conversion and escape sequence parsing, including support for extended Unicode escapes (\\u{...}) and Unicode property escapes (\\p{...}). EscapeSequenceParsing provides a structured way to parse and validate escape sequences. Graph and OrderedHashMap provide deterministic, ordered map and graph traversal capabilities, while Utils offers helper methods for string manipulation and list operations. These changes improve how the ANTLR tool processes character sets, escape sequences, and internal data structures.

tool/src/org/antlr/v4/misc · high confidence

The C++ code generation templates (Cpp.stg, Files.stg) were added to the ANTLR tool, introducing new logic for generating C++ header and source files for lexers, parsers, and listener classes. This includes support for lexer channels, mode names, and package namespaces, enabling the C++ target to produce more complete and modern C++ code structures.

tool/resources/org/antlr/v4/tool/templates/codegen/Cpp · high confidence

The C++ runtime is now built using a comprehensive CMake build system (CMakeLists.txt) that enforces a minimum C++17 standard and supports multiple compilers including Clang, GCC, and MSVC. The build configuration includes options to statically link the CRT on Windows, disable compiler warnings, and optionally build demos. Additionally, CMake package files (antlr4-runtime and antlr4-generator) are generated for easier integration, and deployment scripts for macOS, Windows, and source distribution are provided.

runtime/Cpp · high confidence

The C++ runtime now includes explicit support for precedence DFAs, allowing the parser to track and retrieve start states for specific precedence levels via new \isPrecedenceDfa\, \getPrecedenceStartState\, and \setPrecedenceStartState\ methods on the \DFA\ class. Additionally, the runtime adds \DFA\ and \DFAState\ classes along with \DFASerializer\ and \LexerDFASerializer\ components, enabling the serialization of the deterministic finite automaton into a human-readable string representation for debugging and analysis.

runtime/Cpp/runtime/src/dfa · high confidence

The C++ runtime introduces new classes for error handling and input streams. The error handling API is expanded with \ANTLRErrorListener\ and \ANTLRErrorStrategy\ interfaces, along with concrete implementations like \BailErrorStrategy\ and \BaseErrorListener\. Additionally, \ANTLRInputStream\ and \ANTLRFileStream\ are added to handle UTF-8 encoded input, including automatic detection and removal of the UTF-8 byte order mark (BOM).

runtime/Cpp/runtime/src · high confidence

The Dart runtime now exposes the core ANTLR4 classes—including the recognizer, parser, lexer, and related utilities—via the main library entry point. This allows users to import a single package to access the full set of runtime components needed for parsing and lexing.

runtime/Dart/lib · high confidence

The Dart runtime now exposes the MultiMap and Pair classes, which were previously internal or missing from the public API. Users can now import and use these utilities directly from the misc module, enabling easier handling of multi-valued maps and key-value pairs in their applications.

runtime/Dart/lib/src/misc · medium confidence

The Dart runtime now exposes the internal error handling components for grammar parsing. Users can now access the \ErrorListener\ interface and its \DiagnosticErrorListener\ implementation to receive detailed reports on parser ambiguities, context sensitivity, and full-context decision conflicts. Additionally, the \ErrorStrategy\ interface and its \DefaultErrorStrategy\ implementation are exported, allowing for custom error recovery and reporting logic. The core exception types (\RecognitionException\, \NoViableAltException\, \InputMismatchException\, etc.) are also exported, enabling users to catch and handle specific parsing errors.

runtime/Dart/lib/src/error · high confidence

Adds the initial C\# target templates (CSharp.stg) to the ANTLR tool, enabling the generation of C\# parser, listener, base listener, and visitor code. This includes the structural definitions for namespaces, interfaces, and classes required for C\#-based grammars.

tool/resources/org/antlr/v4/tool/templates/codegen/CSharp · high confidence

The antlr4-maven-plugin is added to the project, providing a new Maven goal (antlr4) that invokes the ANTLR 4 tool to generate Java source files from .g4 grammar files. The plugin supports configuring the ANTLR tool via parameters such as -listener, -visitor, and -Doption=value, and includes dependency tracking to ensure grammars are reprocessed when referenced files change. Error and warning messages from the ANTLR tool are reported through the Maven log and the Maven BuildContext for IDE integration.

antlr4-maven-plugin/src/main · high confidence

The \runtime/Java/src/org/antlr/v4/runtime/tree/xpath\ package now includes a new \XPath\ class and supporting classes (\XPathElement\, \XPathLexer\, etc.) that allow users to query ANTLR parse trees using a subset of XPath XML path syntax. This feature enables users to find nodes in a parse tree by specifying paths such as \/root\, \//anywhere\, or \!\ for inversion, providing a more convenient way to navigate and filter parse tree structures compared to manual recursion.

runtime/Java/src/org/antlr/v4/runtime/tree/xpath · high confidence

Added new classes in the analysis package to detect and transform left-recursive rules. The \AnalysisPipeline\ orchestrates the process, delegating to \LeftRecursionDetector\ to identify recursive cycles and \LeftRecursiveRuleAnalyzer\ to classify and analyze the structure of left-recursive rules. The \LeftRecursiveRuleTransformer\ then rewrites the grammar AST to remove left recursion, enabling the rest of the compiler to process the rules correctly.

tool/src/org/antlr/v4/analysis · high confidence

Added a complete set of new grammar files and Java classes to the \tool/src/org/antlr/v4/parse\ directory, including \ANTLRLexer.g\, \ANTLRParser.g\, \ATNBuilder.g\, \ActionSplitter.g\, \BlockSetTransformer.g\, \GrammarTreeVisitor.g\, and \LeftRecursiveRuleWalker.g\, alongside supporting Java classes like \GrammarASTAdaptor\ and \ScopeParser\. This change establishes the foundational components for parsing ANTLR v4 grammars, replacing the previous v3-based parsing logic with a dedicated v4 parser and tree visitor architecture.

tool/src/org/antlr/v4/parse · high confidence

Added new interfaces and base classes for error handling, including ANTLRErrorListener, ANTLRErrorStrategy, and BaseErrorListener, which provide a more structured way to handle syntax errors, report ambiguities, and manage error recovery. Additionally, introduced the CharStreams utility class and updated the CharStream interface to support getting text from marked ranges, while deprecating older input stream classes like ANTLRInputStream and ANTLRFileStream in favor of the new CharStreams API.

runtime/Java/src/org/antlr/v4/runtime · high confidence

The Java runtime now includes a complete system for serializing and deserializing Abstract Syntax Tree (ATN) data. This introduces new classes including ATNDeserializer, ATNSerializer, and ATNDeserializationOptions, which handle the conversion of ATN structures to and from a serialized integer-based format. The ATN class is updated to support this serialization, and the ATNConfig and ATNConfigSet classes are modified to support the new serialization format, including the addition of a 'readonly' flag to the configuration set to prevent modification after it is used as a DFA state.

runtime/Java/src/org/antlr/v4/runtime/atn · high confidence

The DFA implementation now supports precedence decisions. A new \precedenceDfa\ flag and associated methods (\isPrecedenceDfa\, \getPrecedenceStartState\, \setPrecedenceStartState\) allow the parser to maintain separate DFA states for different precedence levels, enabling correct handling of left-recursive rules and precedence-based disambiguation.

runtime/Java/src/org/antlr/v4/runtime/dfa · high confidence

The JavaScript target for ANTLR 4 is now supported, providing templates that generate ES6 classes for parsers, lexers, listeners, and visitors. This enables users to produce JavaScript code that integrates with the antlr4 runtime, including support for named actions, semantic predicates, and custom superclasses.

tool/resources/org/antlr/v4/tool/templates/codegen/Dart, tool/resources/org/antlr/v4/tool/templates/codegen/JavaScript · high confidence

The JavaScript runtime now includes a complete build and configuration setup to support both Node.js and browser environments. This includes a Webpack configuration that generates CommonJS and ES Module bundles for both web and Node targets, a TypeScript configuration for type checking, and Babel settings to ensure compatibility with modern JavaScript features. Additionally, ESLint and code coverage (c8) configurations have been added to standardize code quality and testing processes.

runtime/JavaScript · high confidence

The JavaScript runtime now includes full class implementations and TypeScript type declarations for all error handling components in the \runtime/JavaScript/src/antlr4/error\ directory. This adds \ErrorListener\, \DiagnosticErrorListener\, \BailErrorStrategy\, \DefaultErrorStrategy\, and all related exception classes (e.g., \NoViableAltException\, \InputMismatchException\) with corresponding \.d.ts\ type definitions, enabling static type checking for error strategy and listener usage in JavaScript/TypeScript projects.

runtime/JavaScript/src/antlr4/error · high confidence

The C++ runtime now includes a complete demo application for building and running a test parser and lexer. This adds a CMake build configuration (CMakeLists.txt) that generates source files from the demo grammars (TLexer.g4, TParser.g4) and compiles them into an executable linked against the static ANTLR4 C++ library. The entry also includes platform-specific build scripts (generate.sh, generate.cmd, Mac/build.sh) and a README explaining how to generate and compile the demo, providing a concrete example of using the C++ target.

runtime/Cpp/demo · high confidence

The C++ runtime now includes new components in the \misc\ directory: \InterpreterDataReader\ to parse generated interpreter data files, \Interval\ and \IntervalSet\ for efficient range and set operations, \MurmurHash\ for hashing, and \Predicate\ for parse tree evaluation. These additions support the new option to generate and read interpreter data, improving parsing performance and enabling advanced features like rule name resolution and token stringification.

runtime/Cpp/runtime/src/misc · high confidence

The ANTLR tool now includes a new \TestRig\ command-line utility for testing lexer and parser grammars, supporting options to print parse trees, generate PostScript output, and enable diagnostics. Additionally, the \Trees\ utility class now provides static methods to inspect parse trees in a GUI window or save them as PostScript files, and the \Interpreter\ class has been added to allow interactive interpretation of grammars for debugging purposes.

tool/src/org/antlr/v4/gui · high confidence

The code generator now uses a new set of model classes in the \org.antlr.v4.codegen.model.chunk\ package to represent grammar elements during code generation. This includes \ActionChunk\ and its subclasses (\ActionTemplate\, \ActionText\) for handling code actions, \SymbolRefChunk\ and its subclasses (\LocalRef\, \LabelRef\, \TokenRef\, etc.) for referencing symbols, and \RulePropertyRef\ and its variants for accessing rule properties. These classes provide a structured way to map grammar constructs to generated code.

tool/src/org/antlr/v4/codegen/model/chunk · high confidence

The Java runtime introduces a new parse tree API centered around the \ParseTree\ interface and its implementations (\RuleNode\, \TerminalNode\, \ErrorNode\). This change provides a structured way to traverse and process parse trees using the \ParseTreeVisitor\ and \ParseTreeListener\ interfaces. Users can now implement visitors to transform or analyze the tree structure, or use listeners to react to tree events. The \IterativeParseTreeWalker\ is added as the default walker to handle deep trees without stack overflow. Utility methods in \Trees\ class support tree traversal and inspection.

runtime/Java/src/org/antlr/v4/runtime/tree · high confidence

Added several new scripts to the repository to automate release and deployment workflows. The \update\_antlr\_version.py\ script automates updating version numbers across multiple runtime files listed in \files-to-update.txt\. The \deploy\_to\_website.py\ script handles updating the website with new Javadoc and JAR files, and updating version strings on the site. Additionally, \github\_release\_notes.py\ generates release notes by fetching GitHub issues and pull requests, and \traceatn.sh\ provides a shell script for tracing ANTLR Abstract Syntax Trees.

scripts · high confidence

The semantic pipeline now includes new classes (ActionSniffer, AttributeChecks, BasicSemanticChecks, SymbolChecks, SymbolCollector, UseDefAnalyzer) that perform detailed validation of grammar rules, attributes, and symbols. This enables the tool to detect and report errors such as undefined rule references, label conflicts, and invalid attribute expressions during compilation.

tool/src/org/antlr/v4/semantics · high confidence

Added three new template files to the ANTLR v4 tool: LeftRecursiveRules.stg, which defines how left-recursive rules are generated; depend.stg, which provides templates for generating make-compatible dependency information; and unicodedata.st, which generates a Java utility class for mapping Unicode properties to code point ranges. These templates support the new left-recursion transformation logic and Unicode data handling.

tool/resources/org/antlr/v4/tool/templates · high confidence

The Java runtime introduces a new tree pattern matching system in the \org.antlr.v4.runtime.tree.pattern\ package. Users can now define patterns using special tags (e.g., \\<expr\>\) to match against parse trees. This includes the \ParseTreePatternMatcher\ for compiling and matching patterns, \ParseTreePattern\ to represent compiled patterns, and \ParseTreeMatch\ to hold match results including labels and mismatched nodes. The API supports finding all matches using XPath expressions and provides methods to retrieve matched subtrees by label.

runtime/Java/src/org/antlr/v4/runtime/tree/pattern · high confidence

The ANTLR Java runtime adds several new utility classes to the \org.antlr.v4.runtime.misc\ package to improve performance and memory management. This includes \Array2DHashSet\ and \FlexibleHashMap\ for efficient set and map operations with custom equality comparators, as well as \IntegerList\ and \IntegerStack\ for memory-efficient integer storage. Additionally, \IntervalSet\ and \IntSet\ provide optimized representations for sets of integers, while \MurmurHash\ implements a fast hashing algorithm. The update also introduces \InterpreterDataReader\ for parsing interpreter data files, \LogManager\ for logging, and \ParseCancellationException\ for handling parse cancellations. These changes enhance the runtime's capabilities for handling grammar analysis and execution data.

runtime/Java/src/org/antlr/v4/runtime/misc · high confidence

The Python 3 runtime for the ANTLR parser generator has been added to the codebase. This includes the core ATN (Augmented Transition Network) structures such as ATN, ATNConfig, and ATNConfigSet, along with the LexerATNSimulator and associated lexer action classes. This provides the necessary components for the Python 3 target to perform lexical and syntactic analysis.

runtime/Python3/src/antlr4, runtime/Python3/src/antlr4/atn · high confidence

The Python 3 target templates (Python3.stg) are now included in the ANTLR 4 runtime, providing the foundational structure for generating Python 3 parser, listener, and visitor code. This includes the initial implementation of the template file that defines how Python 3 source code is constructed, including imports, class definitions, and method signatures.

tool/resources/org/antlr/v4/tool/templates/codegen/Python3 · high confidence

The Python3 runtime now includes a new XPath implementation in the \runtime/Python3/src/antlr4/xpath\ directory. This adds the \XPath\ class and its associated lexer (\XPathLexer\), enabling users to query and navigate parse trees using XPath-like path expressions. The implementation supports standard XPath operators such as \/\ (root), \//\ (anywhere), \!\ (invert), and \\*\ (wildcard), allowing developers to find nodes in the parse tree that match specific rules, tokens, or wildcards.

runtime/Python3/src/antlr4/xpath · high confidence

The gUnit v3 test framework files have been removed from the codebase. This includes the \gUnit.g\ grammar, the \gUnitLexer\ and \gUnitParser\ Java classes, and the supporting \ASTVerifier\, \Gen\, \Interp\, \Semantics\, and \jUnitGen\ components. This cleanup removes the older test generation and verification tools, likely in preparation for or following the introduction of gUnit v4.

gunit/src · high confidence

The code generation model classes have been reorganized into the new org.antlr.v4.codegen.model.decl package. This refactoring introduces a structured hierarchy of declaration classes—including RuleContextDecl, StructDecl, and various getter declarations (ContextGetterDecl, TokenDecl, etc.)—to better represent the components of generated parser code. This change improves the internal organization of the code generator, making it easier to maintain and extend for different target languages.

tool/src/org/antlr/v4/codegen/model/decl · high confidence

The C++ runtime has been refactored to improve performance and memory management. The \ATN\ class now caches the set of next tokens for each state to avoid redundant computation. \ATNConfig\ and \ATNConfigSet\ have been restructured to support context merging and better tracking of semantic contexts and outer context depth. Additionally, \ATNDeserializer\ and \ATNState\ have been updated to support new serialization options and state types, while \ATNSimulator\ and related classes have been adjusted to support the new configuration model.

runtime/Cpp/runtime/src/atn · high confidence

The separate DOT template files for generating graph visualizations have been consolidated into a single group file (graphs.stg). This change improves the readability and maintainability of the template logic, making it easier to understand the interactions between different graph components such as ATN, DFA, and state rendering.

tool/resources/org/antlr/v4/tool/templates/dot · medium confidence

The runtime testsuite now runs tests concurrently by default. A new configuration file, junit-platform.properties, has been added to the resources directory, setting JUnit 5 properties to enable parallel execution for all test classes and methods.

runtime-testsuite/resources · high confidence

The Go ANTLR 4 runtime is now available as a proper Go module at \runtime/Go/antlr/v4\, enabling standard \go get\ usage and versioning. This release includes significant performance improvements, such as fixing comparison functions that were previously comparing pointers instead of values, and correcting the PredictionContext cache which had stopped working. The update also includes fixes for \CommonTokenStream\ reset behavior, proper handling of file input stream sizes in runes, and various internal refactors like removing unnecessary interfaces and getters/setters to improve memory usage and speed.

runtime/Go/antlr/v4 · high confidence

The Go code generation templates have been updated to import the Go runtime from the new repository path github.com/antlr4-go/antlr/v4. This change ensures that generated Go code imports the correct package, aligning with the standardization of the Go runtime's location.

tool/resources/org/antlr/v4/tool/templates/codegen/Go · high confidence

The tool module now introduces a new factory-based approach for building the Abstract Syntax Tree (AST) into an Abstract State Machine (ATN), with dedicated factories for parsers and lexers. A new ATN optimizer removes unnecessary epsilon transitions and collapses multiple alternatives into single set transitions, resulting in smaller and more efficient ATNs. The changes also include improved error handling for empty sets, non-greedy loop handling, and better support for lexer commands and character sets.

tool/src/org/antlr/v4/automata · medium confidence

The code generation model has been significantly restructured, introducing new classes such as Action, Choice, and various block types (e.g., LL1AltBlock, LL1Loop) to represent grammar elements. This change implements a new model-based approach for generating code, which includes support for left-recursive rules, exception clauses, and improved handling of labels and contexts. The refactoring aims to make the code generation process more modular and maintainable, with a focus on separating concerns between the model and the templates.

tool/src/org/antlr/v4/codegen/model · medium confidence

The code generation architecture has been significantly reorganized to support a new output model. A new \CodeGenPipeline\ class now orchestrates the generation of parsers, lexers, listeners, and visitors. The \CodeGenerator\ has been refactored to use a factory pattern (\OutputModelFactory\) to create model objects, which are then walked by \OutputModelWalker\ to produce templates. This includes new classes like \ActionTranslator\ for handling semantic actions, \BlankOutputModelFactory\ providing default implementations, and \CodeGeneratorExtension\ for post-processing model objects. These changes enable more flexible and extensible code generation, particularly for supporting multiple target languages and header files.

tool/src/org/antlr/v4/codegen · medium confidence

The Python 3 runtime package now includes a MANIFEST.in file to ensure test files and text resources are properly distributed in the source distribution. Additionally, README.txt and RELEASE-4.5.txt files have been added, documenting the runtime's purpose and listing the changes in version 4.5, which include a fix for non-ascii input streams, support for the visitor pattern and wildcards, and a breaking change removing the tokenNames member in favor of lexicalNames and symbolicNames.

runtime/Python3 · medium confidence

The C++ runtime support library has been restructured and modernized. The custom \antlrcpp::Any\ class has been replaced with the standard library's \std::any\. Utility functions for string manipulation, UTF-8 encoding/decoding, and array handling have been reorganized into new headers and source files (e.g., \Any.h\, \Arrays.h\, \CPPUtils.h\, \StringUtils.h\, \Utf8.h\). This includes adding standalone Unicode encoding and decoding handling, simplifying memory management, and improving thread safety with \thread\_local\ storage for converters.

runtime/Cpp/runtime/src/support · high confidence

The ANTLR JavaScript runtime's DFA (Deterministic Finite Automaton) components have been refactored from legacy function prototypes and CommonJS exports into modern ES6 classes and ES modules. This change updates the internal structure of the DFA, DFASerializer, LexerDFASerializer, and PredPrediction classes to use \class\ syntax and \import\/\export\ statements, improving code maintainability and aligning with modern JavaScript standards.

runtime/JavaScript/src/antlr4/dfa · high confidence

The tool now uses a new \UnicodeDataTemplateController\ to generate parameters for the \unicodedata.st\ template, which produces \UnicodeData.java\. This change replaces the previous static class generation for properties and property aliases, consolidating the logic for Unicode categories, binary properties, script codes, and aliases (including Extended Pictographic and EmojiDefault) into a single controller that feeds data to the template engine.

tool/src/org/antlr/v4/unicode · medium confidence

The C++ runtime's parse tree and visitor infrastructure has been restructured to use \std::any\ for visitor results, replacing the previous template-based approach to avoid virtual template functions. This change introduces new header and source files for \AbstractParseTreeVisitor\, \ErrorNode\, \IterativeParseTreeWalker\, \ParseTree\, \ParseTreeListener\, \ParseTreeProperty\, \ParseTreeType\, \ParseTreeVisitor\, \ParseTreeWalker\, \TerminalNode\, and \Trees\ utilities. The \IterativeParseTreeWalker\ now provides an iterative (non-recursive) tree walking mechanism to handle deeply nested parse trees without stack overflow, while the visitor pattern is updated to support the new \std::any\ return type, allowing for more flexible and type-safe visitor implementations.

runtime/Cpp/runtime/src/tree · high confidence

The JavaScript ANTLR runtime's ATN (Abstract Syntax Tree) components have been refactored to use ES6 classes, modern variable scoping (const/let), and object destructuring. This includes converting classes like ATN, ATNConfig, ATNConfigSet, and simulators to native ES6 syntax, fixing export issues, and ensuring consistent hashing and equality checks for configuration sets.

runtime/JavaScript/src/antlr4/atn · high confidence

The JavaScript runtime's parse tree classes (such as ParseTree, RuleNode, TerminalNode, ErrorNode, and their implementations) have been refactored from prototype-based structures to ES6 classes. Additionally, the ParseTreeVisitor now returns values from visit methods, and the ParseTreeWalker performs a depth-first traversal by default, ensuring that visit methods for terminal and error nodes are properly invoked during tree walking.

runtime/JavaScript/src/antlr4/tree · high confidence

The tool's error reporting system has been refactored to use a structured \ANTLRMessage\ object and an \ANTLRToolListener\ interface. This change replaces the previous direct printing of error strings with a more robust mechanism where errors, warnings, and info messages are wrapped in \ANTLRMessage\ objects and passed to a listener. This allows external tools and IDEs to intercept and process error messages more effectively, as the error data is now encapsulated in a dedicated class rather than being printed directly to standard error.

tool/src/org/antlr/v4/tool · high confidence

The jUnit.stg template used for generating ANTLR v4 gunit tests has been removed. This template previously provided the structure for Java-based unit tests, including the class definition, setup methods, and test case generation. Its removal indicates a shift in how ANTLR v4 gunit tests are structured or generated, likely moving away from this specific template-based approach.

gunit/resources · medium confidence

The \tool/src/org/antlr/v4/tool/ast\ package was reorganized to introduce a new \GrammarASTVisitor\ interface and refactor existing node classes. Key changes include the introduction of \GrammarASTWithOptions\ as a base class for nodes that carry options, and the addition of a \QuantifierAST\ interface to track greedy vs. non-greedy quantifiers in \OptionalBlockAST\, \PlusBlockAST\, and \StarBlockAST\. The \AltAST\ node was updated to track left-recursion transformation info and alternative labels, while \GrammarAST\ gained methods to retrieve outermost alternative nodes and alt labels. These changes support the internal restructuring of left-recursive rule transformation and improve how the tool processes grammar syntax trees.

tool/src/org/antlr/v4/tool/ast · medium confidence

The C\# runtime has been restructured, introducing new classes for handling character input streams (AntlrFileStream, AntlrInputStream) and core ATN (Abstract Syntax Tree) components (ATN, ATNConfig, ATNConfigSet, ATNState, etc.). This change likely improves memory management and parsing performance by optimizing how the parser tracks and merges prediction contexts and configuration states during the parsing process.

runtime/CSharp · medium confidence

The repository now includes standardized configuration files to improve consistency and contributor experience. A \.clang-format\ file enforces C++17 formatting rules, while \.editorconfig\ defines consistent indentation and encoding for Java, C++, and other source files. A default \.gitattributes\ file ensures consistent line-ending handling across platforms. The \.gitignore\ file has been updated to exclude build artifacts, IDE settings, and generated code from version control. Additionally, the project has adopted the Linux Foundation's Developer Certificate of Origin (DCO) for contributor agreements, replacing the previous certificate of origin, and includes a new \ANTLR-HOUSE-RULES.md\ document outlining community conduct expectations.

(repo-wide) · high confidence

The ANTLR C++ Windows demo project has been updated to support Visual Studio 2022, enabling developers to build and run the example application using the latest Visual Studio toolset. The update includes new project configuration files (.vcxproj and .sln) that configure the demo to compile as both a static and a dynamic library, supporting Debug and Release builds for x86 and x64 architectures.

runtime/Cpp/demo/Windows · medium confidence

The Xcode project file (project.pbxproj) for the ANTLR C++ runtime has been updated to reflect the current state of the source code. This ensures that the Xcode build environment correctly includes all relevant header and source files, such as those for ATN, lexer actions, and error handling, maintaining synchronization between the IDE project and the underlying codebase.

runtime/Cpp/runtime/antlrcpp.xcodeproj · medium confidence

The message formatting templates for ANTLR v4 have been updated to support multiple output styles. New template files have been added for 'gnu' (mimicking GCC output), 'vs2005' (Visual Studio 2005 format), and 'antlr' (default format). These changes alter how compiler errors and warnings are displayed in the console or IDE, allowing users to choose a format compatible with their build tools or editors.

tool/resources/org/antlr/v4/tool/templates/messages · high confidence

The Java code generation templates (Java.stg) have been updated to reflect the current state of the ANTLR 4 Java target. This includes the addition of a BSD license header, the inclusion of a \javaTypeInitMap\ for default value initialization, and the generation of listener and visitor interfaces/classes with proper package declarations, imports, and Javadoc comments. The templates now support generating leaner code by conditionally including specific imports and class structures based on grammar options.

tool/resources/org/antlr/v4/tool/templates/codegen/Java · medium confidence

The JavaScript runtime has been updated to version 4.13.2, introducing several behavioral changes and fixes. The \CharStream\ implementation now supports decoding input as Unicode code points, enabling correct handling of characters outside the Basic Multilingual Plane (e.g., emojis or rare CJK characters) in both Node.js and browser environments. Additionally, the \FileStream\ class now explicitly throws an error when instantiated in a non-Node.js environment, clarifying platform compatibility. The \TokenStreamRewriter\ has been ported to JavaScript, allowing for programmatic modification of token streams. These changes improve Unicode support, clarify environment constraints, and expand the toolkit for token manipulation.

runtime/JavaScript/src/antlr4 · medium confidence

The Xcode project file (project.pbxproj) and associated workspace and scheme files for the Mac demo have been updated to reflect structural changes in the C++ runtime. This includes reorganizing the project's include structure, fixing build configurations for the demo and its unit tests, and ensuring the project correctly links against the antlr4-runtime framework. These changes ensure the Mac demo builds and runs correctly with the latest C++ runtime changes.

runtime/Cpp/demo/Mac/antlrcpp-demo.xcodeproj · high confidence

The Xcode project's workspace configuration has been updated to include a new workspace file (contents.xcworkspacedata) and an IDE workspace checks plist (IDEWorkspaceChecks.plist) that explicitly enables the 32-bit warning suppression. This change reflects the split of the Xcode demo and libraries into individual projects, ensuring the workspace correctly references the self-project and applies the necessary build settings for the C++ runtime.

runtime/Cpp/runtime/antlrcpp.xcodeproj/project.xcworkspace · medium confidence

The ANTLR 4 Maven plugin documentation has been updated to reflect the new v4 behavior, including default directory structures (src/main/antlr4 and src/main/antlr4/imports), configuration examples for the antlr4 goal, and explanations for handling imported grammars and vocabulary files. This provides users with accurate guidance on setting up the plugin in their Maven projects.

antlr4-maven-plugin/src/site/apt · high confidence

The Dart benchmarking tool previously crashed when parsing high index options. This fix resolves the crash, allowing the benchmark to run correctly with high index configurations.

runtime/Dart/benchmark · medium confidence

Add C++ runtime test runner and test suite; Added C\# and Dart runtime test suites; Added Go runtime test suite; Added Java runtime API tests for token streams, visitors, and grammar parsing; Added Java runtime test infrastructure and unit tests; Added JavaScript runtime tests; Added JavaScript tests for TokenStreamRewriter; Added Linux test runner for Swift runtime tests; Added Mac unit tests for ANTLR C++ runtime; Added Python 3 test suite for the ANTLR runtime; Added Swift runtime tests for core components; Added performance tests for lexer speed and memory footprint; Added runtime test templates for C\#, C++, Dart, Go, Java, JavaScript, PHP, Python 3, Swift, and TypeScript; Added test coverage for JavaScript runtime collections; Added test harness templates for all supported runtimes; Added test runners for PHP and Python 3 runtimes; Added test state classes for the Java runtime; Added tests for CodePointCharStream behavior; Added tests for TypeScript support with different compiler options; Added tests for UTF-8 encoding and decoding; Added unit tests for the ANTLR4 Maven plugin; Added unit tests for the Dart BitSet utility; Introduce new runtime test infrastructure and test descriptors.

Added build system files to support multiple language targets and tooling. This includes a Swift Package Manager manifest (Package.swift) defining static and dynamic library products; Maven POMs for the Java runtime, the Maven plugin, and test suites; C\# project files (.csproj) for the .NET runtime and tests; a Dart pubspec.yaml; a Go module definition (go.mod/sum); and package.json files for the JavaScript runtime and its import tests. These files establish the build, dependency, and packaging configurations for these specific components.

(dependencies) · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• CAI 60 → 59 (-1.7)
• Rubric changed (rubric-2026.08.19 → rubric-2026.09.15) — scores are not directly comparable.

• Code Health 56 → 58 (+1.7)
• Architecture 94 → 92 (-2.6)
• Maturity 63 → 71 (+8.0)
• Readiness 60 → 52 (-8.4)
• Security 61 → 66 (+5.0)
• Performance 70 (new)

• Dependency hygiene not measured — no packages were read
• Dimension evaluation failed
• Duplicated block (10 lines × 2) (tool/src/org/antlr/v4/automata/ParserATNFactory.java)
• Duplicated block (10 lines × 2) (tool/src/org/antlr/v4/codegen/ParserFactory.java)
• Duplicated block (10 lines × 2) (tool/src/org/antlr/v4/gui/Interpreter.java)
• Duplicated block (10 lines × 2) (tool/src/org/antlr/v4/parse/ScopeParser.java)
• Duplicated block (12 lines × 2) (runtime/Java/src/org/antlr/v4/runtime/atn/ParserATNSimulator.java)
• Duplicated block (14 lines × 2) (tool/src/org/antlr/v4/gui/TreeViewer.java)
• Duplicated block (6 lines × 2) (runtime/Java/src/org/antlr/v4/runtime/atn/ATNConfig.java)
• Duplicated block (6 lines × 2) (runtime/Java/src/org/antlr/v4/runtime/atn/ParserATNSimulator.java)
• Duplicated block (6 lines × 2) (runtime/Java/src/org/antlr/v4/runtime/tree/Trees.java)
• Duplicated block (6 lines × 2) (tool/src/org/antlr/v4/semantics/AttributeChecks.java)
• Duplicated block (6 lines × 2) (tool/src/org/antlr/v4/tool/GrammarTransformPipeline.java)
• Duplicated block (7 lines × 2) (tool/src/org/antlr/v4/codegen/model/ElementFrequenciesVisitor.java)
• Duplicated block (7 lines × 2) (tool/src/org/antlr/v4/codegen/model/decl/AltLabelStructDecl.java)
• Duplicated block (7 lines × 5) (tool/src/org/antlr/v4/unicode/UnicodeDataTemplateController.java)
• Duplicated block (8 lines × 2) (runtime/Java/src/org/antlr/v4/runtime/LexerInterpreter.java)
• Duplicated block (8 lines × 2) (runtime/Java/src/org/antlr/v4/runtime/misc/InterpreterDataReader.java)
• Duplicated block (8 lines × 2) (tool/src/org/antlr/v4/analysis/LeftRecursiveRuleAnalyzer.java)
• Duplicated block (8 lines × 2) (tool/src/org/antlr/v4/tool/DOTGenerator.java)
• …and 48 more

• ATNConfigSet.applyPrecedenceFilter (cognitive 17) (runtime/Swift/Sources/Antlr4/atn/ATNConfigSet.swift)
• ATNDeserializer.CombineChainedEpsilons (cognitive 49) (runtime/CSharp/src/Atn/ATNDeserializer.cs)
• ATNDeserializer.CombineChainedEpsilons (cyclomatic 21) (runtime/CSharp/src/Atn/ATNDeserializer.cs)
• ATNDeserializer.GenerateRuleBypassTransitions (cognitive 44) (runtime/CSharp/src/Atn/ATNDeserializer.cs)
• ATNDeserializer.GenerateRuleBypassTransitions (cyclomatic 17) (runtime/CSharp/src/Atn/ATNDeserializer.cs)
• ATNDeserializer.InlineSetRules (cognitive 52) (runtime/CSharp/src/Atn/ATNDeserializer.cs)
• ATNDeserializer.InlineSetRules (cyclomatic 31) (runtime/CSharp/src/Atn/ATNDeserializer.cs)
• ATNDeserializer.OptimizeSets (cognitive 51) (runtime/CSharp/src/Atn/ATNDeserializer.cs)
• ATNDeserializer.OptimizeSets (cyclomatic 22) (runtime/CSharp/src/Atn/ATNDeserializer.cs)
• ATNDeserializer.ReadEdges (cognitive 39) (runtime/CSharp/src/Atn/ATNDeserializer.cs)
• ATNDeserializer.ReadEdges (cyclomatic 17) (runtime/CSharp/src/Atn/ATNDeserializer.cs)
• ATNDeserializer.TestTailCall (cognitive 18) (runtime/CSharp/src/Atn/ATNDeserializer.cs)
• ATNDeserializer.VerifyATN (cognitive 32) (runtime/CSharp/src/Atn/ATNDeserializer.cs)
• ATNDeserializer.VerifyATN (cyclomatic 16) (runtime/CSharp/src/Atn/ATNDeserializer.cs)
• ATNDeserializer.deriveEdgesForRuleStopStates (cognitive 20) (runtime/Swift/Sources/Antlr4/atn/ATNDeserializer.swift)
• ATNDeserializer.deserialize (cognitive 127) (runtime/Java/src/org/antlr/v4/runtime/atn/ATNDeserializer.java)
• ATNDeserializer.deserialize (cognitive 23) (runtime/Swift/Sources/Antlr4/atn/ATNDeserializer.swift)
• ATNDeserializer.deserialize (cyclomatic 19) (runtime/Swift/Sources/Antlr4/atn/ATNDeserializer.swift)
• ATNDeserializer.deserialize (cyclomatic 54) (runtime/Java/src/org/antlr/v4/runtime/atn/ATNDeserializer.java)
• ATNDeserializer.generateRuleBypassTransition (cognitive 19) (runtime/Go/antlr/v4/atn_deserializer.go)
• …and 890 more

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

antlr/antlr4 was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.