guardian/typerighter

Typerighter is an editorial rule management and text checking service designed to validate content against configurable style, grammar, and spelling rules. It consists of a Rule Manager for creating, testing, and publishing rules, and a Checker service that applies these rules to text using dictionary, regex, and LanguageTool matchers. The system supports draft-to-live rule workflows, integrates with external content APIs, and provides streaming check capabilities with priority-based matching.

29 changes.

This period established the foundational architecture for the Typerighter project, introducing the Rule Manager and Checker services alongside shared libraries and AWS CDK infrastructure. It involved migrating from legacy Play Framework patterns to a modernized setup with AWS SDK v2, custom matcher implementations, and comprehensive testing suites.

25 changes.

This period focused on rebuilding the rule management system with a draft-and-live lifecycle, introducing a new client interface, and supporting dictionary-based rules. Concurrently, the infrastructure was consolidated into a single CDK stack, and the checker service was enhanced with UK English spell-checking capabilities.

The checker now supports spell-checking against the Collins Dictionary for English. This change introduces new service components in the \collins\ directory, including a \CollinsEnglish\ language definition, a \MorfologikCollinsSpellerRule\ that categorizes Collins-specific errors, and Scala-based dictionary builders (\DictionaryBuilder\, \SpellDictionaryBuilder\) to generate the required Morfologik binary dictionaries. This enables users to detect spelling errors specifically flagged by the Collins Dictionary resource.

apps/checker/app/services/collins · high confidence

A new pre-commit hook has been added to the repository to enforce code formatting using scalafmt. When a commit is made, the hook checks for a \.scalafmt.conf\ file and runs scalafmt in test mode on changed files. If formatting fails, the commit is blocked, and users are instructed to run \scalafmt\ to fix the issues or use the \--no-verify\ flag to bypass the check. The hook attempts to use the \scalafmt\ CLI directly for performance, falling back to \sbt scalafmtAll\ if the CLI is not available.

script/hooks · high confidence

The spellchecker now includes a new UK English (en\_GB) dictionary resource, consisting of a configuration file (collins.info) and a word list (en\_gb\_wordlist.xml). This addition enables the spellchecker to validate words against a UK-specific vocabulary and apply locale-aware rules, such as ignoring diacritics and handling specific replacement pairs (e.g., 'ph' to 'f').

apps/checker/conf/resources · high confidence

The CDK directory now contains the foundational TypeScript project structure, including configuration files for the AWS CDK Toolkit (cdk.json), TypeScript compilation (tsconfig.json), and Jest testing (jest.config.js, jest.setup.js). This setup enables the synthesis of CloudFormation templates and the management of infrastructure-as-code for the Rule-Manager component, with specific exclusions for generated JavaScript files and test artifacts in the build process.

cdk · high confidence

This change introduces the foundational configuration files for the Rule Manager application. It defines the Play Framework application loader, allowed hosts, and database connection settings using HikariCP, including automatic evolution application. It also configures Logback for structured JSON logging to stdout (for ELK) and file storage, and establishes the complete set of API routes for managing rules (CRUD, batch operations, CSV import, archiving, publishing) and tags, alongside static asset serving and client-side routing fallback.

apps/rule-manager/conf · high confidence

This change introduces the foundational configuration and documentation for the Typerighter service, establishing the project structure for its two core components: the Rule Manager and the Checker. It adds essential developer tooling and environment setup, including \.nvmrc\ (Node 24.13.1), \.java-version\ (Java 17), and \.scalafmt.conf\ (v3.7.17) to standardize the development environment. A \docker-compose.yml\ is provided to spin up local dependencies, specifically a Postgres database and a Localstack instance for S3 emulation. The repository also includes a \riff-raff.yaml\ for deployment definitions, a \.prout.json\ for monitoring health checks, and comprehensive documentation (\README.md\, \vision.md\) explaining the architecture, usage, and setup procedures.

(repo-wide) · high confidence

The Rule Manager service is now available, providing endpoints to manage editorial rules and tags. The application exposes a home endpoint, a rules controller for creating and managing rules via Sheets and S3, and a tags controller for CRUD operations on tags. It integrates with the Guardian Content API for content checks, uses AWS S3 (with LocalStack support for local development) for rule storage, and persists data to a database with automatic schema evolution.

apps/rule-manager/app · high confidence

Adds a new CDK stack definition for the Typerighter application, consolidating the infrastructure for the checker and rule-manager services. The stack provisions two public-facing EC2 instances (t4g.small) running the checker and rule-manager apps, configures CloudFront distributions with specific caching and CORS policies, sets up a Postgres RDS database instance, and establishes DNS records for the checker and manager subdomains. It also handles IAM policies for S3 access (permissions cache, pan-domain auth) and configures user data scripts to install the application packages and write configuration files.

cdk/lib · high confidence

The rule manager now supports a new 'dictionary' rule type, ingesting word lists from S3 (Collins Dictionary and lemmatised lists) and allowing specific words to be excluded from publication. A new service enables testing rules against live CAPI content via paginated streams, and the system now handles CSV imports and bulk archiving by tag, with all rule data (draft, live, and history) returned to the client for accurate state management.

apps/rule-manager/app/service · high confidence

The Checker application now exposes dedicated controllers for its core functionality. The API controller adds endpoints to check single rules (checkSingleRule) with HMAC authentication, in addition to existing bulk checking and streaming capabilities. A new CAPI proxy controller provides search endpoints for content, tags, and sections. Management of matcher rules is now handled by a dedicated Rules controller, and the health check has been updated to verify rule availability and include the commit ID in error responses.

apps/checker/app/controllers · high confidence

The checker application now includes a set of new utility components in the \utils\ package to support its operational logic. \CheckerConfig\ centralizes application settings, including paths for n-gram data, Google credentials, and NER API details, while extending common configuration logic. A new \CloudWatchClient\ enables the publication of custom metrics (such as rules ingested or matcher job duration) to AWS CloudWatch for monitoring. The \Timer\ utility provides synchronous and asynchronous timing functions that log performance data and warn when operations exceed a configurable slow-log threshold. Additionally, \Matcher\ defines the interface for rule-checking services, and \RuleMatchHelpers\ offers logic to filter out overlapping rule matches, ensuring cleaner results for users.

apps/checker/app/utils · high confidence

The rule manager now exposes a set of authenticated API endpoints for managing matcher rules and tags. Users can create, update, list, and publish rules, as well as perform batch updates and refresh rules from source sheets or dictionaries. Tag management endpoints allow listing tags with associated rule counts, creating, updating, and deleting tags. All write operations (create, update, publish, refresh, tag management) require the 'manage\_rules' permission, enforced via the PandaAuthController and PermissionsHandler. The API returns structured JSON responses, including validation errors and rule data, and handles 404s for non-existent rules or tags.

apps/rule-manager/app/controllers · high confidence

A new abstract controller, PandaAuthController, has been added to the common library to centralize authentication behavior. It integrates PanDomain and HMAC authentication actions, configures OAuth callback URLs based on service and stage settings, and implements user validation logic. This change provides a reusable base for controllers requiring these specific authentication mechanisms.

apps/common-lib/src/main/scala/com/gu/typerighter/controllers · high confidence

The matcher pool now includes three new matcher types: DictionaryMatcher, which uses the Collins dictionary via LanguageTool and excludes matches falling within named entities to reduce false positives; RegexMatcher, which applies regular expression rules with sentence-start awareness; and LanguageToolMatcher, which wraps a JLanguageTool instance to support custom XML-based pattern rules and specific core rules. These matchers are wired into the application to provide distinct matching capabilities for dictionary checks, pattern-based regex checks, and configurable LanguageTool grammar checks.

apps/checker/app/matchers · high confidence

The checker service now integrates an external NER (Named Entity Recognition) API via a new EntityHelper service, allowing it to identify entities like organizations and locations. This capability is used by the DictionaryMatcher to exclude dictionary matches that overlap with recognized named entities, preventing false positives on proper nouns. Additionally, the MatcherPool service now supports streaming responses via a new checkStream method, which returns progress percentages as jobs complete, and a checkSingle method for checking individual rules. The service also introduces a priority system for matchers, ensuring regex rules are evaluated before dictionary rules, which are evaluated before language tool rules.

apps/checker/app/services · high confidence

The rule manager now includes a suite of new utility classes to support its core functionality. Dictionary.scala adds the ability to parse Collins dictionary XML files into structured word lists and definitions. Permissions.scala introduces a handler that checks user permissions against a remote store and serializes user/permission data to JSON for the frontend. RuleManagerConfig.scala centralizes application configuration, including database credentials and service URLs. Additional utilities include FormHelpers for standardizing form error responses, Errors.scala for specific exception types, LocalStack.scala for local AWS S3 integration, and StringHelpers for camel-to-snake case conversion.

apps/rule-manager/app/utils · high confidence

The Rule Manager client has been rebuilt with a new UI framework, introducing a comprehensive rule editing experience. Users can now create, edit, and manage rules via a form that supports Markdown previews for descriptions, automatic debounced saving, and validation. The interface includes a diff view to compare live and draft rule changes, a publication history timeline, and batch editing capabilities for multiple rules. Additionally, a dedicated tags management page allows users to create, edit, and delete tags, while feature switches enable controlled rollout of specific functionalities like destructive reloads.

apps/rule-manager/client/src · high confidence

The legacy ApiController, which previously handled the root index and a form-encoded text-checking endpoint via the LanguageTool service, has been removed from the application. This deletion indicates that the original simple validation interface is no longer part of the codebase, likely replaced by the newer controller architecture and API endpoints introduced in recent commits.

app/controllers · high confidence

The \app/services/LanguageTool.scala\ file, which provided a wrapper around the JLanguageTool library for checking text against English (GB) rules, has been removed. This change eliminates the previous implementation that instantiated a JLanguageTool with a fixed cache size and user configuration, likely as part of a broader refactoring to support new features such as ngram models or rule persistence.

app/services · high confidence

The project now includes a new \script/lib/check\ entry point that explicitly sources and runs validation scripts for AWS credentials, Java version, and Node version before proceeding. Users will now receive immediate, clear error messages if their environment is misconfigured—such as expired or missing AWS credentials for the 'composer' profile, or a mismatch between the installed and required versions of Java or Node—preventing downstream failures during development or deployment.

script/lib · high confidence

The checker application has been refactored to use the AWS SDK v2 and a new centralized configuration system. The app now initializes via a dedicated AppLoader and AppComponents, supporting both production AWS environments and local development via LocalStack S3 with path-style access enabled. Configuration values such as region, identity, and credentials are now passed explicitly, and the app integrates with the Typerighter bucket for rule management, NER service for entity recognition, and CloudWatch for monitoring. The UI assets have been updated with new styles for the sidebar, controls, and loading indicators.

apps/checker/app · high confidence

The checker service now includes its own configuration files, introducing a new application.conf that sets a Content Security Policy, allows larger HTTP payloads (4MB buffer), and configures the matcher-pool dispatcher. Routing is defined for API endpoints including /check, /checkStream, and /checkSingle, alongside OAuth and health check paths. Logging has been updated to use a Logstash encoder for stdout and changed the root log level to INFO, with logs directed to a configurable home directory.

apps/checker/conf · high confidence

The CDK application entry point in cdk/bin/index.ts has been rewritten to instantiate the Typerighter stack directly for both CODE and PROD environments. This change replaces the previous multi-stack structure (which included separate rule-manager and rule-manager-db stacks) with a single consolidated stack configuration, setting instance counts to 1 for CODE and 3 for PROD, and assigning specific domain suffixes for each stage.

cdk/bin · high confidence

The rule-manager database schema has been evolved through 17 migrations to support a draft-and-live rule model. Key changes include splitting rules into 'rules\_draft' and 'rules\_live' tables, introducing 'external\_id' as a required unique identifier, and adding support for tags via new 'tags' and join tables. The schema now tracks revision history, enforces active rule ordering, supports archiving, and includes optimized GIN indexes for full-text search and pattern matching.

apps/rule-manager/conf/evolutions · high confidence

Added new case classes in the checker app's model package to define the structure of validation requests and responses. The \Check\ model now supports filtering by category IDs and explicitly excludes specified categories via \excludeCategoryIds\. The \CheckResult\ model includes a new \percentageRequestComplete\ field to report progress. Additionally, new models \MatcherError\, \MatcherResponse\, and \MatcherWorkComplete\ define the specific response types for validation errors, successful matches, and work completion signals.

apps/checker/app/model · high confidence

The database access layer has been restructured to support a distinct draft and live rule lifecycle. New models (DbRuleDraft, DbRuleLive) and tables (rules\_draft, rules\_live) separate editable rules from published ones, introducing fields like revisionId, ruleOrder, isPublished, and isArchived to track state and ordering. Tag management is now explicit via dedicated join tables (rule\_tag\_draft, rule\_tag\_live) and a Tags model, allowing rules to be associated with tags in both draft and live states. The DB connection helper (DB.scala) and common rule schema (DbRule.scala) provide the foundation for these operations, including batch inserts and specific queries for finding, publishing, and archiving rules.

apps/rule-manager/app/db · high confidence

The Typerighter model layer now supports checking against a single rule and enforces a strict priority hierarchy among rule types: Regex rules take the highest priority, dictionary rules the second highest, and all other types (such as LanguageTool matches) the lowest. This ensures that Style Guide rules outrank dictionary matches, which in turn outrank language tool matches. The \RuleMatch\ model now includes a \priority\ field and a \groupKey\ for serialization, while \CheckerRule\ implementations explicitly define their priority levels to control matching precedence.

apps/common-lib/src/main/scala/com/gu/typerighter/model · high confidence

This change introduces a new common library (apps/common-lib) that consolidates shared infrastructure for the Typerighter services. It provides a unified AppSetup and CommonConfig for managing AWS credentials, SSM configuration, and PanDomain authentication settings. The library also includes a ContentClient for interacting with the Guardian Content API, an HMACClient for generating authentication headers, and utilities for JSON serialization (including newline-delimited JSON) and safe XML parsing, standardizing how these services handle configuration and external API calls.

apps/common-lib/src/main/scala/com/gu/typerighter/lib · high confidence

The localstack environment now includes an init-aws.sh script that automatically creates the typerighter-app-local S3 bucket and populates it with essential dictionary files (collins-dictionary.xml, collins-lemmatised-list.xml, and words-to-not-publish.json) copied from the /etc/gu/typerighter/ directory, ensuring these resources are available for local development.

localstack · high confidence

The system now stores checker rules in S3 as newline-delimited JSON (one rule per line) instead of a single JSON array, and retrieves them by parsing each line individually. To ensure continuity during the transition, the new storage implementation automatically falls back to reading the legacy single-file JSON format if the new arteact is missing or unreadable. This change is implemented in the new BucketRuleResource class, which handles both writing the new format and reading from either the new or legacy keys.

apps/common-lib/src/main/scala/com/gu/typerighter/rules · high confidence

The checker application now features a new user interface built on a shared layout template. This update introduces a breadcrumb navigation component for better context awareness and integrates pixel-based telemetry tracking to monitor page views. The main layout includes a navigation bar with links to Home and Rules, while the rules view displays matcher statistics and directs users to the rule manager for refresh operations.

apps/checker/app/views · high confidence

Added \compare-rule-xml.js\ to detect changes in specific LanguageTool rule definitions between two XML files by reading a list of rule IDs from a text file, and \dump-db.ts\ to export the \rules\_draft\ table from a PostgreSQL database to CSV. The comparison script now uses \libxml2-wasm\ for XML parsing instead of the previously used \libxml2js\, addressing a critical vulnerability in the older library.

script/js · high confidence

Developers can now use the new \script/start\ command to launch both the rule checker and rule manager services simultaneously, replacing previous ad-hoc startup methods. This entry point delegates to new \script/start-checker\ and \script/start-manager\ scripts, which handle service-specific setup such as running pre-flight checks, configuring debug modes, and managing Docker dependencies via \docker compose\. The \start-manager\ script also ensures that the client application is built and started in watch mode, and automatically tears down Docker containers and kills background processes on exit.

script · high confidence

The legacy \conf/application.conf\ and \conf/routes\ files have been deleted. This removes the default Play Framework application loader configuration and the hardcoded route definitions for the home (\/\) and check (\/check\) endpoints, indicating a migration away from the standard Play routing and configuration structure.

conf · high confidence

The legacy \Rule\ and \RuleMatch\ case classes, which previously wrapped LanguageTool's rule and match objects for JSON serialization, have been removed from the \app/model\ directory. This change eliminates the old data structures that mapped directly to \org.languagetool.rules\ entities, reflecting a shift in the internal model architecture to support more complex matching logic and multiple matcher types.

app/model · high confidence

The application's previous bootstrap structure, including the \AppComponents\ and \AppLoader\ classes that wired the \ApiController\ and \LanguageTool\ instance, has been removed. Additionally, the \JsonImplicits\ utility object, which provided JSON serialization for \URL\ objects, has been deleted. These changes indicate a shift away from the previous Play Framework component wiring and JSON handling approach.

app · high confidence

The rule-manager now uses dedicated Play Framework form models to handle input validation and data binding for rule and tag operations. New models include CreateRuleForm and UpdateRuleForm, which validate rule types (regex, languageToolCore, languageToolXML, dictionary) and fields like category, tags, and externalId. Specific forms for individual rule types (CheckerRuleForm) enforce pattern-specific constraints, such as regex syntax and XML validity. Additionally, BatchUpdateRuleForm enables bulk updates to rule categories and tags, while CreateTagForm and PublishRuleForm handle tag creation and rule publishing respectively. PaginatedResponse supports structured API responses.

apps/rule-manager/app/model · high confidence

The rule-manager view now loads frontend assets via the Vite development server (localhost:5173) when the application stage is set to 'dev', enabling hot module replacement during development. For non-development stages, it falls back to serving static build files from the /build directory. Additionally, the view now injects a stage-specific favicon and passes user permission data and telemetry configuration to the frontend via script tags and a JSON data block.

apps/rule-manager/app/views · high confidence

Added database integration tests for rule and tag management; Added test coverage for checker services and performance simulation; Added test coverage for dictionary, language tool, and regex matchers; Added test fixtures for CAPI responses and rule generation; Added test fixtures for rule-manager resources; Added test resources for Gatling simulations and spell-checker dictionary; Added tests for CheckerRuleForm validation; Added tests for dictionary XML parsing utilities; Added tests for rule testing service; Added tests for the Timer utility; Added unit tests for text range and block manipulation logic; Removal of HomeControllerSpec test suite; RuleMatch fixtures now include groupKey; Updated CDK infrastructure snapshot for Typerighter stack.

This change introduces the initial dependency manifests and lockfiles for three new areas of the project: the rule-manager client application, the CDK infrastructure code, and the JS utility scripts. The rule-manager client (apps/rule-manager/client) is set up with React 17, Vite 4.5.9, TypeScript 4.5.3, and the Elastic EUI library. The CDK module (cdk) pins the AWS CDK library to version 2.1100.3 and the internal @guardian/cdk to 62.3.2. The JS scripts module (script/js) adds dependencies for PostgreSQL interaction (pg 8.13.1) and XML processing (libxml2-wasm).

(dependencies) · high confidence

The project has upgraded the Play framework from version 2.6.20 to 2.9.10, which includes fixes for security vulnerabilities in underlying dependencies like Jackson. The build environment has also been updated, bumping sbt to version 1.12.6 and adding several new plugins including Gatling for testing, SBT Riff-Raff for artifact management, and ScalikeJDBC for database model generation.

project · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• First survey — no prior run to compare against. CAI 48.

• Code Health 72
• Architecture 43
• Maturity 75
• Readiness 37
• Security 76

• 300 commits — 267 feature/other, 33 fixes

• (repo) — 87 commits
• (root) — 75 commits
• apps/rule-manager — 66 commits
• apps/checker — 21 commits
• cdk/lib — 18 commits
• .github/workflows — 10 commits
• cdk/package.json — 6 commits
• cdk/yarn.lock — 4 commits
• project/plugins.sbt — 3 commits
• apps/common-lib — 2 commits
• cdk/tsconfig.json — 2 commits
• project/build.properties — 2 commits
• script/js — 2 commits
• .github/CODEOWNERS — 1 commit
• script/citest — 1 commit

• fix: Add commit ID to Checker healthcheck to fix prout
• fix: Add thrift as direct dependency to fix vulnerability
• fix: Attempt to fix test
• fix: Bump GuCDK to use the fixed version of fast-xml-parser
• fix: Bump simple-configuration to fix a netty vulnerability
• fix: Fix CSP and stray braces
• fix: Fix GHA CI
• fix: Fix errors due to changed interface of LanguageTool
• fix: Fix lz4 and okhttp3 vulnerability
• fix: Fix test
• fix: Fix test
• fix: Fix tests
• fix: Generate jest snapshot to fix failing cdk test
• fix: Merge branch 'main' into fix-prout
• fix: Merge branch 'main' into sg/fix-cdk-jest-testing
• fix: Merge pull request #456 from guardian/sg/fix-cdk-jest-testing
• fix: Merge pull request #464 from guardian/upgrade-to-play-29-and-fix-scala-steward
• fix: Merge pull request #471 from guardian/jsh/fix-pagination-results
• fix: Merge pull request #503 from guardian/fix-dependabot-vulnerability-221
• fix: Merge pull request #510 from guardian/fix-prout
• …and 280 more

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

guardian/typerighter was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.