Skip to content
CAI
Software that uses CAICheck a score

Shopify/roast

62.4

Adequate · 6 November 2026

10.3k

lines of production code

Ruby

primary language

4

measurements over time

CAI band scale
CAI trend line
CAI lens gauges

What this system is

Features

Added Sorbet RBI type definitions for core gems and libraries

The project now includes generated Sorbet RBI files for several key dependencies, including Active Support 8.0.2, Addressable 2.8.7, Async 2.34.0, Base64, Benchmark, AST, Faraday, Minitest, Mocha, Rainbow, and Webmock. These annotations provide static type information for the gems, enabling Sorbet to perform type checking on code that interacts with these libraries. The files are organized into directories for annotations, DSL methods, and gem-specific types, and are marked as generated or vendored in Git attributes to prevent accidental manual edits.

sorbet/rbi · high confidence

Added demo examples for workflow cogs and multi-loading

The examples/demo directory now includes new Ruby files demonstrating how to define and use cogs within workflows. Specifically, local.rb provides a sample local cog implementation, while multiloading.rb and simple\_external\_cog.rb illustrate how to load and execute cogs from both local sources and external plugin gems (plugin\_gem\_example) within a single workflow execution.

examples/demo · high confidence

Added plugin gem and prompt examples

New example files have been added to the repository, including a Ruby gem example (\plugin-gem-example\) demonstrating \Roast::Cog\ implementations and a simple ERB-based prompt template (\simple\_prompt.md.erb\). These additions provide reference implementations for users looking to create plugins or define prompt structures.

examples/plugin-gem-example, examples/prompts · high confidence

Expanded workflow examples covering agents, async execution, and control flow

The examples directory now includes comprehensive demonstrations of new and existing workflow capabilities. Users can see how to configure and use AI agents (including the new Pi provider and multi-prompt sessions), run commands asynchronously in the background, and manage complex control flow with \map\, \reduce\, \repeat\, \skip\, and \break\. Additional examples illustrate accessing outputs from sub-executors via \from\ and \collect\, customizing logging, handling shell sanitization, and working with workflow parameters and temporary directories.

examples · high confidence

Initial Sorbet and Tapioca configuration setup

This change introduces the initial configuration for Sorbet type checking and Tapioca RBI generation. A new \sorbet/tapioca/config.yml\ file is added to exclude \yard\ and \yard-sorbet\ gems from RBI generation, as they are unused by the project. Additionally, a \sorbet/tapioca/require.rb\ file is created to explicitly require dependencies such as \active\_support\, \async\, \rainbow\, and \sqlite3\, ensuring the type checker has the correct context for these libraries.

sorbet/tapioca · high confidence

Initial Sorbet type-checking configuration

The project now includes a Sorbet configuration file that enables the experimental 'requires\_ancestor' and 'rbs\_comments' features. The configuration sets the root directory to the project root and excludes standard directories such as .git, .idea, tmp, vendor, test, bin, and the examples/demo folder from type checking.

sorbet · high confidence

Introduce Agent and Chat cogs with multi-provider support and truncation detection

The \lib/roast/cogs\ directory now includes new \Agent\ and \Chat\ cog implementations. The \Agent\ cog enables coding tasks with local filesystem and tool access, supporting Claude and Pi providers. The \Chat\ cog provides pure LLM interaction without local system access and now supports OpenAI, Anthropic, Perplexity, Gemini, and Amazon Bedrock. A key behavioral change in the \Chat\ cog is the detection of max-token truncation, which raises an error if the response is cut off, and improved stats reporting that falls back to the chat model ID when the response lacks one.

lib/roast/cogs · high confidence

Introduce Pi coding agent provider

Added a new Pi coding agent provider to the agent cogs system. This change introduces the \PiInvocation\ class which manages the execution of the Pi agent, including handling prompts, sessions, tool calls, and timeouts, and exposes configuration options for model selection, working directories, and raw message dumping.

lib/roast/cogs/agent/providers/pi · high confidence

Introduce configurable Agent cog with provider selection, multi-prompt chaining, and usage tracking

The Agent cog now supports configurable agent providers (defaulting to Pi, with Claude also available) and allows overriding the default via the ROAST\_DEFAULT\_AGENT\_PROVIDER environment variable. Users can chain multiple prompts in a single session for complex tasks, set execution timeouts, and view detailed usage statistics including token counts, costs, and per-model breakdowns.

lib/roast/cogs/agent · high confidence

Introduce configurable chat cog with multi-provider support and session management

The chat cog now supports multiple LLM providers (OpenAI, Anthropic, Perplexity, Gemini, and Amazon Bedrock) and allows users to select a provider via configuration or the ROAST\_DEFAULT\_CHAT\_PROVIDER environment variable. It introduces session management to maintain conversational context across multiple invocations, enabling multi-turn interactions. The cog validates API keys and base URLs per provider, raising InvalidConfigError for misconfigurations, and provides input/output structures for prompts, responses, and session state.

lib/roast/cogs/chat · high confidence

Introduce core Roast::Cog infrastructure for configuration, inputs, outputs, and registry

This change establishes the foundational classes for the Roast workflow engine within the \lib/roast/cog\ directory. It adds \Roast::Cog::Config\ to manage cog settings (including async execution and failure handling), \Roast::Cog::Input\ and \Roast::Cog::Output\ to define how data is passed into and retrieved from cogs (with robust JSON and number parsing in outputs), and \Roast::Cog::Registry\ to manage the available cogs in a workflow. These components provide the structural basis for defining and executing cogs.

lib/roast/cog · high confidence

New Sorbet RBI shims for Roast workflow execution and configuration

Added Sorbet RBI shim files for \Roast::CogInputContext\, \Roast::ConfigContext\, and \Roast::ExecutionContext\. These shims define the type signatures and documentation for workflow input methods (such as \target!\, \targets\, \arg?\, \kwarg\), global and cog-specific configuration options (including \async!\, \parallel\, \timeout\, and \provider\ settings for the \agent\ cog), and execution scope controls (including \outputs!\, \outputs\, \call\, and \map\). This improves static type checking and IDE autocomplete for Roast workflow definitions.

sorbet/rbi/shims/lib/roast · high confidence

New internal documentation standards and maintenance tooling

This change introduces a new directory for internal documentation comments (\internal/documentation/comments/\) containing architectural notes and guidelines for both user-facing and developer-facing documentation, establishing standards for cog capabilities (e.g., distinguishing \agent\ vs \chat\ cogs) and IDE shim documentation. It also adds a custom RuboCop cop (\Roast/NoTestClassNesting\) to prevent class nesting in test files that breaks IDE test runners, and introduces two internal maintenance workflows: one to analyze branch diffs for documentation impact and another to update references to deprecated LLM models.

internal · high confidence

New system cogs for calling scopes, mapping collections, and repeating loops

The \lib/roast/system\_cogs\ directory now includes \Call\, \Map\, and \Repeat\ system cogs, enabling workflows to invoke named execution scopes, iterate over collections (with optional parallelism), and loop until a break condition is met. These cogs provide structured input/output handling and integration with the execution manager, allowing users to build more complex, reusable workflow logic within the \Roast\ namespace.

_lib/roast/system\_cog, lib/roast/system\cogs · high confidence

New tutorial series for Roast workflows

Added a comprehensive, multi-chapter tutorial series (chapters 1–6) that guides users through creating and running Roast workflows. The tutorials cover core concepts including the \chat\ and \agent\ cogs, chaining steps, passing targets and parameters, configuring models and providers, control flow with \skip!\ and \fail!\, and creating reusable named scopes with the \call\ cog. The examples demonstrate supported providers like OpenAI, Anthropic, Perplexity, Gemini, and Bedrock, and show how to configure display options and temperature settings.

tutorial · high confidence

Workflow parameter access in config blocks

The \Roast::Cog::Config\ class now exposes methods to inspect workflow invocation details within configuration blocks. Users can access the single target via \target!\, all targets via \targets\, check for flag arguments with \arg?\ and \args\, and retrieve keyword arguments using \kwarg\, \kwarg!\, and \kwargs\. This enables dynamic configuration based on how the workflow was invoked, such as adjusting model settings or display modes based on provided flags or targets.

sorbet/rbi/shims/lib/roast/cog · high confidence

Behavioural changes

Claude provider now formats tool use and result messages for terminal output

The Claude agent provider now displays structured, human-readable summaries of tool interactions (such as Bash, Read, Write, Edit, Glob, Grep, and Skills) in the terminal. Instead of raw JSON or unformatted text, users see concise one-line indicators for tool invocations (e.g., 'BASH ls -la') and results (e.g., 'BASH OK 12 lines · Cloning into roast...'), improving visibility into the agent's actions during execution.

lib/roast/cogs/agent/providers/claude · high confidence

Development environment and tooling overhaul

The project has significantly updated its development setup: the default Ruby version is bumped to 3.4.7, the test suite has migrated from Minitest to RSpec (with corresponding Rake tasks like \minitest\_fast\ replaced by \minitest\_all\ and RSpec-specific configurations), and static analysis now includes Sorbet type checking alongside RuboCop. Configuration files have been restructured with new \.rubocop.yml\ rules, a \dev.yml\ for development commands, and updated \.gitignore\ entries, while legacy documentation files like \CLAUDE.md\ have been removed.

(repo-wide) · high confidence

Introduce new CLI, workflow execution engine, and structured logging system

The \lib/roast\ library has been replaced with a new implementation that introduces a Thor-based CLI (\lib/roast/cli.rb\) for running workflows, a new \Cog\ and \ExecutionManager\ architecture for defining and running workflow steps, and a structured event-driven logging system (\lib/roast/event.rb\, \lib/roast/event\_monitor.rb\, \lib/roast/log.rb\) that captures stdout/stderr and provides colored, path-aware log output. This change also adds a new \CommandRunner\ for safe shell command execution with timeout and streaming support, and a \ConfigManager\ for scoped workflow configuration.

lib/roast · high confidence

Migrates CLI framework and adopts Zeitwerk autoloading

The library has replaced the CLI::Kit framework with a custom implementation using Ruby's standard library (optparse) and Zeitwerk for autoloading, removing the previous CLI::Kit dependencies and command registry structure. This change also consolidates third-party gem requirements (such as ActiveSupport, Rainbow, and RubyLLM) into the main entry point and removes legacy command files like 'example' and 'help' that were tied to the old CLI structure.

lib · high confidence

Structured formatting for Pi agent tool calls and results

The Pi agent provider now includes dedicated formatters for tool calls (bash, read, write, edit, grep, find) and their corresponding results. This change introduces concise, human-readable terminal output for agent actions, such as displaying truncated command previews for bash, file paths and line ranges for read/write, and match counts for grep and find, ensuring that complex tool interactions are presented clearly in the UI.

lib/roast/cogs/agent/providers/pi/messages · high confidence

Structured message types for Claude provider

The Claude provider now uses dedicated message classes (AssistantMessage, ResultMessage, SystemMessage, TextMessage, ThinkingMessage, ToolResultMessage, ToolUseMessage, UnknownMessage, and UserMessage) to parse and format incoming data. This change introduces specific handling for different message types, including support for Claude's 'thinking' messages, improved error message extraction in ResultMessage, and explicit field ignoring to clean up internal metadata before processing.

lib/roast/cogs/agent/providers/claude/messages · high confidence

Support for multiple prompts and session reuse in Claude and Pi agents

The Claude and Pi agent providers now handle multiple prompts within a single invocation. Instead of processing only the first prompt, the agents iterate through all provided prompts, chaining them together by reusing the session from the previous step (avoiding unnecessary forking for subsequent prompts). Additionally, usage statistics (stats) from all individual prompt invocations are summed and returned in the final result, giving users a complete view of resource consumption across the entire multi-prompt sequence.

lib/roast/cogs/agent/providers · high confidence

Switches entry point to use Roast::CLI and loads roast-ai

The executable now requires the 'roast-ai' library instead of 'roast' and invokes the CLI via Roast::CLI.start(ARGV), replacing the previous ErrorHandler/EntryPoint pattern. This change aligns the entry point with the Thor-based CLI implementation and simplifies execution flow.

exe · high confidence

Update bin scripts and remove unnecessary binstubs

The bin directory has been updated to include new Bundler-generated binstubs for development tools (rubocop, spoom, srb, srb-rbi, tapioca) and the main application (roast), while removing the custom bin/setup and bin/testunit scripts. The bin/console script has also been refactored to explicitly add the lib directory to the load path before requiring the application, ensuring correct module loading in interactive sessions.

bin · high confidence

Test coverage

Added VCR-based functional test infrastructure for DSL examples; Added comprehensive test suite for Roast CLI and core components; Added functional tests for Roast example workflows; Added test coverage for Agent cog components; Added test coverage for Claude agent provider components; Added test coverage for system cogs; Added test coverage for the Chat cog; Added test support infrastructure for Minitest assertions and test cogs; Added tests for Claude agent message types; Added tests for Cmd and Ruby cog components; Added tests for Pi agent tool call and result message formatting; Added tests for agent provider session and timeout handling; Added tests for the NoTestClassNesting RuboCop cop; Added tests for the Pi agent invocation provider; Added unit tests for core Cog components; Migrate test suite to Minitest and modernize test infrastructure.

Dependencies

Roast gem renamed to roast-ai with updated dependencies and example projects

The gem has been renamed from 'roast' to 'roast-ai' (including the gemspec and internal references), and its runtime dependencies have been updated to use activesupport (\~\> 8.0), async (\>= 2.34), rainbow (\>= 3.0.0), ruby\_llm (\>= 1.13), type\_toolkit (\>= 0.0.5), and zeitwerk (\>= 2.6). The project now includes example projects for a demo and a plugin-gem-example, each with their own Gemfile and Gemfile.lock, demonstrating how to depend on the renamed roast-ai gem. Development dependencies have also been updated to include guard-minitest, minitest-rg, rubocop-sorbet, sqlite3, and tapioca, among others.

(dependencies) · high confidence

Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

How this codebase got here

Score

  • CAI 53 → 62 (+9.1)
  • Rubric changed (rubric-2026.10.5 → rubric-2026.11.2) — scores are not directly comparable.

Lenses

  • Code Health 85 → 85 (+0.0)
  • Architecture 85 → 84 (-1.7)
  • Maturity 34 → 55 (+21.7)
  • Readiness 56 → 56 (+0.0)
  • Security 87 → 78 (-8.1)

Resolved (2)

  • Medium vulnerability: [GHSA redacted] (Gemfile.lock)
  • Off-boarding risk: anonymized user #1

New (1)

  • Off-boarding risk: anonymized user #1

Changes since last survey

  • 1 commits — 1 feature/other, 0 fixes

By area

  • (root) — 1 commit

Notable commits

  • change: Bump sqlite3 from 2.9.5 to 2.9.6

Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

Survey your own repository

Shopify/roast was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.

About this page

  • The score is its most recent published measurement, taken on 6 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
  • Measured at commit 93b2f5c99982bd3e8df426fa09da02a64a454263 — the exact code this score is about.
  • Scored under rubric-2026.11.2 — the same rubric and the same method as every other entry in this index.
  • Measured by Canine Code Analyzer (cca.canine.dev) using cca measure-2026.11.2+preprod-5c4e66d9318b.