Shopify/roast
62.4
Adequate · 6 November 2026
10.3k
lines of production code
Ruby
primary language
4
measurements over time
What this system is
Features
Added Sorbet RBI type definitions for core gems and libraries
The project now includes generated Sorbet RBI files for several key dependencies, including Active Support 8.0.2, Addressable 2.8.7, Async 2.34.0, Base64, Benchmark, AST, Faraday, Minitest, Mocha, Rainbow, and Webmock. These annotations provide static type information for the gems, enabling Sorbet to perform type checking on code that interacts with these libraries. The files are organized into directories for annotations, DSL methods, and gem-specific types, and are marked as generated or vendored in Git attributes to prevent accidental manual edits.
sorbet/rbi · high confidence
Added demo examples for workflow cogs and multi-loading
The examples/demo directory now includes new Ruby files demonstrating how to define and use cogs within workflows. Specifically, local.rb provides a sample local cog implementation, while multiloading.rb and simple\_external\_cog.rb illustrate how to load and execute cogs from both local sources and external plugin gems (plugin\_gem\_example) within a single workflow execution.
examples/demo · high confidence
Added plugin gem and prompt examples
New example files have been added to the repository, including a Ruby gem example (\plugin-gem-example\) demonstrating \Roast::Cog\ implementations and a simple ERB-based prompt template (\simple\_prompt.md.erb\). These additions provide reference implementations for users looking to create plugins or define prompt structures.
examples/plugin-gem-example, examples/prompts · high confidence
Expanded workflow examples covering agents, async execution, and control flow
The examples directory now includes comprehensive demonstrations of new and existing workflow capabilities. Users can see how to configure and use AI agents (including the new Pi provider and multi-prompt sessions), run commands asynchronously in the background, and manage complex control flow with \map\, \reduce\, \repeat\, \skip\, and \break\. Additional examples illustrate accessing outputs from sub-executors via \from\ and \collect\, customizing logging, handling shell sanitization, and working with workflow parameters and temporary directories.
examples · high confidence
Initial Sorbet and Tapioca configuration setup
This change introduces the initial configuration for Sorbet type checking and Tapioca RBI generation. A new \sorbet/tapioca/config.yml\ file is added to exclude \yard\ and \yard-sorbet\ gems from RBI generation, as they are unused by the project. Additionally, a \sorbet/tapioca/require.rb\ file is created to explicitly require dependencies such as \active\_support\, \async\, \rainbow\, and \sqlite3\, ensuring the type checker has the correct context for these libraries.
sorbet/tapioca · high confidence
Initial Sorbet type-checking configuration
The project now includes a Sorbet configuration file that enables the experimental 'requires\_ancestor' and 'rbs\_comments' features. The configuration sets the root directory to the project root and excludes standard directories such as .git, .idea, tmp, vendor, test, bin, and the examples/demo folder from type checking.
sorbet · high confidence
Introduce Agent and Chat cogs with multi-provider support and truncation detection
The \lib/roast/cogs\ directory now includes new \Agent\ and \Chat\ cog implementations. The \Agent\ cog enables coding tasks with local filesystem and tool access, supporting Claude and Pi providers. The \Chat\ cog provides pure LLM interaction without local system access and now supports OpenAI, Anthropic, Perplexity, Gemini, and Amazon Bedrock. A key behavioral change in the \Chat\ cog is the detection of max-token truncation, which raises an error if the response is cut off, and improved stats reporting that falls back to the chat model ID when the response lacks one.
lib/roast/cogs · high confidence
Introduce Pi coding agent provider
Added a new Pi coding agent provider to the agent cogs system. This change introduces the \PiInvocation\ class which manages the execution of the Pi agent, including handling prompts, sessions, tool calls, and timeouts, and exposes configuration options for model selection, working directories, and raw message dumping.
lib/roast/cogs/agent/providers/pi · high confidence
Introduce configurable Agent cog with provider selection, multi-prompt chaining, and usage tracking
The Agent cog now supports configurable agent providers (defaulting to Pi, with Claude also available) and allows overriding the default via the ROAST\_DEFAULT\_AGENT\_PROVIDER environment variable. Users can chain multiple prompts in a single session for complex tasks, set execution timeouts, and view detailed usage statistics including token counts, costs, and per-model breakdowns.
lib/roast/cogs/agent · high confidence
Introduce configurable chat cog with multi-provider support and session management
The chat cog now supports multiple LLM providers (OpenAI, Anthropic, Perplexity, Gemini, and Amazon Bedrock) and allows users to select a provider via configuration or the ROAST\_DEFAULT\_CHAT\_PROVIDER environment variable. It introduces session management to maintain conversational context across multiple invocations, enabling multi-turn interactions. The cog validates API keys and base URLs per provider, raising InvalidConfigError for misconfigurations, and provides input/output structures for prompts, responses, and session state.
lib/roast/cogs/chat · high confidence
Introduce core Roast::Cog infrastructure for configuration, inputs, outputs, and registry
This change establishes the foundational classes for the Roast workflow engine within the \lib/roast/cog\ directory. It adds \Roast::Cog::Config\ to manage cog settings (including async execution and failure handling), \Roast::Cog::Input\ and \Roast::Cog::Output\ to define how data is passed into and retrieved from cogs (with robust JSON and number parsing in outputs), and \Roast::Cog::Registry\ to manage the available cogs in a workflow. These components provide the structural basis for defining and executing cogs.
lib/roast/cog · high confidence
New Sorbet RBI shims for Roast workflow execution and configuration
Added Sorbet RBI shim files for \Roast::CogInputContext\, \Roast::ConfigContext\, and \Roast::ExecutionContext\. These shims define the type signatures and documentation for workflow input methods (such as \target!\, \targets\, \arg?\, \kwarg\), global and cog-specific configuration options (including \async!\, \parallel\, \timeout\, and \provider\ settings for the \agent\ cog), and execution scope controls (including \outputs!\, \outputs\, \call\, and \map\). This improves static type checking and IDE autocomplete for Roast workflow definitions.
sorbet/rbi/shims/lib/roast · high confidence
New internal documentation standards and maintenance tooling
This change introduces a new directory for internal documentation comments (\internal/documentation/comments/\) containing architectural notes and guidelines for both user-facing and developer-facing documentation, establishing standards for cog capabilities (e.g., distinguishing \agent\ vs \chat\ cogs) and IDE shim documentation. It also adds a custom RuboCop cop (\Roast/NoTestClassNesting\) to prevent class nesting in test files that breaks IDE test runners, and introduces two internal maintenance workflows: one to analyze branch diffs for documentation impact and another to update references to deprecated LLM models.
internal · high confidence
New system cogs for calling scopes, mapping collections, and repeating loops
The \lib/roast/system\_cogs\ directory now includes \Call\, \Map\, and \Repeat\ system cogs, enabling workflows to invoke named execution scopes, iterate over collections (with optional parallelism), and loop until a break condition is met. These cogs provide structured input/output handling and integration with the execution manager, allowing users to build more complex, reusable workflow logic within the \Roast\ namespace.
_lib/roast/system\_cog, lib/roast/system\cogs · high confidence
New tutorial series for Roast workflows
Added a comprehensive, multi-chapter tutorial series (chapters 1–6) that guides users through creating and running Roast workflows. The tutorials cover core concepts including the \chat\ and \agent\ cogs, chaining steps, passing targets and parameters, configuring models and providers, control flow with \skip!\ and \fail!\, and creating reusable named scopes with the \call\ cog. The examples demonstrate supported providers like OpenAI, Anthropic, Perplexity, Gemini, and Bedrock, and show how to configure display options and temperature settings.
tutorial · high confidence
Workflow parameter access in config blocks
The \Roast::Cog::Config\ class now exposes methods to inspect workflow invocation details within configuration blocks. Users can access the single target via \target!\, all targets via \targets\, check for flag arguments with \arg?\ and \args\, and retrieve keyword arguments using \kwarg\, \kwarg!\, and \kwargs\. This enables dynamic configuration based on how the workflow was invoked, such as adjusting model settings or display modes based on provided flags or targets.
sorbet/rbi/shims/lib/roast/cog · high confidence
Behavioural changes
Claude provider now formats tool use and result messages for terminal output
The Claude agent provider now displays structured, human-readable summaries of tool interactions (such as Bash, Read, Write, Edit, Glob, Grep, and Skills) in the terminal. Instead of raw JSON or unformatted text, users see concise one-line indicators for tool invocations (e.g., 'BASH ls -la') and results (e.g., 'BASH OK 12 lines · Cloning into roast...'), improving visibility into the agent's actions during execution.
lib/roast/cogs/agent/providers/claude · high confidence
Development environment and tooling overhaul
The project has significantly updated its development setup: the default Ruby version is bumped to 3.4.7, the test suite has migrated from Minitest to RSpec (with corresponding Rake tasks like \minitest\_fast\ replaced by \minitest\_all\ and RSpec-specific configurations), and static analysis now includes Sorbet type checking alongside RuboCop. Configuration files have been restructured with new \.rubocop.yml\ rules, a \dev.yml\ for development commands, and updated \.gitignore\ entries, while legacy documentation files like \CLAUDE.md\ have been removed.
(repo-wide) · high confidence
Introduce new CLI, workflow execution engine, and structured logging system
The \lib/roast\ library has been replaced with a new implementation that introduces a Thor-based CLI (\lib/roast/cli.rb\) for running workflows, a new \Cog\ and \ExecutionManager\ architecture for defining and running workflow steps, and a structured event-driven logging system (\lib/roast/event.rb\, \lib/roast/event\_monitor.rb\, \lib/roast/log.rb\) that captures stdout/stderr and provides colored, path-aware log output. This change also adds a new \CommandRunner\ for safe shell command execution with timeout and streaming support, and a \ConfigManager\ for scoped workflow configuration.
lib/roast · high confidence
Migrates CLI framework and adopts Zeitwerk autoloading
The library has replaced the CLI::Kit framework with a custom implementation using Ruby's standard library (optparse) and Zeitwerk for autoloading, removing the previous CLI::Kit dependencies and command registry structure. This change also consolidates third-party gem requirements (such as ActiveSupport, Rainbow, and RubyLLM) into the main entry point and removes legacy command files like 'example' and 'help' that were tied to the old CLI structure.
lib · high confidence
Structured formatting for Pi agent tool calls and results
The Pi agent provider now includes dedicated formatters for tool calls (bash, read, write, edit, grep, find) and their corresponding results. This change introduces concise, human-readable terminal output for agent actions, such as displaying truncated command previews for bash, file paths and line ranges for read/write, and match counts for grep and find, ensuring that complex tool interactions are presented clearly in the UI.
lib/roast/cogs/agent/providers/pi/messages · high confidence
Structured message types for Claude provider
The Claude provider now uses dedicated message classes (AssistantMessage, ResultMessage, SystemMessage, TextMessage, ThinkingMessage, ToolResultMessage, ToolUseMessage, UnknownMessage, and UserMessage) to parse and format incoming data. This change introduces specific handling for different message types, including support for Claude's 'thinking' messages, improved error message extraction in ResultMessage, and explicit field ignoring to clean up internal metadata before processing.
lib/roast/cogs/agent/providers/claude/messages · high confidence
Support for multiple prompts and session reuse in Claude and Pi agents
The Claude and Pi agent providers now handle multiple prompts within a single invocation. Instead of processing only the first prompt, the agents iterate through all provided prompts, chaining them together by reusing the session from the previous step (avoiding unnecessary forking for subsequent prompts). Additionally, usage statistics (stats) from all individual prompt invocations are summed and returned in the final result, giving users a complete view of resource consumption across the entire multi-prompt sequence.
lib/roast/cogs/agent/providers · high confidence
Switches entry point to use Roast::CLI and loads roast-ai
The executable now requires the 'roast-ai' library instead of 'roast' and invokes the CLI via Roast::CLI.start(ARGV), replacing the previous ErrorHandler/EntryPoint pattern. This change aligns the entry point with the Thor-based CLI implementation and simplifies execution flow.
exe · high confidence
Update bin scripts and remove unnecessary binstubs
The bin directory has been updated to include new Bundler-generated binstubs for development tools (rubocop, spoom, srb, srb-rbi, tapioca) and the main application (roast), while removing the custom bin/setup and bin/testunit scripts. The bin/console script has also been refactored to explicitly add the lib directory to the load path before requiring the application, ensuring correct module loading in interactive sessions.
bin · high confidence
Test coverage
Added VCR-based functional test infrastructure for DSL examples; Added comprehensive test suite for Roast CLI and core components; Added functional tests for Roast example workflows; Added test coverage for Agent cog components; Added test coverage for Claude agent provider components; Added test coverage for system cogs; Added test coverage for the Chat cog; Added test support infrastructure for Minitest assertions and test cogs; Added tests for Claude agent message types; Added tests for Cmd and Ruby cog components; Added tests for Pi agent tool call and result message formatting; Added tests for agent provider session and timeout handling; Added tests for the NoTestClassNesting RuboCop cop; Added tests for the Pi agent invocation provider; Added unit tests for core Cog components; Migrate test suite to Minitest and modernize test infrastructure.
Dependencies
Roast gem renamed to roast-ai with updated dependencies and example projects
The gem has been renamed from 'roast' to 'roast-ai' (including the gemspec and internal references), and its runtime dependencies have been updated to use activesupport (\~\> 8.0), async (\>= 2.34), rainbow (\>= 3.0.0), ruby\_llm (\>= 1.13), type\_toolkit (\>= 0.0.5), and zeitwerk (\>= 2.6). The project now includes example projects for a demo and a plugin-gem-example, each with their own Gemfile and Gemfile.lock, demonstrating how to depend on the renamed roast-ai gem. Development dependencies have also been updated to include guard-minitest, minitest-rg, rubocop-sorbet, sqlite3, and tapioca, among others.
(dependencies) · high confidence
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
How this codebase got here
Score
- CAI 53 → 62 (+9.1)
- Rubric changed (rubric-2026.10.5 → rubric-2026.11.2) — scores are not directly comparable.
Lenses
- Code Health 85 → 85 (+0.0)
- Architecture 85 → 84 (-1.7)
- Maturity 34 → 55 (+21.7)
- Readiness 56 → 56 (+0.0)
- Security 87 → 78 (-8.1)
Resolved (2)
- Medium vulnerability: [GHSA redacted] (Gemfile.lock)
- Off-boarding risk: anonymized user #1
New (1)
- Off-boarding risk: anonymized user #1
Changes since last survey
- 1 commits — 1 feature/other, 0 fixes
By area
- (root) — 1 commit
Notable commits
- change: Bump sqlite3 from 2.9.5 to 2.9.6
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
Survey your own repository
Shopify/roast was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.
About this page
- The score is its most recent published measurement, taken on 6 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
- Measured at commit 93b2f5c99982bd3e8df426fa09da02a64a454263 — the exact code this score is about.
- Scored under rubric-2026.11.2 — the same rubric and the same method as every other entry in this index.
- Measured by Canine Code Analyzer (cca.canine.dev) using cca measure-2026.11.2+preprod-5c4e66d9318b.