openclaw/Peekaboo

Peekaboo is a macOS desktop automation platform that enables AI agents to interact with applications through screen capture, accessibility inspection, and input simulation. It provides a secure runtime for background-only automation, featuring receipt-based target validation and signed execution traces to prevent stale-target errors. The system exposes these capabilities via a CLI, a menu-bar application, and an MCP server, supporting agent chat, browser control, and visual feedback overlays.

114 changes.

The project executed a comprehensive architectural overhaul for version 4, introducing a new CLI scaffolding, centralized configuration, and a strict background-only execution policy for the AI agent. This period focused on hardening security through signed receipts and process identity verification while rebuilding the macOS app with a modern SwiftUI lifecycle and establishing the core PeekabooDaemon with configurable operational modes.

19 changes.

This period focused on strengthening the security and reliability of the automation infrastructure through significant architectural refactoring. Key changes included rewriting the browser runtime for isolation, centralizing desktop operation planning with strict target validation, and introducing a dedicated certification controller for code identity verification. These core improvements were supported by extensive new test coverage, shell autocompletion, and specialized tools for game bridge detection and coordinate mapping.

The application now automatically generates concise, 2-4 word titles for sessions using the configured AI provider (Anthropic or OpenAI) instead of static naming. This new \SessionTitleGenerator\ service intelligently selects the best available model based on user configuration and authentication status, supporting specific models like Claude Fable 5, Opus 4.8, and GPT-5.6 Sol. To ensure responsiveness, the title generation includes an 8-second timeout; if the AI request takes too long or fails, the system falls back to a generic "New Session" title, ensuring the user interface remains usable even if the AI service is slow or unavailable.

Apps/Mac/Peekaboo/Services · high confidence

Peekaboo now supports automation of SDL/GPU-rendered game windows (e.g., Firestaff) by reading a JSON accessibility manifest instead of relying on the macOS Accessibility API. This allows commands like \peekaboo see\ and \peekaboo click\ to work with game UI elements that are otherwise invisible to standard accessibility tools. The implementation includes validation of atomic manifests and optional byte limits for safety.

Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/GameBridge · high confidence

Users can now install Peekaboo via Homebrew using the newly added \peekaboo.rb\ formula. This formula installs version 4.3.1, requires macOS Sequoia (15.0) or later, and includes instructions for granting Screen Recording permissions and configuring AI providers (such as OpenAI and Anthropic) for the vision analysis features.

homebrew · high confidence

A new macOS application, PeekabooTestHost, has been added to the CLI directory to serve as a test harness for local integration tests. This host app provides a SwiftUI interface that displays the status of required permissions (Screen Recording and Accessibility) and verifies the availability of the Peekaboo CLI binary. It includes controls to run local tests and view logs, offering a stable window target for automated test scripts to interact with during the test suite execution.

Apps/CLI/TestHost · high confidence

Two new shell scripts are introduced in the Playground tools to aid local development and debugging. The \peekaboo-perf.sh\ script provides a local benchmarking harness that runs Peekaboo CLI commands repeatedly, capturing per-run JSON output and generating summary statistics (mean, stddev, median, p95, min, max) for execution times, with results written to a local artifacts directory. The \playground-log.sh\ script serves as a log viewer for the Peekaboo Playground app, allowing users to filter logs by category (e.g., Click, Text, Space, MCP), search for specific text, adjust log levels, and output results in JSON or color-coded text formats using the macOS \log\ command.

Apps/Playground/scripts · high confidence

The CLI binary now includes an Info.plist, entitlements file, and version.json to define its macOS bundle identity and permissions. The bundle identifier is set to boo.peekaboo.peekaboo, the minimum system version is 15.0, and the app is configured as a UI element (LSUIElement) with accessibility and screen recording usage descriptions. Entitlements disable the app sandbox and allow task port access for debugging.

Apps/CLI/Sources/Resources · high confidence

The Peekaboo CLI now supports automatic command, option, and argument completion in Bash, Zsh, and Fish shells. This feature adds a new \Completions\ module that generates shell-specific scripts by introspecting the CLI's command structure (subcommands, flags, and arguments) via the Commander framework. Users can install these completions to enable tab-completion for the \peekaboo\ command in their respective shells.

Apps/CLI/Sources/PeekabooCLI/CLI/Completions · high confidence

A new command-line utility, cgs-menu-probe, has been added to the experiments directory to help diagnose menu bar visibility issues. It compares window counts retrieved from private CGS APIs (SkyLight) against public CoreGraphics APIs, outputting detailed counts and identifiers to assist in debugging menu bar rendering or visibility problems.

experiments · high confidence

The Playground now includes two new view fixtures to help test accessibility scenarios: HiddenFieldsView, which simulates web-style text fields wrapped in accessibility groups, and PermissionBubbleView, which simulates browser permission prompts with unlabeled buttons.

Apps/Playground/Playground/Views/Fixtures · high confidence

A new Xcode scheme file (Peekaboo.xcscheme) has been added to the project, configuring the build, test, launch, profile, analyze, and archive actions for the Peekaboo.app target. This enables developers to run, test, and debug the Mac application directly from Xcode with the specified build configurations and debugger settings.

Apps/Mac/Peekaboo.xcodeproj/xcshareddata · high confidence

The PeekabooAutomationKit now includes a new PathResolver utility that validates file paths against security risks like traversal attacks and access to system directories, while also handling path expansion and safe filename combination respecting macOS filename limits. Additionally, an extension for NSArray provides a safe isEmpty property to prevent infinite recursion issues caused by linter modifications.

Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Core/Utilities, Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Extensions · high confidence

The CLI now includes an AutomationEventLogger that categorizes and logs automation-related actions (such as scroll, gesture, drag, window, and space events) using the os.Logger subsystem. This provides users with detailed, categorized visibility into automation activities for debugging and monitoring purposes.

Apps/CLI/Sources/PeekabooCLI/Logging · high confidence

The CLI now accepts string arguments for capture mode, image format, and capture focus, automatically converting them to the corresponding internal types. This allows users to specify these options directly via command-line flags without needing to know the internal enum values.

Apps/CLI/Sources/PeekabooCLI/CLI/Parsing · high confidence

A new \ExternalDependencies\ module has been introduced to centralize the management of third-party Swift libraries. This change provides a single import point for key dependencies including Algorithms, AsyncAlgorithms, AXorcist, Commander, Logging, OrderedCollections, and SystemPackage, simplifying access for consumers. Additionally, it exposes a \DependencyInfo\ enum that allows users to programmatically inspect the specific versions of these dependencies currently in use, such as AsyncAlgorithms 1.0.4 and Algorithms 1.2.1, facilitating easier version auditing and consistency checks.

Core/PeekabooExternalDependencies · high confidence

Users can now restrict desktop observation to a specific rectangular area within an exact window by providing an ROI (x,y,width,height) in capture options, with the system validating bounds, scale, and output size limits. Additionally, observation results now include SHA-256 content digests for raw and annotated screenshots, allowing callers to verify that captured images have not been tampered with or corrupted, with support for both legacy unsigned and new signed digest modes.

Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/Observation · high confidence

The Peekaboo macOS application project has been initialized with a new Xcode workspace and project file. This configuration establishes the build target for the Peekaboo.app, linking essential frameworks including AppIntents, KeyboardShortcuts, and Sparkle, while integrating local packages such as AXorcist, PeekabooCore, and PeekabooUICore to support the application's core functionality and user interface.

Apps/Mac/Peekaboo.xcodeproj · high confidence

The Apps/CLI directory now contains the initial project structure, including a .gitignore to exclude test output, a .swiftformat configuration for Swift 6.2 code styling, a .swiftlint.yml inheriting parent rules, a comprehensive CHANGELOG.md documenting versions from 4.3.0 to the current unreleased state, a README.md, and a main.swift entry point. This establishes the foundational build and documentation infrastructure for the CLI application.

Apps/CLI · high confidence

The ToolRegistry module now provides a centralized registry that exposes Peekaboo's automation capabilities (such as see, click, and window management) to the AI agent runtime. This change introduces a unified \PeekabooToolDefinition\ structure that standardizes how tool metadata, parameters, and categories are defined, allowing the same tool definitions to serve both the CLI and the agent. The registry automatically converts agent tool schemas into these unified definitions, ensuring that the agent receives a consistent, background-only, shell-free catalog of tools for UI automation.

Core/PeekabooCore/Sources/PeekabooAgentRuntime/ToolRegistry · high confidence

The PeekabooAgentRuntime now exposes three new MCP tools: \app\ for managing application lifecycle (launch, quit, focus, switch, list) with strict foreground/background policy enforcement; \analyze\ for performing AI-driven image analysis and OCR on local files with configurable provider models; and \action\ for invoking named accessibility actions (e.g., AXPress) on specific UI elements. These tools integrate with the existing automation and observation services to provide structured, receipted outcomes for desktop interaction.

Core/PeekabooCore/Sources/PeekabooAgentRuntime/MCP/Tools · high confidence

Added AudioInputService to handle microphone recording and audio transcription. The service manages recording state, enforces a 25MB file size limit, supports common audio formats (wav, mp3, m4a, etc.), and integrates with an AI service for transcription using OpenAI credentials. It exposes properties for recording status and duration, and handles errors such as permission denials, session errors, and API key issues.

Core/PeekabooCore/Sources/PeekabooAutomation/Services/Audio · high confidence

The AI service now includes a new \AnalyzeImageFile\ component that enforces a 10 MiB size limit and performs bounded reads to prevent resource exhaustion when processing image files. Additionally, the \PeekabooAIService\ now supports custom OpenAI-compatible and Anthropic-compatible model providers, allowing users to configure external endpoints with custom base URLs, API keys, and additional headers.

Core/PeekabooCore/Sources/PeekabooAutomation/Services/AI · high confidence

The PeekabooAutomationKit now includes a new set of utility modules in the Utilities directory to standardize and improve desktop automation. AgentDisplayTokens provides a shared set of status markers and tool-specific glyphs for consistent CLI and app output. FocusUtilities introduces a comprehensive window focus management system with automatic focusing, space switching, and verification retries. SpaceManagementService and its supporting files (SpaceCGSPrivateAPI, SpaceModels, SpaceWindowMutationDispatcher) enable advanced macOS Spaces (virtual desktop) operations, including listing, switching, and moving windows between spaces using private CoreGraphics APIs. WindowFiltering adds configurable thresholds for determining which windows are renderable or listable. WindowIdentityUtilities and WindowListMapper provide robust window identification and listing capabilities, combining CGWindow and ScreenCaptureKit data with caching for performance.

Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Utilities · high confidence

The daemon location now includes the \PeekabooDaemon\ class, which serves as the central control point for the background service. It supports multiple operational modes (auto, manual, MCP, embedded MCP) via a new \Configuration\ struct, allowing users to tailor behavior such as window tracking, idle timeouts, and bridge socket paths. Additionally, a new \DaemonBrowserDiagnostics\ component provides a way to observe browser health and connection status without tightly coupling to the browser lifecycle, improving diagnostic visibility for the daemon's state.

Core/PeekabooCore/Sources/PeekabooCore/Daemon · high confidence

This change adds the foundational data structures for the new PeekabooAutomationKit, defining the types used for screen capture, UI automation, and tool execution. It introduces models for window and application inventory (WindowInfo, TargetApplicationInfo), capture session configuration and results (CaptureOptions, CaptureSessionResult, CaptureCadence), and UI automation snapshots (UIAutomationSnapshot, UIElement). It also defines the unified output structure (UnifiedToolOutput) used by CLI, Agent, and MCP surfaces, along with conversation session models (ConversationSession) and automation action types (AutomationAction).

Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Core/Models · high confidence

The automation kit now includes a new support layer for managing desktop state consistency and snapshot lifecycle. A new \DesktopMutationWatermarkStore\ tracks cross-process mutation boundaries to prevent stale UI observations, while \DesktopOperationLaneCoordinator\ enforces strict file-locking for global, process, and window-scoped operations with a 15-second timeout. These are integrated into a new \InMemorySnapshotManager\ that handles snapshot creation, storage, and pruning in memory, supporting mutation leases to ensure observations are refreshed after changes.

Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/Support · high confidence

The CLI now includes a new \browser\ subcommand that wraps the browser MCP tool, allowing users to perform DOM and page operations such as status checks, navigation, snapshots, clicks, and screenshots. This command enforces a security model where background-only actions are available by default, while foreground user activation and arbitrary script evaluation require the explicit \--foreground\ flag. Additionally, a new \BrowserHandoffReceiptStore\ has been added to securely manage signed bridge receipts, ensuring atomic file operations and preventing symlink or race-condition attacks during browser target handoffs between processes.

Apps/CLI/Sources/PeekabooCLI/Commands/MCP · high confidence

A new standalone CLI tool, \peekaboo-certification-controller\, has been added to perform strict, source-owned certification of code identity and process behavior. This controller supports four distinct operational modes: standard multi-target certification, observe-only foreground semantic observation, held-pointer embedding protocol validation, and monitor attestation. It enforces closed-schema JSON plans, validates Apple-anchored code signatures against expected Team IDs and source commits, and ensures process integrity by verifying live code signature hashes, executable paths, and window bounds before executing certified actions or emitting receipts.

Apps/CLI/Sources/PeekabooCertificationController · high confidence

The \scripts/final-qualification\ directory now contains a suite of source-owned tools that close the final physical qualification handshakes. This includes an integrated Computer Use emitter calibrator (Swift) that passively observes live input to authenticate the event source, a crash inventory tool (Node.js) to capture and compare DiagnosticReports, and an executable policy scanner (Node.js) to detect prohibited patterns like AppleScript or virtualization frameworks in native binaries. The suite also provides Node.js scripts to project raw receipts into live bindings, construct a live execution plan, publish coordinator markers atomically via a Swift helper, and manage the coordinator launch lifecycle. These tools enforce strict file permissions, identity verification, and protocol version requirements (1.31/1.32) to ensure evidence integrity.

scripts/final-qualification · high confidence

The PeekabooUICore module now includes a new Inspector feature set, adding \InspectorView\ for the main UI, \InspectorPermissionState\ to manage accessibility permission checks and status, and \OverlayManager\ to handle visual overlays and UI element inspection. This introduces the core logic for monitoring applications, detecting hovered elements, and displaying element details within the inspector interface.

Core/PeekabooUICore/Sources/PeekabooUICore/Inspector · high confidence

Added a new SwiftUI-based permissions checklist view that displays the status of required permissions (Screen Recording, Accessibility) and optional permissions (Event Synthesizing). The view allows users to request these permissions directly from the UI and includes a manual refresh button to update status without restarting the app.

Apps/Mac/Peekaboo/Features/Permissions · high confidence

Added new protocol files defining core types for the Peekaboo system. ServiceProtocols.swift introduces a LogLevel enum with trace through critical levels for logging. UIServiceProtocols.swift defines the DetectedElement struct, which represents UI elements with properties for ID, type, bounds, label, value, and enabled state, facilitating UI interaction and detection.

Core/PeekabooProtocols/Sources · high confidence

The PeekabooUICore module now provides a set of reusable SwiftUI components for the inspector interface, including AllElementsView for browsing and filtering detected UI elements, AppSelectorView for choosing target applications and detail levels, ElementDetailsView for displaying element properties, and PermissionDeniedView for handling accessibility permission states. These components are built using the Observation framework for reactive state management and are exported via the PeekabooUICore module for use by the main application.

Core/PeekabooUICore/Sources/PeekabooUICore/Components · high confidence

The system now uses a new \ApplicationIdentifierMatcher\ to resolve application selectors (such as PID, bundle ID, or name) with a strict precedence hierarchy and SHA-256 candidate set proofs, ensuring that automation targets are identified consistently and securely. This change also introduces \ApplicationMutationEligibility\ to verify that a process is actually targetable for mutation based on credentials and runtime host user ID, rather than relying solely on credential presence. These components are integrated into \ApplicationService\ to provide stable, auditable resolution for application discovery, launch, and window listing operations.

Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/System · high confidence

The PeekabooBridge module now exposes a comprehensive daemon status API, allowing clients to monitor the background daemon's health, including its mode, bridge connection details, permission states, snapshot counts, and window tracking metrics. Additionally, a new signed agent execution tracing system has been added, enabling the Bridge to launch and monitor background agent processes with strict security controls, including process identity verification, coordination receipts, and atomic acknowledgement handling to ensure trusted execution.

Core/PeekabooCore/Sources/PeekabooBridge · high confidence

The Mac app now automatically persists conversation sessions to Application Support and generates AI titles from the first user message. A new visual effects system applies native macOS materials on versions 14–25 and Liquid Glass on macOS 26+, with an extension for easy adoption. Launch behavior is refined via a new policy that keeps deployment-owned Bridge hosts unattended (accessory mode) while preserving user Dock preferences, and a new Dock icon manager coordinates this state. Credential handling is centralized in a coordinator that supports draft editing, retry, and migration from legacy preferences, while a tool formatter bridge aligns Mac app tool display with the CLI's compact format.

Apps/Mac/Peekaboo/Core · high confidence

The ToolFormatting module now provides a comprehensive system for formatting agent tool outputs, including a registry that maps specific tool types (such as vision, UI automation, and system tools) to dedicated formatters. This change adds support for rich metadata in tool results via the ToolEventSummary struct, allowing details like coordinates, element labels, and pointer profiles to be preserved and displayed. It also introduces utility functions for formatting keyboard shortcuts with proper symbols and durations with appropriate precision, ensuring consistent and readable output for users interacting with the AI agent.

Core/PeekabooCore/Sources/PeekabooAgentRuntime/ToolFormatting · high confidence

The PeekabooAutomationKit now exposes the \AutomationFeedbackClient\ protocol, which defines the interface for displaying visual feedback such as click ripples, typing captions, scroll indicators, hotkey overlays, and gesture animations. This abstraction allows the automation engine to request visual cues without depending on a specific UI implementation, and includes a default \NoopAutomationFeedbackClient\ that silently ignores these requests when no visualizer is active.

Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit · high confidence

The MCP server implementation in PeekabooAgentRuntime now supports serving over host-supplied transports (such as HTTP and SSE) in addition to the default stdio, allowing integration with external MCP clients. To ensure reliability, a new EOF-draining transport wrapper keeps the server's receive stream alive after input EOF until all dispatched requests have replied, preventing the process from exiting before responses are written. The server also exposes a comprehensive catalog of native tools (including capture, clipboard, and UI automation) and enforces strict tool filtering and argument validation.

Core/PeekabooCore/Sources/PeekabooAgentRuntime/MCP/Server · high confidence

A new \CLIFormatter\ has been introduced in the \PeekabooAgentRuntime\ module to structure how AI agent tool results are displayed in command-line interfaces. This formatter takes \UnifiedToolOutput\ objects and renders them into readable text, including summary briefs, counts, and categorized highlights (primary, warning, info). It specifically enhances the display of application and window lists by adding indexed numbering, bundle identifiers, process IDs, active/hidden status flags, and detailed window metadata such as bounds, screen information, and minimized/off-screen states. This change provides users with clearer, more structured feedback when interacting with the agent's automation capabilities via the CLI.

Core/PeekabooCore/Sources/PeekabooAgentRuntime/Formatting · high confidence

The Playground app now includes a suite of new SwiftUI views designed to exercise and log specific user interactions. These fixtures provide dedicated screens for testing click behaviors (including single, secondary, and disabled states), standard UI controls (sliders, checkboxes, radio buttons, steppers, and date pickers), file dialog panels (save/open with various modes and alerts), drag-and-drop operations (zones, list reordering, and free-form dragging), keyboard input (key presses, modifier combinations, and sequence recording), and scroll/gesture handling (vertical/horizontal scrolling with offset tracking and accessibility configuration). Each view logs actions to the central action logger and exposes accessibility identifiers to facilitate automated testing and semantic witness verification.

Apps/Playground/Playground/Views · high confidence

The PeekabooAgentRuntime module introduces three configurable agent enhancements. Context injection (AgentEnhancementOptions.contextAware) automatically gathers and injects desktop state—such as the focused app, window title, cursor position, and clipboard—into the LLM prompt before each turn. Visual verification (verifyActions) uses a lightweight AI model to analyze post-action screenshots, allowing the agent to confirm that actions like clicks, typing, or app launches succeeded and to retry if they did not. Smart capture (smartCapture) reduces unnecessary API calls by skipping screenshots when the screen has not changed, with configurable sensitivity and region-focused capture options.

Core/PeekabooCore/Sources/PeekabooAgentRuntime/Agent · high confidence

This change introduces new support components within the PeekabooAgentRuntime module that enhance how the AI agent interacts with the desktop and manages available tools. The new DesktopContextService automatically gathers and formats current desktop state—including the focused window, cursor position, clipboard content, and recent apps—for injection into agent prompts, providing the LLM with immediate context awareness. Additionally, the ToolFiltering module implements a robust allow/deny list system for agent and MCP tools, resolving filters from both environment variables (PEEKABOO\_ALLOW\_TOOLS, PEEKABOO\_DISABLE\_TOOLS) and configuration files, while also dynamically hiding tools that are unavailable under the current input strategy policy. The UIElementSummary struct standardizes the data structure for observed UI elements, ensuring consistency between CLI JSON output and MCP tool responses.

Core/PeekabooCore/Sources/PeekabooAgentRuntime/Support · high confidence

PeekabooUICore now includes two new visualization presets for shared Inspector components: AnnotationPreset, which displays rectangle overlays with persistent labels, and InspectorPreset, which uses circle indicators with hover effects. These presets define specific styling rules for element states (normal, hovered, selected, disabled) and label badges, allowing users to choose between a dense annotation view or an interactive inspector view.

Core/PeekabooUICore/Sources/PeekabooUICore/Presets · high confidence

The CLI now includes a \bridge status\ command to probe and report on available Bridge hosts (including permission and capability details) and a \bridge receipt validate\ command to verify signed protocol 1.29 receipt bundles. Additionally, a new \\_agent-execution-trace\ subcommand allows running a coordinator-owned, signed background Agent execution with explicit trust and timeout controls.

Apps/CLI/Sources/PeekabooCLI/Commands/Core · high confidence

The scripts directory now includes a comprehensive set of automation tools: a \pblog\ unified log viewer for debugging Peekaboo subsystems, a Ruby-based \artifact-tree-manifest\ for generating signed, verifiable artifact catalogs, and an \atomic-rename-exclusive\ utility for safe, race-free file publishing. Build workflows have been expanded with dedicated scripts for the Swift CLI (\build-cli-standalone.sh\, \build-swift-arm.sh\, \build-swift-debug.sh\), the macOS app (\build-mac-debug.sh\), and terminal qualification artifacts (\build-node-runtime-macos.sh\, \build-playground-artifact.sh\). Additionally, a new documentation site builder (\build-docs-site.mjs\) generates the public documentation hub, and a \background-computer-use-catalog.json\ defines the test cases and invariants for background automation.

scripts · high confidence

Peekaboo now uses a unified \ConfigurationManager\ to manage settings, introducing a hierarchical precedence where command-line arguments override environment variables, which override the new \\~/.peekaboo/config.json\ file, which overrides built-in defaults. The configuration file supports JSONC (JSON with comments) and \${VAR\_NAME}\ environment variable expansion. Sensitive credentials are now stored separately in a \\~/.peekaboo/credentials\ file with strict 0o600 permissions, and the system automatically migrates legacy hardcoded API keys from the config file to this new credentials store. The manager also supports custom AI providers (OpenAI/Anthropic compatible) with validation and live probing, and introduces specific toggles for visualizer elements like accessibility bounding boxes.

Core/PeekabooCore/Sources/PeekabooAutomation/Configuration · high confidence

This change introduces a new \PeekabooServices\ container that centralizes the initialization of core capabilities (screen capture, automation, browser, agent, etc.) and adds a \PeekabooBridgeRemoteBrowserSessionTransport\ to handle authenticated remote browser session lifecycles (bootstrap, connect, execute, disconnect) via the bridge. It also adds \PeekabooServices+Automation\ for high-level UI automation with snapshot management, \PeekabooServices+BrowserBridge\ and \PeekabooServices+BrowserSessions\ for browser connection and scoped session execution, \PeekabooServices+Agent\ for agent service refresh with multi-provider support, \PeekabooPermissionOnboardingPolicy\ to manage permission onboarding decisions, and \PeekabooServicesVisualizerInit\ to ensure visualizer connections for CLI usage.

Core/PeekabooCore/Sources/PeekabooCore/Support · high confidence

A new standalone helper executable (Helpers/MenuBarHelper) has been added to the project. This tool initializes an AppKit context to ensure proper GUI WindowServer connectivity, checks for screen recording permissions, and uses private CoreGraphics (CGS) APIs to enumerate and output the IDs of all visible menu bar windows as JSON. It serves as a dedicated utility for retrieving menu bar state, distinct from the main application logic.

Helpers · high confidence

Added \background-computer-use-probe.swift\, a standalone Swift tool that monitors system state (cursor, clipboard, window focus) and emits structured heartbeats with integrity commitments. Added \test-fixture-metadata.rb\ to inspect and classify macOS extended attributes (xattrs) on test fixtures, specifically detecting persistent provenance metadata. Added \test-playground-fixture-compatibility.sh\, a bash harness that validates prebuilt Playground artifacts by running shared and terminal validators, enforcing strict signature checks, and asserting that corrupted receipts, missing locks, or modified plist fields are correctly refused.

scripts/support · high confidence

The Playground app now features a unified ActionLogger that records user interactions (clicks, text input, menus, windows, scrolling, dragging, keyboard, focus, gestures, dialogs, and controls) with timestamps, categories, and details, visible in a new LogViewer window with filtering, search, and export capabilities. The main interface is organized into a tabbed layout (Click Testing, Text Input, Dialogs, Controls, Scroll & Gestures, Window, Drag & Drop, Keyboard) with a status bar showing action counts and the last action. Additionally, the app exposes semantic accessibility witnesses (via PlaygroundSemanticWitness) to allow external verification of UI state changes, and provides fixture window shortcuts for quick access to specific testing contexts.

Apps/Playground/Playground · high confidence

The Playground app project structure has been created, establishing the Xcode project configuration for the application. This setup links the app to the AXorcist and PeekabooCore packages and explicitly includes the AppIntents framework, enabling the app to utilize system-level intent metadata and interactions.

Apps/Playground/Playground.xcodeproj · high confidence

The macOS status bar popover has been rebuilt to display a chronological Unified Activity Feed that shows session messages, tool executions, and thinking states in a single scrollable list. The menu bar icon now features an animated ghost silhouette that floats and glances while the agent is processing, replacing the previous static icon. Additionally, the status bar can now display icons for automated applications (automation targets) that are currently active, and the popover includes a dedicated input field for submitting new tasks directly from the menu bar.

Apps/Mac/Peekaboo/Features/StatusBar · high confidence

The Mac app's main interface has been rebuilt to feature a dedicated session sidebar with search, duplicate, and export capabilities, replacing the previous session dialog. The chat view now includes real-time progress indicators and animated SF Symbols that visually reflect the status of tool executions (such as bouncing for clicks or pulsing for typing), providing immediate visual feedback on agent actions. Additionally, helper utilities for model name formatting and session duration tracking have been added to enhance the user experience.

Apps/Mac/Peekaboo/Features/Main · high confidence

The CLI now supports structured logging and machine-readable output. A new \CLILogger\ provides configurable log levels (trace through critical) that default to warning, controllable via the \PEEKABOO\_LOG\_LEVEL\ environment variable or the \--log-level\ flag. When JSON output is enabled, logs are buffered rather than written to stderr, ensuring they do not corrupt the structured command results. Additionally, \DeferredCommandOutput\ captures stdout and stderr during command execution and replays them only after the primary JSON result is emitted, preserving output integrity for both interactive and automated (CI/piped) environments.

Apps/CLI/Sources/PeekabooCLI/CLI/Output · high confidence

The automation kit now includes a new \CaptureCoordinateMapper\ service that allows click coordinates to be converted from image-pixel or normalized spaces into global display points. This enables automation actions to remain accurate even when the captured reference image differs in size or resolution from the live screen, by mapping coordinates relative to the logical bounds of the original capture rather than assuming a fixed screen layout.

Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/Core · high confidence

A change to existing behaviour in (repo-wide) — 1647 commits (518 fixs), 24 files.

(repo-wide) · low confidence · unverified

The agent chat command now features a redesigned terminal interface with a dedicated event delegate that renders tool calls, thinking steps, and assistant messages in real-time. Users can now queue prompts during active agent runs, cancel turns with the Escape key, and use audio input for transcription. The command structure has been migrated to the Commander library, introducing subcommands for run, resume, sessions, and chat, along with new flags like --allow-foreground and --no-desktop-context to control UI authority and context collection.

Apps/CLI/Sources/PeekabooCLI/Commands/AI · high confidence

The CLI now enforces a security release gate for Agent execution via \AgentExecutionReleaseGate\, which validates environment descriptors and cryptographic challenges before allowing the agent to run, while also restricting child process creation limits. Additionally, \Version.swift\ introduces a structured metadata system that resolves version details from the info dictionary or falls back to defaults, exposing git commit, branch, and build date information for better provenance tracking.

Apps/CLI/Sources/PeekabooCLI · high confidence

The \app launch\, \app relaunch\, and \app unhide\ subcommands now enforce strict foreground consent: launching a new instance, delivering open targets (URLs/documents), or unhide-activating an app will fail if the \--foreground\ flag is not provided. This prevents background automation from unexpectedly activating applications. The \app list\ command has been updated to filter out hidden and background apps by default, requiring \--include-hidden\ or \--include-background\ to view them. Additionally, the \app quit\ command now supports \--except\ to exclude specific apps when quitting all, and \app relaunch\ supports a \--wait\ duration to pause between termination and restart.

Apps/CLI/Sources/PeekabooCLI/Commands/System · high confidence

The browser automation engine has been completely rewritten to provide stronger isolation, security auditing, and reliability. The new runtime introduces a session pool that isolates untrusted historical hosts and binds channel connections to native endpoints, ensuring that blocked calls in one session do not block others. It enforces strict auditing of the underlying Chrome DevTools MCP provider (pinned to version 1.10.1) by verifying code signatures and patching dependencies to disable unsafe redirects and enforce explicit reconnection logic. Execution results now distinguish between read-only and mutating actions, allowing the system to correctly report partial failures and prevent unsafe retries. Additionally, the runtime adds foreground-only receipt-bound DOM clicks, persists browser MCP sessions across shutdowns, and centralizes action result transforms for consistent behavior.

Core/PeekabooCore/Sources/PeekabooAgentRuntime/Browser · high confidence

The CLI configuration layer has been refactored to delegate core management tasks to PeekabooCore while introducing a new centralized CommandRegistry. This registry organizes all CLI commands into specific categories (core, interaction, system, vision, ai, mcp) and exposes their definitions, enabling structured command discovery and consistent metadata handling across the tool.

Apps/CLI/Sources/PeekabooCLI/CLI/Configuration · high confidence

The Peekaboo CLI executable now checks an AgentExecutionReleaseGate before running. If the gate is configured and denies execution, the CLI exits with a configuration error instead of proceeding. In debug builds, a special probe argument allows testing the gate's process-creation limits without running the full CLI.

Apps/CLI/Sources/PeekabooExec · high confidence

The CLI helper modules have been rewritten to depend on the \PeekabooServiceProviding\ protocol rather than direct system calls, centralizing logic for drag destination resolution, menu bar click verification, and permission status reporting. This change introduces robust process-waiting utilities with hard timeouts to prevent the CLI from hanging on wedged child processes, and adds detailed permission hints to clarify which host application requires specific macOS grants when using the remote bridge. Additionally, new helpers for JSON formatting, terminal color output, and time formatting have been added to improve CLI usability and output clarity.

Apps/CLI/Sources/PeekabooCLI/Helpers · high confidence

The interaction commands (click, drag, move, action, etc.) have been refactored to use the Commander framework and a centralized runtime pattern. This change introduces stricter validation for background delivery, requiring explicit targets (app, window, or snapshot) for keyboard and coordinate-based actions to prevent mis-delivery. It also adds support for human-like cursor profiles in drag and move operations, enforces foreground consent for actions that can raise UI (like AXPress), and standardizes JSON output envelopes. Users will see more robust error handling, clearer help text, and improved reliability when automating background tasks.

Apps/CLI/Sources/PeekabooCLI/Commands/Interaction · high confidence

The CLI introduces the ApplicationResolvable protocol to standardize how commands identify target applications. Users can now explicitly target a process by its ID using the --pid flag or by passing a 'PID:\<id\>' string to the --app flag. The system validates inputs to ensure consistency: if both --app and --pid are provided, they must refer to the same process ID, otherwise the command fails with an error. This change centralizes the logic for resolving application identifiers and prevents ambiguous or conflicting targeting.

Apps/CLI/Sources/PeekabooCLI/CLI/Protocols · high confidence

The Peekaboo CLI now uses the Commander framework for command routing, execution, and help rendering. This introduces a new runtime pipeline that handles argument resolution, command execution, and result envelope formatting. To assist users upgrading from previous versions, the CLI now provides specific migration guidance for removed commands (e.g., 'hotkey' → 'peekaboo press'), options, and subcommands, ensuring a smoother transition to the v4 command structure.

Apps/CLI/Sources/PeekabooCLI/CLI · high confidence

The CLI now includes a BuildStalenessChecker that runs on debug startup to detect if the binary is out of sync with the current git state, comparing both the embedded commit hash and file modification times against the working tree, and aborting with a clear rebuild instruction if a mismatch is found. Error reporting has been refactored to use structured ResultEnvelopeError and DesktopActionFailure types, providing consistent JSON output envelopes and detailed human-readable diagnostics (including screen capture ownership checks) for both JSON and console modes.

Apps/CLI/Sources/PeekabooCLI/CLI/Utilities · high confidence

The automation strategy layer has been refactored to centralize how desktop input operations are planned, executed, and validated. Operations are now modeled as \DesktopOperationPlan\ structs that define a clear separation between action-based and synthesis-based input strategies, with an executor (\DesktopOperationExecutor\) that routes execution through a coordination lane. Target identity is strictly enforced via \DesktopActionTargetReceipt\ and \DesktopTargetEvidenceAdapter\, ensuring that mutations are bound to specific process generations and window identities. A new \DesktopOperationSnapshotReceiptValidator\ revalidates snapshot evidence before dispatch to prevent stale-target failures, while \BackgroundKeyboardTargetPlanner\ handles completeness checks and eligibility for background keyboard input. This change improves reliability by ensuring that automation actions are always targeted against verified, generation-pinned application and window states.

Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Strategy · high confidence

The Peekaboo agent now operates under a strict background-only execution ceiling by default, preventing foreground UI interactions, app activation, and shell commands unless explicitly permitted. This change introduces a comprehensive system prompt (\AgentSystemPrompt.swift\) that guides the agent to use receipt-pinned raw presses, snapshot-pinned typing, and exact targeted dialog mutations while refusing foreground routes. Users will see the agent fail-closed on foreground requests and must use \need\_info\ if a foreground-capable session is required, ensuring automation respects user focus and security boundaries.

Core/PeekabooCore/Sources/PeekabooAgentRuntime/Agent/Tools · high confidence

The capture service now enforces strict integrity checks on all captured artifacts, verifying PNG structure and CRC checksums, validating video file ownership and invariants, and ensuring metadata matches the in-memory result before publication. Legacy screen capture paths have been refactored to use a dedicated operator that validates PNG output via a custom validator, supports private ScreenCaptureKit window lookups with safe fallbacks to system screencapture, and ensures exact window bounds and scale are respected. These changes improve reliability and security of captured images and videos, particularly in complex multi-display or concurrent capture scenarios.

Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/Capture · high confidence

The Xcode project for the Inspector application has been renamed from PeekabooInspector to Inspector. The build configuration now links against the AXorcist, PeekabooUICore, and PeekabooCore packages, and explicitly includes the AppIntents framework to support metadata linking on macOS.

Apps/PeekabooInspector/Inspector.xcodeproj · high confidence

The Inspector window has been updated to ensure it accepts mouse events (ignoresMouseEvents set to false), which is critical for local monitor functionality to work correctly. Additionally, the window now displays the complete Inspector view from PeekabooUICore, ensuring all session activity including tool execution is visible to the user.

Apps/Mac/Peekaboo/Features/Inspector · high confidence

The Inspector application now includes dedicated asset catalog entries for its macOS app icon and accent color. The app icon is configured with standard resolutions (16x16 to 512x512) for macOS, and an empty accent color definition has been added to support future theming.

Apps/PeekabooInspector/Inspector · high confidence

The PeekabooUICore module now includes a complete set of new SwiftUI views for rendering UI element overlays: AllAppsOverlayView displays elements across all applications, AppOverlayView handles per-application elements, OverlayView renders individual element highlights with labels and styles, and OverlayWindowController manages the underlying transparent NSWindows and screen-change monitoring. This refactors the overlay rendering logic into a modern, observation-driven architecture while introducing new visual behaviors for hover/selection states and screen adaptation.

Core/PeekabooUICore/Sources/PeekabooUICore/Overlay · high confidence

The Peekaboo MCP runtime now enforces a strict background-only execution policy by default, preventing tools from mutating shared system UI surfaces (such as the Dock, Control Center, or Siri) and requiring explicit foreground authorization for privileged operations. To prevent unsafe mutations, the system introduces receipt-bound mutation authority that pins desktop targets to specific process generations and window identities, ensuring that background actions fail closed if the target application or window changes identity before dispatch. Additionally, the runtime now includes rigorous semantic validation for tool arguments, rejecting undeclared properties and unsafe numeric inputs, and isolates browser sessions to prevent cross-caller observation or operation.

Core/PeekabooCore/Sources/PeekabooAgentRuntime/MCP · high confidence

The automation utility layer now includes a new \TypedValue\ enum that replaces type-erased patterns with a strongly-typed representation for heterogeneous data, complete with JSON conversion and migration helpers for legacy \AnyCodable\ usage. Additionally, a new \AIProviderParser\ utility has been added to parse provider configurations from string formats, enforce restrictions on unsupported legacy models (such as older GPT and Claude versions), and determine default models based on available providers like OpenAI, Anthropic, Gemini, MiniMax, and Ollama.

Core/PeekabooCore/Sources/PeekabooAutomation/Utils · high confidence

The Peekaboo agent runtime now uses dedicated formatters for every tool category—Application, Communication, Dock, Element, Menu/Dialog, System, UI Automation, and Vision—replacing the previous unified approach. Users will see richer, structured summaries for tool actions and results, including detailed breakdowns for app lifecycle events (launch, quit, focus), element queries (with confidence scores, position, and state), menu navigation paths, shell command outputs (with exit codes and timing), and UI gestures (clicks, typing, scrolling, dragging). This change ensures that the agent's tool usage is presented with consistent, human-readable context across the entire automation surface.

Core/PeekabooCore/Sources/PeekabooAgentRuntime/ToolFormatting/Formatters · high confidence

The PeekabooFoundation module now provides the canonical types and policies that govern automation safety and reliability. It introduces AccessibilityActionPolicy to classify actions as background-safe or foreground-capable, enforcing explicit foreground consent for UI-altering operations. DesktopActionOutcome and its sequence accumulator replace legacy boolean flags with a rigorous state machine that distinguishes between definite and possible mutations, ensuring that retry safety is never granted when dispatch is uncertain. Additionally, BrowserToolActionSemantics auditors Chrome DevTools MCP tools into read-only and mutating categories to prevent unintended page changes, while ApplicationLifecycleRefusalError standardizes failure reporting for background launch and unhide refusals.

Core/PeekabooFoundation · high confidence

The automation service protocols now require exact process-generation identities (ApplicationProcessIdentity) for all application mutations (launch, relaunch, quit, activate, hide) and dialog operations, ensuring that actions are pinned to the specific process instance discovered at selection time rather than relying on reusable PIDs. This change adds validation logic to reject mutations if the target process has restarted, and introduces signed operation receipts and selection proofs (DialogDiscoverySelectionProof, ResolvedDialogTargetEvidence) to verify that dialog interactions occurred against the exact, attested window state, significantly improving the reliability and security of background automation by preventing race conditions and stale-target errors.

Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/Core/Protocols · high confidence

The automation kit now provides a comprehensive error management system. New error types like CaptureArtifactCleanupError, CaptureNoValidFramesError, and InputDeliveryIndeterminateError offer detailed, user-friendly diagnostics for capture failures and input delivery uncertainties, with the latter explicitly preventing unsafe retries. A CaptureDiagnosticSanitizer ensures error messages are safely truncated and cleaned of control characters. Additionally, the kit introduces a RetryPolicy and RetryHandler, allowing operations to automatically retry on transient failures (such as timeouts or capture issues) using configurable strategies like standard, aggressive, or conservative backoff.

Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Core/Errors · high confidence

The macOS application has been re-architected to use a modern SwiftUI-based lifecycle (PeekabooApp.swift) with explicit window management for Sessions, Inspector, and Settings, replacing the previous structure. A new Info.plist defines the app as a menu-bar-only utility (LSUIElement=true), registers the 'boo.peekaboo.visualizer' Mach service, and configures Sparkle auto-update checks pointing to the official appcast. Entitlements have been standardized to a clean file, and the app now integrates Tachikoma for AI provider configuration and credential management directly from settings.

Apps/Mac/Peekaboo · high confidence

Users will now see a dedicated onboarding window when launching Peekaboo 3.9.6, explaining that the app has moved to the OpenClaw Foundation signing identity. This change may cause macOS to treat the app as a new client, prompting users to re-grant Screen Recording, Accessibility, and Event Synthesizing permissions if they rely on background hotkeys. The flow includes a checklist of current permissions and a direct link to open the system Settings panel for easy management.

Apps/Mac/Peekaboo/Features/Onboarding · high confidence

The PeekabooCore module has been refactored into an umbrella structure that re-exports its sub-modules (PeekabooAgentRuntime, PeekabooAutomation, and PeekabooVisualizer) via a new PeekabooCoreExports file, simplifying imports for consumers. This change accompanies a broader reorganization of the core codebase into logical directories (Core, AI, Services, Configuration) to improve maintainability and code reuse, as documented in the new README. Additionally, new process termination handling logic has been introduced via ProcessTerminationSignalSource and ProcessTerminationGate to manage asynchronous shutdowns cleanly.

Core/PeekabooCore/Sources/PeekabooCore · high confidence

The Mac app's Settings window has been restructured into six distinct tabs: General, Agent, Providers, Visualizer, Permissions, and About. The new Providers tab centralizes API key management (with secure field support for OpenAI, Anthropic, Grok, Gemini, MiniMax, and Ollama), local Ollama configuration, and a new step-by-step wizard for adding custom AI providers (such as OpenRouter or self-hosted endpoints). The Visualizer tab replaces previous feedback controls with granular toggles for agent cursor, input HUD, and capture indicators, along with sliders for animation speed and effect intensity. The General tab now includes global keyboard shortcut recorders and startup options, while the About tab displays version details and project links.

Apps/Mac/Peekaboo/Features/Settings · high confidence

The menu bar popover has been refactored into a set of modular SwiftUI components (SessionComponents, StatusBarActions, StatusBarContent, StatusBarHeader, StatusBarInput) to provide a more structured and native macOS interface. Users now see a compact header with app status, a main content area that displays the current session's activity feed or recent sessions, and a dedicated input field for asking follow-up questions directly from the status bar. This change improves the visual organization and usability of the status bar interface without altering the underlying agent logic.

Apps/Mac/Peekaboo/Features/StatusBar/StatusBarComponents · high confidence

The main chat view now uses a new set of components to render conversation messages with greater clarity. Users will see distinct visual styles for different message types: thinking messages are highlighted in purple with an animated icon, while error and warning messages are clearly marked with red or orange indicators and include a 'Retry' button for failures. Tool usage is now more transparent; a dedicated icon represents tool interactions, and users can expand a message to view the specific arguments passed to and results returned by each tool. If a tool generates an image, it is displayed inline within the expanded view and can be tapped to open a full inspector. Additionally, running tools show a live progress indicator with an elapsed time counter.

Apps/Mac/Peekaboo/Features/Main/MessageComponents · high confidence

A new SettingsOpener utility and a hidden background window have been added to ensure the Settings window opens reliably when triggered from the menu bar. This addresses issues where standard SwiftUI settings links fail in MenuBarExtra apps by using a hidden window workaround, programmatic window activation, and precise window positioning to bring the Settings interface to the front.

Apps/Mac/Peekaboo/Utilities · high confidence

The CLI now uses a centralized base infrastructure for command execution, replacing ad-hoc implementations with a consistent runtime model. Error handling is standardized via the \ErrorHandlingCommand\ protocol, which maps internal errors to specific \ErrorCode\ values and outputs structured JSON envelopes or human-readable messages depending on the \--json\ flag. Output formatting is unified through the \OutputFormattable\ protocol, ensuring consistent status lines and refusal guidance for all commands. A new \CommandRuntime\ and \CommandRuntimeOptions\ system centralizes configuration for logging, capture engine preferences, and remote host selection, while \CommanderBinder\ handles the injection of these options into commands. This change ensures that all CLI commands behave consistently regarding error reporting, help rendering, and runtime context.

Apps/CLI/Sources/PeekabooCLI/Commands/Base · high confidence

The CLI now uses a shared set of command-layer utilities to standardize how users target applications and windows, manage focus, and resolve interaction coordinates. Users can now target windows deterministically using the new --window-id flag, alongside existing --app, --pid, --window-title, and --window-index selectors. Focus behavior is controlled via a consistent set of flags (--foreground, --no-auto-focus, --focus-timeout, --focus-retry-count, --space-switch, --bring-to-current-space), with background input delivery becoming the default for interactions. The system also introduces stricter snapshot lifecycle management: UI snapshots are now invalidated after desktop mutations (such as clicks), and commands will fail with clear diagnostics if a required snapshot is missing or stale, ensuring that interactions are always performed against fresh, accurate UI state.

Apps/CLI/Sources/PeekabooCLI/Commands/Shared · high confidence

The automation feedback client now explicitly handles coordinate translation between global display points and AppKit screen geometry. This ensures that visual feedback for clicks, typing, scrolling, gestures, and hotkeys is rendered correctly relative to the primary display's origin, resolving previous regressions in coordinate mapping and secure typing contexts.

Core/PeekabooCore/Sources/PeekabooAutomation · high confidence

The PeekabooVisualizer renderer has been completely rewritten to replace the previous click reticle and mouse comet with an animated macOS-style cursor that follows pointer movement with a curved path. The new architecture introduces an AnimationOverlayManager to handle overlay windows with crossfading and replace-keys, preventing stacking of identical feedback types. An OptimizedAnimationQueue manages animation concurrency and batching, while a PerformanceMonitor tracks FPS and memory usage. The visualizer now uses a TargetWindowVisibilityEvaluator to ensure feedback only appears on visible target windows, and includes new views for agent cursor, click pulses, input HUDs, and element detection overlays.

Core/PeekabooVisualizer · high confidence

The Peekaboo workspace has been updated to explicitly reference the renamed Inspector project (previously PeekabooInspector) and includes a new CodeQL build scheme. This scheme configures the build process to compile the CLI tools, the main Mac app, the Playground, and the Inspector app, ensuring that internal CLI target identities are isolated for security analysis.

Apps/Peekaboo.xcworkspace · high confidence

The CLI runtime now enforces stricter policies for screen capture and daemon management to improve reliability and security. It introduces a \BridgeCapabilityPolicy\ that validates host capabilities (such as ScreenCaptureKit process ownership and classic capture fallback) before selecting a remote host, refusing operations if readiness checks fail or required permissions are missing. A new \DaemonStartupGate\ serializes daemon launches using file locking to prevent race conditions, while \ManagedAutoDaemonRegistry\ tracks and prunes stale daemon records. Additionally, \DaemonLaunchPolicy\ now supports migration rollbacks to ensure state consistency if a launch is cancelled, and \BridgeExplicitSocketUnavailableError\ provides clearer diagnostics when a specified bridge socket is unreachable.

Apps/CLI/Sources/PeekabooCLI/Commands/Base/Runtime · high confidence

The menu bar window enumeration logic now validates results from private CGS APIs before allocating memory, preventing potential crashes or undefined behavior when the system returns invalid counts. This change, located in the CGS bridge services, ensures that failures in retrieving window lists are handled gracefully by returning empty results rather than causing allocation errors, leading to more stable detection of status items.

Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/UI/CGS · high confidence

This change introduces new internal services to harden the automation of dialogs and file interactions. AXChildWindowMessagingTimeout ensures raw Accessibility operations on child windows respect explicit deadlines, preventing hangs. AXDescriptorReader refines how element metadata is read, distinguishing between truly incomplete reads and harmless attribute errors to avoid dropping valid UI nodes. AXMutationObservation and AXObservationReadDiagnostics provide precise, process-scoped observation of element changes and detailed diagnostics for failed reads, ensuring that background input and dialog targeting remain reliable even when accessibility attributes are unstable or slow.

Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/UI · high confidence

Add test fixtures for background hotkey probing, exact window capture, and MCP stubbing; Added CLI runtime tests for agent execution limits, bridge status, and capture engine routing; Added automated policy and certification tests; Added integration and unit tests for agent execution and UI logic; Added smoke tests for OverlayManager; Added synthetic DevTools WebSocket fixture for testing; Added test coverage for the Peekaboo automation capture subsystem; Added test helper utilities for UI automation snapshotting; Added test support fixtures and utilities for CLI automation; Added test support for TTY credential supervisor process management; Added test support for subprocess execution in CLI runtime tests; Added tests for AI service provider configuration and model resolution; Added tests for ActionLogger, keyboard context, and semantic witnesses; Added tests for CLI click command parsing and validation; Added tests for CertificationController evidence policies, identity validation, and lifecycle gates; Added tests for MCP Stdio transport; Added tests for MenuService merging, display title resolution, and traversal budgeting; Added tests for Peekaboo Bridge security, protocol, and automation contracts; Added tests for input and tool configuration handling; Added tests for model contract behaviors; Added tests for the PeekabooAgentRuntime module; Added unit tests for Dialog and Dock UI services; Added unit tests for Mac app launch policy, dock icon management, and permission checks; Added unit tests for Mac app services and UI components; Added unit tests for Mac automation target tracking and status bar menu behavior; Added unit tests for conversation session, message, and tool call models; Added unit tests for inspector permissions and overlay management; Added unit tests for the Peekaboo agent service; Expanded CLI automation test coverage; Expanded CLI automation test harness and support infrastructure; Expanded test coverage for CLI agent, capture, and runtime behaviors; Expanded test coverage for MCP tool behaviors and infrastructure; Expanded test coverage for accessibility automation and timeout handling; Expanded test coverage for agent runtime, AI provider parsing, and accessibility traversal budgets; Migrate Peekaboo Mac tests to Swift Testing with credential coordination fixtures; New centralized test fixtures and peers for Bridge protocol testing; New test support infrastructure for deterministic automation testing.

The entire codebase has been upgraded to Swift 6.2, enabling strict concurrency checking, ExistentialAny, and NonisolatedNonsendingByDefault across all Swift Package Manager manifests. This ensures the CLI, Mac app, Inspector, Playground, and core modules are built with modern safety guarantees. Additionally, the CLI now uses the Commander package instead of ArgumentParser, and the Mac app has updated its Sparkle dependency to version 2.10.0.

(dependencies) · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• CAI 55 → 51 (-3.9)
• Rubric changed (rubric-2026.09.11 → rubric-2026.09.18) — scores are not directly comparable.

• Code Health 41 → 41 (+0.2)
• Architecture 98 → 90 (-7.4)
• Maturity 66 → 66 (-0.1)
• Readiness 68 → 54 (-14.0)
• Security 63 → 71 (+8.8)
• Performance 61 (new)

• ActionInputDriver.performScrollbarScroll (cognitive 39) (Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/UI/ActionInputDriver.swift)
• ActionInputDriver.performScrollbarScroll (cyclomatic 25) (Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/UI/ActionInputDriver.swift)
• ActionInputDriver.value (cognitive 16) (Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/UI/ActionInputDriver.swift)
• AgentExecutionResult.executionTrace (cognitive 19) (Core/PeekabooCore/Sources/PeekabooAgentRuntime/Agent/AgentCompatibilityTypes.swift)
• BackgroundInputDriver.performFocusedTextKey (cognitive 22) (Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/UI/BackgroundInputDriver.swift)
• BackgroundInputDriver.performFocusedTextKey (cyclomatic 19) (Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/UI/BackgroundInputDriver.swift)
• Change coupling: ScreenCaptureService+Support.swift ↔ ScreenCaptureServiceProtocol.swift (Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/Capture/ScreenCaptureService+Support.swift)
• ClickService.clickElementByQuery (cognitive 16) (Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/UI/ClickService.swift)
• Dependency hygiene PARTLY measured — npm pinning read, dependency currency not (no pnpm-resolved versions to grade)
• Documentation: no usage examples (README.md)
• Duplicated block (10 lines × 2) (Core/PeekabooCore/Sources/PeekabooAgentRuntime/MCP/Tools/ActionTool.swift)
• Duplicated block (10 lines × 3) (Core/PeekabooCore/Sources/PeekabooAgentRuntime/Browser/BrowserMCPService.swift)
• Duplicated block (11 lines × 2) (Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/System/ApplicationService.swift)
• Duplicated block (11 lines × 3) (Apps/CLI/Sources/PeekabooCLI/Commands/Interaction/PasteCommand+CommanderMetadata.swift)
• Duplicated block (11 lines × 5) (Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/UI/DetachedExactWindowFocusReader.swift)
• Duplicated block (12–18 lines × 4) (Apps/CLI/Sources/PeekabooCLI/Commands/AI/SeeCommand+CommanderMetadata.swift)
• Duplicated block (13 lines × 2) (Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/UI/HotkeyService.swift)
• Duplicated block (13 lines × 2) (Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/UI/UIAutomationService+TypingOperations.swift)
• Duplicated block (13 lines × 3) (Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/UI/DetachedExactWindowFocusReader.swift)
• Duplicated block (14 lines × 2) (Apps/CLI/Sources/PeekabooCLI/Commands/Base/Runtime/RuntimeHostResolver+ScreenCaptureKitOwner.swift)
• …and 116 more

• APIKeyField.body (cognitive 19) (Apps/Mac/Peekaboo/Features/Settings/APIKeyField.swift)
• ActionInputDriver.observeMutation (cognitive 34) (Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/UI/ActionInputDriver.swift)
• ActionInputDriver.observeMutation (cyclomatic 21) (Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/UI/ActionInputDriver.swift)
• ActionInputDriver.performScrollbarActions (cognitive 21) (Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/UI/ActionInputDriver.swift)
• ActionInputDriver.setValue (cyclomatic 23) (Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/UI/ActionInputDriver.swift)
• AgentCommand.makeAgentJSONResponse (cognitive 16) (Apps/CLI/Sources/PeekabooCLI/Commands/AI/AgentCommand.swift)
• AgentExecutionTrace.init (cognitive 19) (Core/PeekabooCore/Sources/PeekabooAgentRuntime/Agent/AgentExecutionTrace.swift)
• AgentToolArguments.stringDictionary (cognitive 18) (Core/PeekabooCore/Sources/PeekabooAgentRuntime/Agent/PeekabooAgentService+Enhancements.swift)
• AllElementsView.filteredElements (cyclomatic 18) (Core/PeekabooUICore/Sources/PeekabooUICore/Components/AllElementsView.swift)
• ApplicationService.bindSelectorResolution (cognitive 17) (Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/System/ApplicationService.swift)
• ApplicationService.listApplications (cyclomatic 17) (Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/System/ApplicationService.swift)
• ApplicationService.readMutationInventory (cyclomatic 18) (Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/System/ApplicationService.swift)
• BoundedBackgroundWindowAX.setGeometryValue (cognitive 20) (Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/UI/WindowManagementService+StateOperations.swift)
• BoundedBackgroundWindowAX.setGeometryValue (cyclomatic 22) (Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/UI/WindowManagementService+StateOperations.swift)
• BrowserMCPProviderConnection.version (cognitive 17) (Core/PeekabooCore/Sources/PeekabooAgentRuntime/Browser/BrowserMCPProviderConnection.swift)
• BuildStalenessChecker.swift.parseGitStatusOutput (cognitive 24) (Apps/CLI/Sources/PeekabooCLI/CLI/Utilities/BuildStalenessChecker.swift)
• CaptureError.errorDescription (cognitive 16) (Core/PeekabooFoundation/Sources/PeekabooFoundation/ErrorTypes.swift)
• CaptureError.errorDescription (cyclomatic 35) (Core/PeekabooFoundation/Sources/PeekabooFoundation/ErrorTypes.swift)
• Change coupling: ErrorHandling.swift ↔ AgentCommand.swift (Apps/CLI/Sources/PeekabooCLI/CLI/Utilities/ErrorHandling.swift)
• Change coupling: ErrorHandling.swift ↔ DragCommand.swift (Apps/CLI/Sources/PeekabooCLI/CLI/Utilities/ErrorHandling.swift)
• …and 359 more

• 127 commits — 49 feature/other, 78 fixes

• Apps/CLI — 37 commits
• Core/PeekabooAutomationKit — 33 commits
• Core/PeekabooCore — 28 commits
• (root) — 21 commits
• .github/workflows — 3 commits
• Apps/Mac — 1 commit
• Apps/Playground — 1 commit
• Core/PeekabooFoundation — 1 commit
• Core/PeekabooUICore — 1 commit
• scripts/release-binaries.sh — 1 commit

• fix: fix(agent): adopt quiet compatible streaming requests (#865)
• fix: fix(agent): distinguish narrative from recorded outcomes (#818)
• fix: fix(agent): keep automatic captures on a proven classic host (#778)
• fix: fix(agent): normalize nested model prefixes without recursion (#752)
• fix: fix(agent): preserve compatible stream tool arguments (#866)
• fix: fix(agent): preserve native scalar argument unions (#834)
• fix: fix(agent): preserve progress on step-limit failure (#814)
• fix: fix(agent): restore streaming callbacks and elide unused events (#863)
• fix: fix(app): align canonical launch paths with selector proofs (#875)
• fix: fix(app): bind companion installer quits to process generations (#874)
• fix: fix(app): propagate ambiguous running-state checks (#795)
• fix: fix(automation): bound desktop lane lock waits at 15s (#794)
• fix: fix(automation): preserve positional click value policy (#857)
• fix: fix(automation): report hidden windows as off screen (#779)
• fix: fix(automation): retain native hotkey target identities (#851)
• fix: fix(automation): settle background AX writes before dependent input (#823)
• fix: fix(automation): settle dependent window geometry writes (#859)
• fix: fix(bridge): attest AXPress coordinate click outcomes (#854)
• fix: fix(bridge): preserve background Select All receipts (#837)
• fix: fix(bridge): preserve deadlines across transport queueing (#827)
• …and 107 more

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

openclaw/Peekaboo was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.