iFurySt/open-codex-computer-use

This system is a cross-platform desktop automation framework that enables AI agents to interact with macOS, Linux, and Windows user interfaces. It provides a core engine for UI control, including click, drag, scroll, and text input operations, alongside configurable accessibility snapshotting and cursor motion modeling. The project includes native runtimes for each operating system, a CLI for probing tools, and extensive tooling for building, testing, and distributing the automation capabilities.

Added a new standalone directory \scripts/cursor-motion-re\ containing Python scripts for reverse-engineering and reconstructing cursor motion behavior from the official \Codex Computer Use.app\ binary. The \reconstruct\_cursor\_motion.py\ CLI tool allows users to inspect recovered Swift types, fields, and constants (such as \VelocityVerletSimulation\ parameters), generate candidate motion paths between start and end points, and perform sensitivity analysis on slider interactions. These scripts operate independently of the main runtime and provide binary-lifted geometric and timing evidence for cursor movement.

scripts/cursor-motion-re · high confidence

A new Python script, \official\_cursor\_motion.py\, has been added to the \scripts/cursor-motion-re/\ directory. This tool parses Mach-O binaries to extract Swift motion type information, specifically targeting cursor-related animation classes such as Bezier, Spring, and VelocityVerlet. It enables the reconstruction of cursor motion assets by analyzing binary sections for motion configuration and field metadata.

python · high confidence

New shell scripts have been added to the plugin's scripts directory to locate and execute the Open Computer Use native runtime and its JavaScript REPL adapter. These scripts search for the application binaries across various build outputs (including development and release variants for macOS, Linux, and Windows) and the system PATH, ensuring the plugin can start correctly regardless of where the runtime was installed or built.

plugins/open-computer-use/scripts · high confidence

The scripts/npm directory now includes build-packages.mjs and publish-packages.mjs, which automate the creation of npm packages for open-computer-use, open-computer-use-mcp, and open-codex-computer-use-mcp across macOS (arm64/x64), Linux (arm64/amd64), and Windows (arm64/amd64). The publishing script supports trusted publishing via GitHub Actions OIDC with provenance, falls back to NODE\_AUTH\_TOKEN, and includes retry logic with exponential backoff to handle transient network issues during release.

scripts/npm · high confidence

This change adds a new Linux-specific Computer Use runtime (Go service and Python AT-SPI backend) that enables UI automation on Linux desktops. Users can now control text extraction volume via a new \text\_limit\ parameter (replacing the previous \show\_full\_text\ switch) and select from multiple click strategies (\auto\, \accessibility\, \app\_post\, \sky\_click\, \global\) to interact with applications.

apps/OpenComputerUseLinux · high confidence

This release adds the OpenComputerUseKit package, providing the core engine for the Open Computer Use agent. It introduces configurable accessibility snapshotting with text and tree limits, a multi-strategy click system (including a new SkyLight background click method and global pointer fallbacks), and native input simulation for clicks, drags, scrolling, and text entry. The kit also includes app discovery via Spotlight, a visual cursor motion model, and a fixture bridge for headless testing.

packages/OpenComputerUseKit · high confidence

Adds a new experimental Windows runtime (OpenComputerUseWindows) that enables AI agents to interact with Windows applications via UI Automation. The service supports configurable click methods (auto, accessibility, app\_post, sky\_click, global) and replaces the previous full-text snapshot toggle with a \text\_limit\ parameter (defaulting to 500 characters) alongside new \max\_tree\_nodes\ and \max\_tree\_depth\ budget controls to manage accessibility tree size. The implementation includes a PowerShell backend for executing UI actions and a Go service layer with corresponding unit tests for tool definitions, argument parsing, and error handling.

apps/OpenComputerUseWindows · high confidence

A new Go-based CLI tool is added to scripts/computer-use-cli to help users probe and interact with the Codex desktop-control (computer-use) plugin. It supports two transport modes—direct (stdio MCP) and app-server (proxying through the signed Codex host)—and provides subcommands to resolve the server target, list available tools, invoke individual tool calls, and execute sequential tool calls via a JSON file. The CLI auto-discovers the bundled computer-use plugin (preferring version 1.0.750 when available) and allows overrides via flags or environment variables for plugin root, version, and server binary.

scripts/computer-use-cli · high confidence

The \scripts/node-repl\ directory now provides a persistent JavaScript REPL and a CLI interface (\ocu js\, \ocu repl\) to interact with the Open Computer Use native runtime. Users can run one-shot JavaScript evaluations or start an interactive session that persists bindings and the \cua\ API across calls, enabling complex desktop automation workflows. The CLI also includes commands to inspect runtime capabilities, install MCP server configurations, and manage the native tool session, with support for timeouts, JSON output, and error attribution to the specific call that produced them.

scripts/node-repl · high confidence

A new macOS application, OpenComputerUseFixture, has been added to serve as a controlled environment for smoke testing computer-use capabilities. The app provides a UI with specific interactive elements—including an increment button, editable text field, key capture view, scrollable area, and drag pad—allowing automated tests to verify keyboard input, mouse clicks, scrolling, and drag-and-drop interactions.

apps/OpenComputerUseFixture · high confidence

The scripts directory now includes a comprehensive set of automation tools for building, packaging, and installing the Open Computer Use runtime. Build scripts (build-apple-iconset.sh, build-cursor-motion-dmg.sh, build-open-computer-use-app.sh, build-open-computer-use-linux.sh, build-open-computer-use-windows.sh) handle macOS icon generation, DMG packaging, and native binary compilation for macOS, Linux, and Windows. Installation scripts (install-claude-mcp.sh, install-codex-mcp.sh, install-codex-plugin.sh, install-dsh-mcp.sh, install-gemini-mcp.sh, install-opencode-mcp.sh) automate the configuration of MCP server entries for various host applications (Claude, Codex, DeepSeek Harness, Gemini, OpenCode). Additional scripts (codex\_dump.py, package-github-release-assets.sh, package-skill.sh, render-open-computer-use-icon.swift) support traffic capture, release asset packaging, skill distribution, and icon rendering.

scripts · high confidence

This change introduces two new standalone Swift applications in the experiments directory to explore cursor motion behaviors independently from the main product code. The \CursorMotion\ demo provides a visual lab with a SwiftUI interface for tuning trajectory parameters (such as start/end handles, arc size, arc flow, and spring dynamics) and visualizing candidate paths, while the \StandaloneCursor\ app serves as a direct Swift port of the previously validated Python reconstruction logic, offering a binary-guided baseline with 20 candidate paths and raw spring timing. Both demos include their own model implementations, UI components, and test coverage to facilitate comparison and further reverse-engineering efforts.

experiments · high confidence

The Open Computer Use app now routes macOS automation through a dedicated background agent process to handle app-scoped permissions, ensuring that CLI commands and MCP interactions are executed with the correct entitlements. This change introduces a new \call\ command for native computer-use operations and adds a visual cursor overlay for click actions, improving the feedback loop during automated interactions. Additionally, the permission onboarding flow has been refined to launch from the doctor command and refreshes automatically after grants are made.

apps/OpenComputerUse · high confidence

Added tests for the DSH MCP installer and Open Computer Use fixture; Introduce OpenComputerUseSmokeSuite for automated MCP tool testing.

This change introduces the foundational dependency manifests for the Open Computer Use project across Swift, Go, and Node.js environments. It adds a Swift Package Manager configuration defining the core library and executable targets, Go module files for Linux and Windows runtime applications, and an npm package.json for repository-level build and release scripts. Additionally, it establishes the Go module definition for the computer-use CLI script, explicitly pinning the modelcontextprotocol/go-sdk to version 1.4.1 along with its indirect dependencies.

(dependencies) · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• CAI 61 → 52 (-9.3)
• Rubric changed (rubric-2026.09.11 → rubric-2026.09.18) — scores are not directly comparable.

• Code Health 54 → 57 (+3.0)
• Architecture 94 → 96 (+2.6)
• Maturity 71 → 71 (-0.2)
• Readiness 55 → 38 (-17.0)
• Security 78 → 84 (+6.2)
• Performance 61 (new)

• Documentation: no architecture or design documentation (docs/references/codex-computer-use-reverse-engineering/README.md)
• Documentation: no architecture or design documentation (docs/references/codex-computer-use-reverse-engineering/assets/README.md)
• Documentation: no installation or build instructions (README.md)
• Duplicated block (25–29 lines × 3) (experiments/CursorMotion/Sources/CursorMotion/SynthesizedCursorGlyphView.swift)
• Duplicated block (5 lines × 2) (packages/OpenComputerUseKit/Sources/OpenComputerUseKit/ComputerUseService.swift)
• Duplicated block (9 lines × 2) (apps/OpenComputerUse/Sources/OpenComputerUse/PermissionOnboardingApp.swift)
• FunctionTooLong: build-packages.renderLauncher (scripts/npm/build-packages.mjs)
• Hotspot: apps/OpenComputerUse/Sources/OpenComputerUse/OpenComputerUseMain.swift (apps/OpenComputerUse/Sources/OpenComputerUse/OpenComputerUseMain.swift)
• Near-duplicate member family (3 members, 25 shared lines) (experiments/CursorMotion/Sources/CursorMotion/SynthesizedCursorGlyphView.swift)
• Off-boarding risk: anonymized user #1
• Workflow token grant is wider than its jobs use

• AccessibilitySnapshot.swift.isPlainGenericTextContainer (cognitive 19) (packages/OpenComputerUseKit/Sources/OpenComputerUseKit/AccessibilitySnapshot.swift)
• AccessibilitySnapshot.swift.preferredDisplayTitle (cognitive 19) (packages/OpenComputerUseKit/Sources/OpenComputerUseKit/AccessibilitySnapshot.swift)
• AccessibilitySnapshot.swift.preferredDisplayTitle (cyclomatic 20) (packages/OpenComputerUseKit/Sources/OpenComputerUseKit/AccessibilitySnapshot.swift)
• AccessibilitySnapshot.swift.shouldElideNode (cognitive 16) (packages/OpenComputerUseKit/Sources/OpenComputerUseKit/AccessibilitySnapshot.swift)
• AccessibilitySnapshot.swift.shouldElideNode (cyclomatic 17) (packages/OpenComputerUseKit/Sources/OpenComputerUseKit/AccessibilitySnapshot.swift)
• AccessibilitySnapshot.swift.summaryImageDescendants (cognitive 17) (packages/OpenComputerUseKit/Sources/OpenComputerUseKit/AccessibilitySnapshot.swift)
• ComputerUseService.swift.isLikelySyntheticSideActionCandidate (cognitive 21) (packages/OpenComputerUseKit/Sources/OpenComputerUseKit/ComputerUseService.swift)
• ComputerUseService.swift.isLikelySyntheticSideActionCandidate (cyclomatic 20) (packages/OpenComputerUseKit/Sources/OpenComputerUseKit/ComputerUseService.swift)
• Coverage not measured — Swift suite
• Documentation: written for insiders (docs/REPO_COLLAB_GUIDE.md)
• Duplicated block (10 lines × 2) (apps/OpenComputerUseLinux/main.go)
• Duplicated block (10 lines × 2) (apps/OpenComputerUseLinux/main.go)
• Duplicated block (10 lines × 2) (scripts/computer-use-cli/main.go)
• Duplicated block (11 lines × 2) (apps/OpenComputerUseLinux/main.go)
• Duplicated block (11 lines × 2) (apps/OpenComputerUseLinux/main.go)
• Duplicated block (11 lines × 2) (experiments/StandaloneCursor/Sources/StandaloneCursor/StandaloneCursorRootView.swift)
• Duplicated block (11 lines × 2) (scripts/computer-use-cli/app_server.go)
• Duplicated block (12 lines × 2) (apps/OpenComputerUse/Sources/OpenComputerUse/PermissionOnboardingApp.swift)
• Duplicated block (12 lines × 2) (apps/OpenComputerUseLinux/main.go)
• Duplicated block (12 lines × 2) (apps/OpenComputerUseLinux/main.go)
• …and 95 more

• 13 commits — 11 feature/other, 2 fixes

• (root) — 4 commits
• (repo) — 1 commit
• docs/exec-plans — 1 commit
• docs/histories — 1 commit
• docs/releases — 1 commit
• packages/OpenComputerUseKit — 1 commit
• plugins/open-computer-use — 1 commit
• scripts/node-repl — 1 commit
• scripts/tests — 1 commit
• skills/open-computer-use — 1 commit

• fix: fix(dsh): skill 安装改为非破坏性，避免覆盖用户已有技能
• fix: fix(dsh): validate compatibility integration
• change: Merge pull request #78 from iFurySt/feat/js-repl-support
• change: docs(dsh): 记录钩子文件改动不会立即生效的运维注意事项
• change: docs: close JS REPL CLI plan (#81)
• change: docs: remove attribution notices and SkyLight reference
• change: feat(cli): add JavaScript and REPL commands (#80)
• change: feat(dsh): 补齐 DeepSeek Harness 宿主集成
• change: feat(plugin): add persistent JavaScript REPL
• change: feat(release): publish direct download artifacts
• change: js repl: attribute errors and output to the call that produced them
• change: js repl: surface thrown errors, skip action read-back on macOS
• change: release: prepare v0.3.6

• Unchanged — 0 containers · 1 contexts · 0 edges

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

iFurySt/open-codex-computer-use was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.