Computer Use¶

Screenshot, click, type, and browser control via Chrome DevTools Protocol.

Overview¶

Computer Use gives the agent the ability to interact with the desktop and browser:

```ts import { ComputerService } from "quark-agent";

const computer = new ComputerService();

// Take a screenshot const screenshot = await computer.screenshot(); // -> { buffer: Buffer, width: 1920, height: 1080 }

// Click at coordinates await computer.click(100, 200);

// Type text await computer.type("Hello world");

// Launch an app await computer.launchApp("Safari");

// List windows const windows = await computer.listWindows(); ```

Native WebSocket connection to Chrome DevTools Protocol:

```ts // Navigate to a URL await computer.cdpNavigate("https://example.com");

// Execute JavaScript const title = await computer.cdpEvaluate("document.title");

// Take a browser screenshot const screenshot = await computer.browserScreenshot(); ```

Feature	macOS	Linux	Windows
Screenshot	✅ `screencapture`	✅ `scrot`	⚠️ PowerShell
Click	✅ `cliclick`	✅ `xdotool`	⚠️ PowerShell
Type	✅ `cliclick`	✅ `xdotool`	⚠️ PowerShell
Window mgmt	✅	✅	⚠️
CDP browser	✅	✅	✅