Kubi Docs
Guides

Drive the browser with Playwright

Attach Playwright or Puppeteer to the Chromium already open on the screen, with its logins.

Screenshots and clicks work on any app. For web pages, an agent can also drive Chromium through its own page model: click a button by its label, fill a field, read text, wait for a request. Chromium on a screen starts with its debugging port open on 127.0.0.1:9222, inside the screen only, so Playwright or Puppeteer run on the screen attach to the browser that is already open, with its tabs and logins.

Attach from a script on the screen

Save this as /home/user/attach.mjs (with write_file over MCP, or the files API), install Playwright once, and run it through exec:

JavascriptCode
import { chromium } from "playwright-core";

const browser = await chromium.connectOverCDP("http://127.0.0.1:9222");
const page = browser.contexts()[0].pages()[0] ?? (await browser.contexts()[0].newPage());
await page.goto("https://example.com");
await page.getByRole("link", { name: "More information" }).click();
console.log(await page.title());
await browser.close(); // closes the connection; Chromium stays open
TerminalCode
cd /home/user && npm install playwright-core && node attach.mjs

playwright-core brings no browser of its own: it uses the screen's Chromium. Puppeteer works the same way with puppeteer.connect({ browserURL: "http://127.0.0.1:9222" }).

Good to know

  • Chromium must be running. Open it from the dock, or start it with chromium & through exec with timeout_s: 0.
  • The person watching the live view sees every step, and screenshots still work alongside the script.
  • The port answers only inside the screen: nothing outside it, your other screens included, can reach it.
  • Anything that runs on the screen can use the port, as anything running as the desktop user can already reach the browser. Keep the screen's keys to the agents you trust.
Run your agent inside a screenErrors