Reference

Tools

Every tool the MCP server offers, its inputs and what it returns, read from the server itself.

The server offers 17 tools. This page is built from the server's own tool list, the one your agent app reads when it connects, so it always matches what runs.

ToolPermissionRead-only
screenshotreadyes
read_treereadyes
zoomreadyes
actcontrolno
waitreadyes
execexecno
exec_statusexecyes
list_screensreadyes
statusreadyes
viewer_linkreadno
clipboardcontrolno
read_fileexecyes
write_fileexecno
appsreadyes
pagecontrolno
run_scriptcontrolno
guidereadyes

Every tool takes screen, a screen id or name, except list_screens. Leave it out when the agent can use only one screen.

screenshot

The screen now, as an image, with its size and screenshot_id. windows: true lists the open windows (id, title, app, place, state). marks: true draws each element id of the active window on the image, for element in act.

Needs read. Read-only.

InputTypeRequiredNotes
windowsboolean
marksboolean

read_tree

The accessibility tree of the active window (or window, or the part in region) as text rows: [e12] role "name" @x,y wxh (states). Act on the ids with act (element, set_value, invoke) and the same screenshot_id; any input makes them stale.

Needs read. Read-only.

InputTypeRequiredNotes
windowstring (like win_[0-9a-f]{1,8}) or "all"A window id, or "all" for every window (use with query to find something). Default: active.
region4 of integer 0 to 9999x0,y0,x1,y1 in the screenshot: keep the elements there.
querystring (at most 200)Keep rows whose role, name or value contains this (a/b), with their parents.
max_elementsinteger 1 to 1000
screenshot_idstring (at most 40)Bind the ids to this screenshot and use its pixels.

zoom

A region of a screenshot at full resolution, for small text. The region is x0,y0,x1,y1 in that screenshot.

Needs read. Read-only.

InputTypeRequiredNotes
screenshot_idstring (at most 40)yes
region4 of integer 0 to 9999yes
max_upscale"fit" or number 1 to 8

act

1 to 20 actions in order (click, move, drag, scroll, type, key, set_value, invoke, select, check, launch, window, focus and the rest), planned on screenshot_id, by point or by element id from read_tree, then the screen after them, taken once it stops changing (up to 2 s), so short waits are rarely needed. To wait for an app or dialog to open, use expect with its window_title (up to 10 s) rather than a long wait. tree adds the tree after. No screenshot or zoom inside.

Needs control. Can change what is on the screen.

InputTypeRequiredNotes
screenshot_idstring (at most 40)yes
actionslist of object, 1 to 20yes
space2 of integer 0 to 9999
screenshotbooleanfalse: no image back, only the result as text with the screenshot_id after it, for the next act. Default true.
treeboolean or objectAlso return the tree after the actions, with fresh ids: the window acted on by element, else the active one. true, or { window, query, max_elements }.
expectobjectNot an action: after them, wait until all hold or timeout_ms; reports satisfied, unsatisfied or unknown. { window_title: "Calc", timeout_ms: 8000 }.
expect.conditionslist of object, 1 to 8All must hold.
expect.conditions[].elementstring (like e[0-9]{1,5})
expect.conditions[].window_titlestring (at most 200)The active window's title contains this.
expect.conditions[].valuestring (at most 503)
expect.conditions[].value_containsstring (at most 503)
expect.conditions[].name_containsstring (at most 203)
expect.conditions[].enabledboolean
expect.conditions[].selectedboolean
expect.conditions[].checkedboolean
expect.conditions[].focusedboolean
expect.conditions[].textstring (at most 200)Text anywhere in the active window's tree.
expect.conditions[].gonebooleanWith text: wait until it is absent.
expect.window_titlestring (at most 200)Short for conditions: [{ window_title }].
expect.textstring (at most 200)Short for conditions: [{ text }].
expect.goneboolean
expect.timeout_msinteger 0 to 10000How long to wait for them; default 2000.

wait

Pause up to 20 seconds; returns an image only if the screen changed since screenshot_id.

Needs read. Read-only.

InputTypeRequiredNotes
screenshot_idstring (at most 40)yes
secondsnumber 0 to 20yes

exec

Run a shell command as the desktop user. Waits up to 25 s: then the exit code and the last 8 KiB of output, else an exec id for exec_status.

Needs exec. Can change what is on the screen.

InputTypeRequiredNotes
commandstring (at most 16384)yes
cwdstring (at most 4096)
timeout_sinteger 0 to 86400

exec_status

Wait up to 25 s for a command started by exec to finish, and read its output from the offsets given.

Needs exec. Read-only.

InputTypeRequiredNotes
exec_idstring (at most 40)yes
stdout_frominteger 0 to 9007199254740991
stderr_frominteger 0 to 9007199254740991

list_screens

The screens you can use: id, name, kind and state.

Needs read. Read-only.

No inputs.

status

A screen's state and health.

Needs read. Read-only.

No inputs.

A view-only live link to show the person what is happening. Open within 60 s, once; it ends after 60 minutes.

Needs read. Not read-only.

No inputs.

clipboard

Read the screen's clipboard text, or set it when text is given. To paste text into the focused field, use act's type with via: paste.

Needs control. Not read-only.

InputTypeRequiredNotes
textstring (at most 100000)

read_file

Read a file on the screen, up to 64 KiB: text as text, anything else as base64. Bigger files: the API, the SDK, or exec.

Needs exec. Read-only.

InputTypeRequiredNotes
pathstring (at most 1024)yesAbsolute, or relative to /home/user.

write_file

Write a file on the screen, up to 64 KiB, from text or from base64 (one of them), creating its folders and replacing what was there.

Needs exec. Can change what is on the screen.

InputTypeRequiredNotes
pathstring (at most 1024)yesAbsolute, or relative to /home/user.
textstring (at most 65536)
base64string (at most 87384, like [A-Za-z0-9+/]*={0,2})

apps

The apps installed on the screen: id, name, what each is for, whether it is open. Start one with act [{ type: "launch", app }].

Needs read. Read-only.

No inputs.

page

A browser tab by ref: op read lists links, buttons and fields as [r3] textbox "Email" = ""; click, fill (value), select (value), check (value "true"/"false"), press (keys), navigate (url), back act on a ref and answer with the page after. prepare says whether the browser can be read.

Needs control. Can change what is on the screen.

InputTypeRequiredNotes
opone of prepare, read, click, fill, select, check, press, navigate, backyes
refstring (like r[0-9]{1,5})
valuestring (at most 10000)
keysstring (like [A-Za-z0-9_+ -]{1,100})A key or +-joined chord: enter, ctrl+s, alt+tab. xdotool, OpenAI and Playwright names work.
urlstring (at most 2000)
querystring (at most 200)read: only rows containing this (a/b).
tabstring (at most 64)
relaunchbooleanprepare: restart the browser with its debugging port.

run_script

Run JavaScript inside the screen that calls screenshot(), tree(q), find(q), act(actions, opts), wait(ms), expect(cond), apps(), launch(id), page(req), log(); for loops and read-then-act in one call. return gives the result. 28 s at most. screenshot: true attaches the screen after. guide topic scripts has examples.

Needs control. Can change what is on the screen.

InputTypeRequiredNotes
codestring (at most 32768)yes
screenshotboolean

guide

How to use these tools well, by topic: loop, elements, apps (and windows), browser, scripts, errors.

Needs read. Read-only.

InputTypeRequiredNotes
topicone of loop, elements, apps, browser, scripts, errorsyes