Desktop

Give a running sandbox a screen: programs open windows on it, an agent reads it and clicks on it, and you watch it in a browser.

A sandbox has no screen until one is asked for. Turned on, an Xfce desktop runs in the sandbox's machine, on display :0. It is a program in a machine you already run, and is not charged for by itself.

Turn it on#

curl -X PUT https://agental.io/v1/sandboxes/{id}/desktop \
  -H "Authorization: Bearer $AGENTAL_API_KEY"

{
  "enabled": true,
  "enabledAt": "2026-10-14T09:02:11.480Z",
  "running": true,
  "width": 1280,
  "height": 800,
  "display": ":0",
  "url": "https://3f9c2a17b8d04e6a51c0.agental.site/vnc.html?autoconnect=1&resize=scale&reconnect=1"
}

• The sandbox has to be running. The call answers when the desktop is up.
• The screen is 1280 by 800. For another size send {"width": 1920, "height": 1080}, from 640 by 480 to 2560 by 1600.
• Asked again, the call changes nothing that is already so. Asked with another size, the desktop is started again at that size, and what was open on it is closed.
• A sandbox made before the desktop was added answers 409 desktop_unavailable. Create a new one.

Watch it#

Open url in a browser. It shows the screen, and takes your mouse and keyboard.

• The address is hard to guess and asks for no login: whoever has it sees the screen and can click and type on it. Keep it as you would a password, and do not paste it into a chat or a log.
• It is another address each time the desktop is turned on. To cut off whoever has the link, turn the desktop off and on again. To let someone watch without that, make them a link of their own, as below.
• It answers while the sandbox runs. While the sandbox is stopped it answers 503.
• The page fits the screen to your window. It does not change the size of the screen.
• More than one person can watch at once. They share the one pointer.

Open a window on it#

A command opens a window on the desktop when DISPLAY is :0. A program with a window keeps running, so start it with the call that does not wait:

curl -X POST https://agental.io/v1/sandboxes/{id}/commands \
  -H "Authorization: Bearer $AGENTAL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"command":["sh","-c","DISPLAY=:0 google-chrome https://example.com"]}'

• DISPLAY=:0 playwright-cli open --headed https://example.com does the same with the browser an agent is driving.
• A sandbox has one screen. Two agents on one screen get in each other's way: give each a sandbox of its own.

Let an agent use it#

An agent uses the screen as a person does: it looks, then clicks and types. Each of those is one call.

curl -X GET "https://agental.io/v1/sandboxes/{id}/desktop/screenshot?encoding=base64" \
  -H "Authorization: Bearer $AGENTAL_API_KEY"

• Without encoding=base64 the answer is the picture itself, as image/png. With it, the answer is JSON with mediaType, width, height, region and the picture in base64, which is what a model is handed.
• The picture shows the pointer.
• 1280 by 800 is small enough that a model reading the picture is shown it as it is, and not a smaller copy.

Smaller pictures#

Each picture a model reads costs tokens. Ask for less when less will do:

curl -X GET "https://agental.io/v1/sandboxes/{id}/desktop/screenshot?format=jpeg&quality=60&x=0&y=0&width=640&height=400&scale=0.5&encoding=base64" \
  -H "Authorization: Bearer $AGENTAL_API_KEY"

{
  "mediaType": "image/jpeg",
  "width": 320,
  "height": 200,
  "region": { "x": 0, "y": 0, "width": 640, "height": 400 },
  "base64": "/9j/4AAQSkZJRgABAQ…"
}

• width and height in the answer are the picture's. region is the part of the screen it shows.
• A click is always in pixels of the screen. A point at px, py in the picture is at region.x + px * region.width / width, region.y + py * region.height / height on the screen.
• In the SDKs, screenshot gives the bytes, and screenshotBase64 (screenshot_base64 in Python) the JSON above, with region.
• Each is done in the sandbox's machine, and costs nothing more.

curl -X POST https://agental.io/v1/sandboxes/{id}/desktop/click \
  -H "Authorization: Bearer $AGENTAL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"x":640,"y":400}'

• Each answers 204 when it is done, with nothing in it. Take a new picture to see what came of it.
• A point is in pixels from the top left corner of the screen, and has to be on it.
• Keys are named as xdotool names them: Return, Escape, Tab, BackSpace, Page_Down, F5. Keys held together have + between them, and several with spaces between are pressed in turn: ctrl+a ctrl+c.
• These are the actions a model's computer tool asks for, one call each. The SDKs have them as sandbox.desktop.click, move, drag, scroll, type and key.
• None of them is sent again by the SDKs when it gets no answer: a click that was cut off may have been made.
• The same can be done from inside the sandbox with scrot and xdotool, which are in the image, with DISPLAY=:0.

Paste instead of typing#

type types about a hundred characters a second, a key at a time. Longer text is put on the desktop's clipboard and pasted at once:

curl -X PUT https://agental.io/v1/sandboxes/{id}/desktop/clipboard \
  -H "Authorization: Bearer $AGENTAL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"text":"a long text to paste"}'

• It answers 204, and pastes nothing by itself: click the field, then press ctrl+v with the key call.
• Up to 1 MB of UTF-8. What was on the clipboard before is replaced.
• GET /v1/sandboxes/{id}/desktop/clipboard answers {"text": "…"}: what was copied on the screen, or "" when nothing was.
• It is the clipboard that ctrl+c and ctrl+v use. It goes with the desktop: it is empty after a stop.

Links to watch or to use#

url gives whoever has it the screen, for as long as the desktop is on. A link of its own gives someone less: the screen to watch only, or for a while.

curl -X POST https://agental.io/v1/sandboxes/{id}/desktop/links \
  -H "Authorization: Bearer $AGENTAL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"access":"view","expiresIn":600}'

{
  "id": "dln_4f1c2a9b0d3e7c55",
  "access": "view",
  "url": "https://8c41e07d2b9f6a1355de.agental.site/vnc.html?autoconnect=1&resize=scale&reconnect=1&view_only=1",
  "createdAt": "2026-10-14T09:02:11.480Z",
  "expiresAt": "2026-10-14T09:12:11.480Z",
  "cutOnExpiry": true
}

• Each link is an address of its own, hard to guess, with no login. The desktop's url stays as it is.
• The desktop has to be on and running. A sandbox has up to 20 links that have not expired.
• GET /v1/sandboxes/{id}/desktop/links lists them. DELETE /v1/sandboxes/{id}/desktop/links/{linkId} ends one at once, and cuts what is open through it. Turning the desktop off ends them all.
• A sandbox made before links to watch were added answers 409 desktop_unavailable to access view. control works there. A sandbox made now has both.

After a stop#

The desktop stays turned on for the sandbox. It ends when the sandbox stops, and is started again by itself a moment after the sandbox runs again, with nothing open on it: a window is a process, and no process outlasts a stop.

• running is false while the sandbox is not running, and for a moment after a start. url is null then.
• The address is the same after a start.

Turn it off#

Stops the desktop, which closes what is open on it. Its address leads nowhere from then on.

Errors#

The full machine-readable reference is OpenAPI 3.1. Every page as Markdown, for an agent: llms.txt. Questions: [email protected]