> ## Documentation Index > Fetch the complete documentation index at: https://docs.stateset.com/llms.txt > Use this file to discover all available pages before exploring further. # Automate a back-office tool that has no API > Hand a browser-only task to the Computer Use API end to end — preview, trigger, stream progress, approve the risky step, read the result and screenshots, shut it down — then run the same task through the MCP server. Most warehouses, carriers and 3PLs still have one console that does something important and exposes no API for it. This guide takes one such task — *mark a return as received in a 3PL's returns console and read back the confirmation number* — and runs it as a Computer Use job: an agent works the console in a sandboxed browser and hands you a summary plus the screenshots that prove what it did. | | | | ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | | Base URL | `https://api.computer.stateset.app/api/v1` | | Auth | `X-API-Key: ` on every call — a tenant key issued at `POST /api/v1/keys` | | Scopes used here | `trigger:write` (preview, trigger) · `jobs:read` (status, events, result) · `jobs:write` (approve, cancel) · `admin` (approval policies) | `api.computer.stateset.app` is the production host the platform's MCP client defaults to, and it is still being provisioned — at the time of writing it answers `503`. Everything below is the versioned v1 contract, so you can run it today against a local instance: from the repository's `dashboard/` directory, `docker compose up -d` brings the API up at `http://localhost:8000` (see the repo README). Point `CUA_URL` at `http://localhost:8000/api/v1` and the calls are identical. ```bash theme={null} export CUA_URL="https://api.computer.stateset.app/api/v1" export STATESET_CUA_API_KEY="" ``` ## How a job moves A job is queued, a worker provisions an Xvfb desktop sandbox with a browser, the agent loops screenshot → decision → action until it is done, and the worker stops the sandbox. You never hold a connection to the sandbox — the job id is the handle for everything. Status runs `queued → running → succeeded | failed | cancelled`, with a detour through `awaiting_approval` whenever a policy pauses a step. `GET /jobs/{job_id}/timeline` shows the same lifecycle as annotated phases: `job_created`, `sandbox_ready`, one `tool_call` per step, then `task_completed`, `task_failed` or `task_cancelled`. `GET /auth/whoami` echoes the auth context the key resolves to. A successful response proves the key is active and within rate limit; `scopes` tells you whether the approval step later on will be allowed. An empty list means a full-access key; an `admin` key can mint a narrower one with `POST /keys` (`{"name": "rma-receiver", "scopes": ["trigger:write", "jobs:read", "jobs:write"]}`) — the plaintext comes back once in that response and never again. ```bash theme={null} curl "$CUA_URL/auth/whoami" --header "X-API-Key: $STATESET_CUA_API_KEY" ``` The console's *Mark received* button is the one action in this task that changes a system of record. An approval policy makes the job pause before a matching tool call and wait for a reviewer. Policies are admin-scoped and match on tool-name patterns. ```bash theme={null} curl "$CUA_URL/approval-policies" \ --header "X-API-Key: $STATESET_CUA_ADMIN_KEY" \ --header "Content-Type: application/json" \ --data '{ "name": "returns-console: review computer actions", "tool_patterns": ["computer:*"], "auto_approve_rules": { "allow_safe_subset": true } }' ``` `computer:*` matches every action of the computer tool; without `auto_approve_rules` each one would pause the job. The spec names `allow_safe_subset` and `max_per_hour` as supported rule keys — their semantics are in the platform's operator docs. Skip this step on a first run if you prefer: the job then never enters `awaiting_approval`. The agent cannot tell an instruction from text it reads on a page. Give it a console account with the narrowest access that completes the task, and set `max_cost_usd` on every job. Save the payload as `job.json`. `general` is the free-form built-in template: the `instruction` is the whole brief — URL, record id, and the exact shape of the answer you want back. ```json theme={null} { "agent_type": "general", "instruction": "Open https://console.example-3pl.com/returns and sign in with the saved credentials. Search for RMA-48213. Mark it received with condition 'resellable' and today's date. After the console confirms, read the confirmation number. Reply with a single JSON object: {\"rma\": \"RMA-48213\", \"confirmation_number\": \"...\", \"status\": \"received\"}. If the RMA cannot be found or the console blocks the action, set status to not_found or blocked and leave confirmation_number empty.", "params": {}, "model": "sonnet", "effort": "medium", "max_cost_usd": 2.0, "tags": ["flow:rma-receive", "rma:48213"], "output_schema": { "type": "object", "properties": { "rma": { "type": "string" }, "confirmation_number": { "type": "string" }, "status": { "type": "string", "enum": ["received", "not_found", "blocked"] } }, "required": ["rma", "confirmation_number", "status"] } } ``` `POST /jobs/preview` takes that exact payload, resolves template defaults, the model alias and agent mode, and checks quota — without creating a job or consuming an idempotency key. ```bash theme={null} curl "$CUA_URL/jobs/preview" \ --header "X-API-Key: $STATESET_CUA_API_KEY" \ --header "Content-Type: application/json" \ --data @job.json ``` The response echoes `agent_type_kind` (`builtin` here), `resolved_model` (`claude-sonnet-4-6`), `resolved_agent_mode` (`claude`) and `quota_ok`. An `agent_type_kind` of `unknown` means `/trigger` would reject the template name — preview exists so a typo surfaces here rather than as a wasted run. Same payload, different endpoint. `Idempotency-Key` (scoped to your tenant and `/trigger`) makes a retry return the existing job instead of running the task twice — a replay carries `Idempotency-Replayed: true`. Key it on the record; add a run counter for a deliberate rerun. ```bash theme={null} curl "$CUA_URL/trigger" \ --header "X-API-Key: $STATESET_CUA_API_KEY" \ --header "Content-Type: application/json" \ --header "Idempotency-Key: rma-receive-48213" \ --data @job.json ``` ```json theme={null} { "job_id": "9f1c7e2a-5b64-4c1e-9a0b-2d7f3e8c1a44", "status": "queued" } ``` **Stream events.** `GET /jobs/{job_id}/events` is Server-Sent Events. The first frame is a snapshot, so a late subscriber still gets one event; later frames carry status changes, the sandbox id once provisioned, and running token and cost totals. It closes after a terminal status. ```bash theme={null} curl --no-buffer "$CUA_URL/jobs/$JOB_ID/events" --header "X-API-Key: $STATESET_CUA_API_KEY" ``` ``` data: {"id":"9f1c…","status":"queued","template":"general","input_tokens":null,"output_tokens":null,"cost_usd":null,"summary":null,"error":null} data: {"id":"9f1c…","status":"running","sandbox_id":"sbx_01j8…"} data: {"id":"9f1c…","status":"running","input_tokens":18422,"output_tokens":1310,"cost_usd":0.0748} : keepalive data: {"id":"9f1c…","status":"awaiting_approval"} ``` `: keepalive` comments arrive every 15 s; after the 30-minute lifetime cap you get `event: timeout` — reconnect, and the first frame is a fresh snapshot. `GET /events/stream` is the same shape across every job in the tenant. **Block in a shell.** `POST /jobs/{job_id}/wait` returns as soon as the job is terminal, or after `timeout_seconds` (1–60) with `is_terminal: false` — loop until it is true. ```bash theme={null} curl "$CUA_URL/jobs/$JOB_ID/wait" --header "X-API-Key: $STATESET_CUA_API_KEY" \ --header "Content-Type: application/json" --data '{"timeout_seconds": 60}' ``` **Poll.** `GET /jobs/{job_id}` supports conditional GET — send the previous `ETag` back as `If-None-Match` and an unchanged job returns `304` with no body. `updated_at` refreshes every iteration, so a stalled timestamp is a real signal. When the job hits a gated action it sits in `awaiting_approval` until a reviewer answers. List what is pending (`pending_only=true` is the default); `screenshot_key` is the screen the agent was looking at when it asked. ```bash theme={null} curl "$CUA_URL/approvals" --header "X-API-Key: $STATESET_CUA_API_KEY" ``` ```json theme={null} [{ "id": "5c2e0b7a-1d3f-4e8a-9c21-7b6f0d4a9e10", "job_id": "9f1c…", "tool_name": "computer", "tool_input": { "action": "left_click", "coordinate": [812, 604] }, "reasoning": "Clicking 'Mark received' for RMA-48213 with condition set to resellable.", "screenshot_key": "outputs/9f1c…/screenshot_toolu_01H….png", "status": "pending", "expires_at": "2026-08-31T15:42:10Z", "…": "…" }] ``` Approve and the worker resumes the job; reject and it treats the call as refused, so the agent has to find another way or report that it was blocked. Both take an optional `comment` for the audit trail and both need `jobs:write` (`403` otherwise — check `whoami`). ```bash theme={null} curl "$CUA_URL/approvals/5c2e0b7a-1d3f-4e8a-9c21-7b6f0d4a9e10/approve" \ --header "X-API-Key: $STATESET_CUA_API_KEY" \ --header "Content-Type: application/json" \ --data '{"comment": "Checked the RMA against the inbound manifest — receive it."}' ``` One other intervention exists while a job is live: `POST /jobs/{job_id}/extend-budget` raises `max_cost_usd` on a job about to trip its cap (raise-only; `409` once the job is terminal). `GET /jobs/{job_id}/result` is the one-call answer: status, summary, cost and artifact pointers. ```bash theme={null} curl "$CUA_URL/jobs/$JOB_ID/result" --header "X-API-Key: $STATESET_CUA_API_KEY" ``` ```json theme={null} { "id": "9f1c…", "status": "succeeded", "cost_usd": 0.41, "summary": "{\"rma\": \"RMA-48213\", \"confirmation_number\": \"RC-2026-088131\", \"status\": \"received\"}", "artifacts": [ { "id": "…", "type": "image", "storage_key": "outputs/9f1c…/screenshot_toolu_01H….png", "url": "https://…" }, { "id": "…", "type": "text", "storage_key": "outputs/9f1c…/transcript_2026-08-31T15:14:02+00:00.txt", "url": "https://…" } ] } ``` `output_schema` is enforced caller-side today: parse `summary` and validate it against the schema you sent before writing the confirmation number back to your own returns record. `summary` is the agent's final text, truncated to roughly five segments of 280 characters — a small JSON object fits, a report does not. Artifacts are the evidence: every screenshot is an `image` artifact, every tool output and assistant message a `text` one; `url` is a short-lived presigned download, `storage_key` the stable pointer. Four views over the same run: | Endpoint | What you get | | ------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | | `GET /jobs/{job_id}/artifacts` | The artifact list on its own | | `GET /jobs/{job_id}/replay` | One row per tool call — input, output (truncated to 2000 chars), `screenshot_key`, `duration_ms` | | `GET /jobs/{job_id}/replay/{exec_id}/screenshot` | A presigned URL for one step's screenshot; it expires in 300 s, so fetch immediately or re-call | | `GET /jobs/{job_id}/transcript` | The whole run as Markdown — instruction, summary, error, every tool call with full output. Keep this and the approved step's screenshot for the audit file | A finished job has nothing left to close — the worker stops the sandbox at a terminal status. Shutting down is about jobs still live and the history you keep. ```bash theme={null} # A live job — queued, running or awaiting_approval. The reason lands on job.error. curl "$CUA_URL/jobs/$JOB_ID/cancel" \ --header "X-API-Key: $STATESET_CUA_API_KEY" \ --header "Content-Type: application/json" \ --data '{"reason": "operator received the RMA by hand"}' ``` Cancel sets a signal the in-flight worker sees at its next iteration checkpoint, so token spend stops within one step. `POST /jobs/cancel-by-tag` (`{"tags": ["flow:rma-receive"]}`) does the same for every job carrying the tag; `POST /jobs/cleanup` deletes terminal jobs older than `older_than_days` (minimum 7). Both default to `dry_run: true` and neither touches a live job. ## The same task through the MCP server The hosted `stateset-computer-use` MCP server is a thin client for exactly these endpoints, so an agent in Claude Desktop, Claude Code or your own host runs the task without writing HTTP. Configure it with `STATESET_CUA_API_URL` and `STATESET_CUA_API_KEY` as on the [MCP servers page](/computer-use-mcp): | Step above | Tool | Notes | | -------------- | ------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Preview | `stateset_cua_preview_task` | Same fields as the REST payload — `agent_type`, `instruction`, `params`, `model`, `effort`, `max_cost_usd`, `agent_mode`, `tags`, `output_schema`, `webhook_url` | | Trigger | `stateset_cua_trigger_task` | Adds `idempotency_key`, sent as the `Idempotency-Key` header | | Watch | `stateset_cua_wait_for_result` | Polls `GET /jobs/{id}` every `poll_interval_ms` (default 3000) up to `timeout_ms` (default 300000); returns the `/result` body, or `{"timed_out": true, "last_job": …}` | | Status, result | `stateset_cua_get_job`, `stateset_cua_get_result` | Status with cost fields; summary and artifact pointers | A host prompt that drives it: *"Use stateset-computer-use. Preview first, then trigger with idempotency\_key `rma-receive-48213`, max\_cost\_usd 2, tags `flow:rma-receive` and `rma:48213`, and the instruction in job.json. Wait for the result with timeout\_ms 600000 and give me confirmation\_number from the summary."* The hosted MCP server has no approval tool. If a policy pauses the job, `wait_for_result` keeps polling through `awaiting_approval`; a person approves in the dashboard or with `POST /approvals/{approval_id}/approve` as above. Keep `timeout_ms` generous when a gate is in play. ## When it does not go to plan The cost cap tripped: the worker cancels before the next model call once `max_cost_usd` is crossed, and `error` says so. Read `/replay` to see how far it got; raise the cap next run. The agent hit something it could not resolve — a login wall, a record it could not find. The `status` enum in the schema gives it a legitimate way to say `not_found` or `blocked`; the transcript has the reason. ## What you built | Piece | Endpoint | Purpose | | -------------------- | ----------------------------------------------------------------- | ------------------------------------------------------- | | Scoped access | `GET /auth/whoami`, `POST /keys` | A key that can trigger, read and approve — and no more | | Human gate | `POST /approval-policies` | The record-changing click pauses for a reviewer | | Dry run | `POST /jobs/preview` | Template, model and quota resolved before anything runs | | Safe submission | `POST /trigger` + `Idempotency-Key` | One task, one job, even across retries | | Live view | `GET /jobs/{id}/events`, `POST /jobs/{id}/wait`, `GET /jobs/{id}` | Stream, block, or poll with ETags | | Approval | `GET /approvals`, `POST /approvals/{id}/approve` | The reviewer's yes, with a comment in the audit trail | | Outcome and evidence | `GET /jobs/{id}/result`, `/artifacts`, `/replay`, `/transcript` | The confirmation number, and the screenshots behind it | | Shutdown | `POST /jobs/{id}/cancel`, `/jobs/cancel-by-tag`, `/jobs/cleanup` | Stop live work; keep or purge history deliberately | ## Next steps `POST /jobs/{id}/clone-to-template` turns this run into a named template you trigger by `agent_type`. Register a tenant webhook for `job.completed` and `job.failed` instead of polling — and verify it per the [webhook security guide](/guides/webhook-security). Every endpoint used here, generated from the platform's own OpenAPI.