Skip to main content

​
Engines and Commerce MCP Servers

The StateSet Computer Use Agent now runs on three interchangeable engines and can drive commerce APIs and the GUI in a single loop.

​
Engines

Select the engine per job — from the dashboard, or with --engine {claude,openai,nsr} on the CLI. All three share the cost and summary path, so switching engines does not change how a job is accounted for or reported.

​
NSR web capability

NSR drives a headless Playwright browser with no X display and no API calls. A bridge implements the two methods NSR calls — execute and take_screenshot — feeding the a11y_first perception real DOM element boxes rather than pixels. On the realistic vertical benchmark, action-task accuracy went from 0% to 80% (12/15). The remaining misses are the three hardest multi-step purchase and transfer flows. NSR also gained:
  • Real <select> dropdown support (select_option).
  • Perceive-and-scroll to reach below-the-fold elements.
  • Expanded money-action steps so multi-step finance flows complete.

​
Commerce MCP stack

One MCPManager aggregates 25 commerce tools across three servers, so the agent uses APIs where they exist and falls back to the GUI where they don’t — API-first, GUI-fallback, in one loop. Each is a FastMCP stdio server with an agent preset. The Dsco server implements a faithful OAuth2 client-credentials lifecycle per the v3 Standard API guide — mint, cache to expires_in, bearer, and refresh on 401.
Multi-server aggregation required an MCPManager handshake fix; the multi-server plus computer-use loop is verified live.

​
OSWorld benchmarking

A real OSWorld score is runnable locally for all three engines, with a Docker/KVM run path.
The runner ships a docker.yaml config and an osworld_os_smoke.jsonl of 8 real os-domain tasks drawn from OSWorld’s evaluation examples — terminal and file operations, settings, and one infeasible task.
The OSWorld Docker provider runs with NET_ADMIN but no tun device, which costs QEMU its port forwarding and leaves the VM unreachable. The runner injects /dev/net/tun and privileged for the OSWorld image — this requires /dev/net/tun on the host. With it the VM boots in about 24 seconds; without it, readiness times out.
NSR has been validated end to end on the real Dockerized OSWorld VM, scored by OSWorld’s own evaluator. There is also a cross-domain benchmark grader for computer-use-preview.