# DOCS QUALITY Source: https://docs.stateset.com/DOCS-QUALITY # Documentation quality review Reviewed September 20, 2026. **Provisional editorial grade: 95/100 (A).** This is a judgment of the revised repository, not a measured success rate or a certification of every service. A 100/100 assessment remains unverified. | Area | Score | Evidence and remaining limit | | ------------------------------------- | ---------: | -------------------------------------------------------------------------------------------------------- | | Contract accuracy | 24/25 | Source-backed schema and authentication corrections; hosted behavior still needs verification. | | Onboarding and examples | 24/25 | Executed embedded programs and tested Sandbox failure/cleanup paths; hosted walkthroughs used mocks. | | Maintenance and regression protection | 20/20 | Required checks fail on findings, generated example validation, generator and maintenance regressions. | | Coverage and navigation | 14/15 | Internal link checks and complete Commerce/ResponseCX endpoint coverage; narrative review was selective. | | Presentation and accessibility | 13/15 | Sixteen browser views passed; no comprehensive keyboard or assistive-technology audit. | | **Total** | **95/100** | **Provisional; remaining work requires evidence, not a higher self-assigned score.** | ## Changes made * Rebuilt the contributor entry point, rate-limit guidance, retry example, and Sandbox walkthrough. * Corrected Sandbox API-key authentication and source-backed request/response contracts in Sandbox, Temporal, EDI, NSR, Sync, and Computer Use overlays. * Expanded generated nested request fields, constraints, alternatives, header parameters, streaming responses, and payload-file examples. Regenerated 619 pages across Commerce, ResponseCX, Sandbox, and Computer Use, plus selected corrected endpoints in other services. * Marked hosted prerequisites and legacy examples explicitly; removed unpublished-package quickstarts and unsupported blanket rate-limit promises. * Corrected host-status wording: a 503 is not proof of provisioning, and a 404 is not proof of no deployment. Preserved the original observation date rather than implying a new probe. * Fixed a browser runtime error caused by a Node-only maintenance script using a `.js` extension. * Added regression checks for generated payloads, retry behavior, Sandbox cleanup, service-scoped coverage, and build failures. Preserved preexisting Console/Desktop documentation changes. ## Verification | Check | Result | | ------------------------------- | --------------------------------------------------------------------------------------- | | `npm run check` | Passed; 32 regression tests, 1,944 MDX pages compiled | | Documentation validator | Zero errors; warnings reduced from 98 to 11 | | Automated correctness audit | Zero findings across 1,944 pages; heuristic coverage only | | Internal anchors and images | Zero broken references detected; external URLs were not fetched | | Published JSON request examples | 242 checked against merged schemas across four services | | `npm run test:examples` | 18 programs from 16 pages passed against embedded engine 1.35.1 | | Commerce endpoint coverage | 442/442, no undocumented or unverified operations against the local spec | | ResponseCX endpoint coverage | 38/38, no undocumented or unverified operations against the local spec | | Browser smoke review | Four pages × two themes × two widths; all HTTP 200, no page errors or document overflow | Browser sampling covered the introduction, rate-limit guide, Commerce create-order reference, and Sandbox quickstart at 320px and 1280px. This checks rendering and runtime behavior, not all interactive states or accessibility conformance. Schema validation does not prove business validity, credential scope, or compatibility with a deployed version. ## Evidence required for 100/100 1. Run authenticated hosted onboarding journeys on confirmed deployments, recording versions, request/response behavior, permission failures, verification, and cleanup. No hosted writes were performed in this review. 2. Resolve or explicitly certify the 11 remaining GET/DELETE request-body advisories with service owners, including the legacy Cloud contracts. Do not silently change an API contract to satisfy a documentation linter. 3. Complete keyboard, focus, screen-reader, contrast, and interactive-component checks across representative page types; inspect the remaining narrative guides and external destinations. Keep these gaps visible. Automated structural scores alone cannot establish documentation correctness or a reader's ability to finish a real integration. # UPSTREAM FINDINGS Source: https://docs.stateset.com/UPSTREAM-FINDINGS # Upstream findings from the docs verification pass (2026-08-31) Everything below was found while grounding docs.stateset.com in each server's source and live behaviour, and is **already worked around in the docs** (spec overlays in `spec/overlays/`, generated pages, or corrected prose). Fixing it upstream lets the overlays shrink to zero and keeps the published contracts honest at the source. One section per repo — each item is paste-ready for an issue. ## stateset-sync-server * **`src/openapi.rs` references 67 component schemas it does not export** (`JobQueuedResponse`, `WebhookResponse`, `PaginatedResponse`, `Order`, `ScheduleInfo`, …): 67 operations' responses dangle, and 28 request bodies with them. The docs type them from the handler structs (`spec/overlays/sync.json`) — the fix is registering the types in `components(schemas(...))`. * `docs/SHOPIFY_NETSUITE_GETTING_STARTED.md` predates `POST /sync/netsuite` and says there is no built-in Shopify→NetSuite job; the code has one. The doc points at ACP instead. ## next-temporal-rs * **`GET /v1/workflows/order-fulfillment/{id}/status` returns `Json`** (`":"`), while `docs/openapi.yaml` declares `WorkflowStatusResponse`. * **`BRANDED_WORKFLOW_PREFIXES` in `events.rs` omits `order-fulfillment-`**, so a brand-scoped key gets 400 on the event stream for its own workflow; the docs tell readers to use a global key as a workaround. * \~137 operations are `GenericObject` in the exported spec, and 33 signal/cancel routes list a `200` they never return (they answer `202`; api-key revoke answers `204`). ## stateset-phone-server (rust-phone-server) * **utoipa declares bare arrays for four list endpoints** (`/callback-tasks`, `/operator-queues`, `/operator-queues/stats`, `/supervisor-actions`) whose handlers wrap in `{ok, tenant, count, …}` envelopes. * The published spec covers 61 tenant paths but not the agent-management plane (`/voice/agents`, `/voice/phone-numbers`, `/voice/web-calls`, `/voice/calls`, `/api-keys`) that `VOICE_ENGINE_API.md` documents and the server serves. ## stateset-sandbox * **The live `/openapi.json` (43 ops) omits six routes the server serves behind auth** (tunnels ×3, inference, files/list, files/download, create-from-template — all verified 401). * **Registered paths vs express routers disagree in the checkout**: OpenAPI registers `/files/write` and `/files/read`; the routers in the current source serve those operations at `POST/GET /sandbox/:id/files`. The live deploy answers both styles — the build and the checkout have drifted. * `POST /api/v1/webhooks` returns `200`; its own spec declares `201`. ## stateset-nsr * `POST /api/v1/model/export` is an always-`501` (`model_handlers.rs`); `ExportModelResponse` is dead code. The docs document the 501 honestly. * The spec's `/v1/decisions` responses do not list the `503` fail-closed path that `errors.rs` implements and the product story depends on. * Several NSR spec parameters are declared `in: path` for what the handlers read as query. ## stateset-computer-use-agent * The approval gate (`ApprovalPolicy`/`ApprovalRequest`, `awaiting_approval`) is fully specified in the API models and routes, but nothing in `worker.py`/`agent/` constructs an `ApprovalRequest` — the pause may not be wired in the deployed worker. * `api.computer.stateset.app` answers 503 on every path (docs mark it "being provisioned"). ## stateset-agents * `spec/sdk-surface.json` in the docs pins `@stateset/cli` 1.28.0; source is 1.28.1 (surface identical). The skill's `stateset heartbeat …` example is accepted as natural language but no heartbeat tool exists in `src/tools`. * The API's own spec `servers` list names `api.stateset.io` / `staging-api.stateset.io`, neither of which serves this API. ## stateset-edi * `stateset-edi/overview.mdx` (docs, fixed) showed a JSON response for outbound 810; the handler returns wire-ready X12 (`application/edi-x12`). Upstream: the utoipa annotation for `GET /v1/usage` claims bare `UsageSummary`; the handler wraps it in `{success, data, period}`. ## response-chat-widget * `package.json` says `response-chat-widget` / the admin docs imply `@stateset/…` naming; the package is unpublished either way. The widget's `docs/ADMIN-API.md` route table is accurate (verified against `src/server/admin`). ## Packages referenced by docs but not on npm (\~16) `@stateset/icommerce-skills` (repo name `icommerce-skills` — align before publishing), `@stateset/cctp-sdk`, `stateset-cctp`, `@stateset/client`, `@stateset/agent-sdk`, `@stateset/acp`, `@stateset/types`, `ss-onboard-agent`, and friends. Every referencing page carries a "not yet published" note; this list is the publish checklist. ## Platform * `status.stateset.com` is a 404 (docs now link the probed [host-status page](https://docs.stateset.com/api-reference/hosts) instead). * `prod-api.stateset.cloud.stateset.app` (another session pointed `api-reference/v1/*` at it) answers 503 — when it goes live, the docs adopt it (see `memory: prod-api-host-candidate`). * `kb.stateset.com` serves 401 on every path including `/openapi.json` — a real service with no public contract; decide whether it is integrator-facing. # Skills Source: https://docs.stateset.com/agent-skills Skill files an AI agent can fetch and follow — one per StateSet system — with the URL convention, what each covers, and how to load them into an agent harness. A **skill** is a short document written for an agent rather than a person: what a system is, how to authenticate, the handful of calls that matter, the encodings that trip agents up, and the guardrails to keep. An agent that has read the skill can use the system without having read the reference. Every skill page here is also published as plain Markdown at the same path with a `.md` extension, which is the form an agent should fetch: ``` https://docs.stateset.com/.md ``` ## The skills | Skill | Fetch | Covers | | --------------------------------------------------------------------------- | -------------------------------- | ------------------------------------------------------------------------------------ | | [iCommerce Engine](/stateset-icommerce-skill) | `/stateset-icommerce-skill.md` | The embedded commerce engine, its CLI, sequencer sync, and the 938-tool MCP surface | | [NSR](/stateset-nsr-skill) | `/stateset-nsr-skill.md` | Verified decisions with proofs; teaching rules; verifying and reporting outcomes | | [Sandbox](/stateset-sandbox-skill) | `/stateset-sandbox-skill.md` | Isolated execution: create, run, read files, tear down | | [Voice](/stateset-voice-skill) | `/stateset-voice-skill.md` | Voice agents, number routing, calls, transcripts, latency, the MCP server | | [Computer Use](/computer-use-skill) | `/computer-use-skill.md` | Preview, trigger, wait for and read the result of a computer-use job | | [Sync Server](/stateset-sync-server-skill) | `/stateset-sync-server-skill.md` | Tenant-scoped order and inventory sync across 180+ integrations, by REST, CLI or MCP | | [iCommerce skills catalogue](/stateset-icommerce/stateset-icommerce-skills) | — | The skills the iCommerce agent itself ships with, and how to add one | ## Loading a skill ```bash Claude Code theme={null} # skills live in ~/.claude/skills//SKILL.md mkdir -p ~/.claude/skills/stateset-nsr curl -sL https://docs.stateset.com/stateset-nsr-skill.md -o ~/.claude/skills/stateset-nsr/SKILL.md ``` ```bash Any agent theme={null} # fetch at run time and put the text in the system prompt or a tool result curl -sL https://docs.stateset.com/stateset-nsr-skill.md ``` Each skill opens with YAML frontmatter — `name`, `description` — so a harness that indexes skills by description can pick the right one from the task alone. ## Skills and MCP servers A skill tells an agent *how to think about* a system; an [MCP server](/mcp-servers) gives it *tools to act on* one. They pair: the skill names which tools to prefer, which to preview before running, and which take an idempotency key. Load the skill, connect the server. ## Writing your own The skills above share a shape worth copying: what the system is in two sentences, how to connect, the two or three calls that cover most work, the encodings that break, the rule to keep. Under a hundred and fifty lines. A skill that tries to be the reference is one the agent will not finish reading. # Agentic Commerce Protocol Checkout Guide Source: https://docs.stateset.com/agentic-commerce/agentic-commerce-checkout-guide Learn how to use the Agentic Commerce Protocol to create, update, and complete checkout flows. # Build the Agentic Commerce Protocol checkout endpoints Learn about the Agentic Commerce Protocol specification. You can use the Agentic Commerce Protocol (ACP) to enable AI agents to manage commerce transactions between buyers and sellers. This specification defines the methods and data structures for creating, updating, and completing checkout flows. You can find examples for REST integrations below. **Where ACP fits** ACP is the transaction layer of StateSet’s agentic commerce offering. For the full conversation‑to‑fulfillment architecture, see [Stateset iCommerce Architecture](/icommerce-architecture). ## Create a Checkout Session You can create a new Checkout Session with buyer details, line items, and shipping information. ### Request Specify the parameters required for your request. | Parameter | Type | Description | | ------------------------ | ----------------- | ------------------------------------------- | | **items** | `array` | Array of items you can purchase. (Required) | | **buyer** | `hash` (optional) | Information about the buyer. | | **fulfillment\_address** | `hash` (optional) | Address where the order will ship. | **Example request:** ```http theme={null} POST /checkouts { "items": [ { "id": "item_123", "quantity": 2 } ], "buyer": { "first_name": "John", "last_name": "Doe", "email": "john.doe@example.com", "phone_number": "+1234567890" }, "fulfillment_address": { "name": "John Doe", "line_one": "123 Main St", "line_two": "Apt 4B", "city": "San Francisco", "state": "CA", "country": "US", "postal_code": "94105" } } ``` ### Response The response returns the current state of the checkout from the seller. | Parameter | Type | Description | | --------------------- | ----------------- | ------------------------------------------------------------- | | **id** | `string` | Unique identifier for the Checkout Session. (Required) | | **buyer** | `hash` (optional) | Information about the buyer. | | **payment\_provider** | `hash` (optional) | Payment provider configuration and supported payment methods. | | **status** | `string` | Current status of the checkout process. (Required) | Possible values: `not_ready_for_payment` | `ready_for_payment` | `completed` | `canceled` | `in_progress` | \| **currency** | `string` | Three-letter ISO currency code, in lowercase. (Required) | \| **line\_items** | `array` | Array of line items in the checkout process. (Required) | \| **fulfillment\_address** | `hash` (optional) | Address where the order will ship. | \| **fulfillment\_options** | `array` | Available shipping and fulfillment options. (Required) | \| **fulfillment\_option\_id** | `string` (optional) | ID of the currently selected fulfillment option. | \| **totals** | `array` | Overview of charges and discounts. (Required) | \| **messages** | `array` | Array of messages or notifications related to the checkout process. (Required) | \| **links** | `array` | Array of links related to the checkout process. (Required) | **Example response:** ```json theme={null} { "id": "checkout_abc123", "buyer": { "first_name": "John", "last_name": "Doe", "email": "john.doe@example.com", "phone_number": "+1234567890" }, "payment_provider": { "provider": "stripe", "supported_payment_methods": ["card"] }, "status": "ready_for_payment", "currency": "usd", "line_items": [ { "id": "item_123", "item": { "id": "item_123", "quantity": 2 }, "base_amount": 2000, "discount": 0, "total": 2000, "subtotal": 2000, "tax": 0 } ], "fulfillment_address": { "name": "John Doe", "line_one": "123 Main St", "line_two": "Apt 4B", "city": "San Francisco", "state": "CA", "country": "US", "postal_code": "94105" }, "fulfillment_options": [ { "type": "shipping", "id": "shipping_fast", "title": "Express Shipping", "subtitle": "2-3 business days", "carrier": "Shipping Co", "subtotal": 150, "tax": 0, "total": 150 } ], "fulfillment_option_id": "shipping_fast", "totals": [ { "type": "subtotal", "display_text": "Subtotal", "amount": 2000 }, { "type": "fulfillment", "display_text": "Shipping", "amount": 150 }, { "type": "tax", "display_text": "Tax", "amount": 100 }, { "type": "total", "display_text": "Total", "amount": 2250 } ], "messages": [], "links": [] } ``` ## Retrieve a Checkout object To retrieve an existing Checkout Session using its ID, make a request to the appropriate API endpoint with the ID included in the request. ### Request Specify the parameters required for your request. | Parameter | Type | Description | | --------- | -------- | ------------------------------------------------------ | | **id** | `string` | Unique identifier for the checkout process. (Required) | **Example request:** ```http theme={null} GET /checkouts/:id ``` ### Response The response returns the current state of the checkout from the seller. | Parameter | Type | Description | | --------------------- | ----------------- | ------------------------------------------------------------- | | **id** | `string` | Unique identifier for the checkout session. (Required) | | **buyer** | `hash` (optional) | Information about the buyer. | | **payment\_provider** | `hash` (optional) | Payment provider configuration and supported payment methods. | | **status** | `string` | Current status of the checkout process. (Required) | Possible values: `not_ready_for_payment` | `ready_for_payment` | `completed` | `canceled` | `in_progress` | \| **currency** | `string` | Three-letter ISO currency code, in lowercase. (Required) | \| **line\_items** | `array` | Array of line items in the checkout process. (Required) | \| **fulfillment\_address** | `hash` (optional) | Address where the order will ship. | \| **fulfillment\_options** | `array` | Available shipping and fulfillment options. (Required) | \| **fulfillment\_option\_id** | `string` (optional) | ID of the currently selected fulfillment option. | \| **totals** | `array` | Overview of charges and discounts. (Required) | \| **messages** | `array` | Array of messages or notifications related to the checkout process. (Required) | \| **links** | `array` | Array of links related to the checkout process. (Required) | **Example response:** ```json theme={null} { "id": "checkout_abc123", "buyer": { "first_name": "John", "last_name": "Doe", "email": "john.doe@example.com", "phone_number": "+1234567890" }, "payment_provider": { "provider": "stripe", "supported_payment_methods": ["card"] }, "status": "ready_for_payment", "currency": "usd", "line_items": [ { "id": "item_123", "item": { "id": "item_123", "quantity": 2 }, "base_amount": 2000, "discount": 0, "total": 2000, "subtotal": 2000, "tax": 0 } ], "fulfillment_address": { "name": "John Doe", "line_one": "123 Main St", "line_two": "Apt 4B", "city": "San Francisco", "state": "CA", "country": "US", "postal_code": "94105" }, "fulfillment_options": [ { "type": "shipping", "id": "shipping_fast", "title": "Express Shipping", "subtitle": "2-3 business days", "carrier": "Shipping Co", "subtotal": 150, "tax": 0, "total": 150 } ], "fulfillment_option_id": "shipping_fast", "totals": [ { "type": "subtotal", "display_text": "Subtotal", "amount": 2000 }, { "type": "fulfillment", "display_text": "Shipping", "amount": 150 }, { "type": "tax", "display_text": "Tax", "amount": 100 }, { "type": "total", "display_text": "Total", "amount": 2250 } ], "messages": [], "links": [] } ``` ## Update a Checkout Session You can update an existing Checkout Session by modifying line items, shipping address, or fulfillment options. ### Request Specify the parameters required for your request. | Parameter | Type | Description | | --------------------------- | ------------------- | ------------------------------------------------------ | | **id** | `string` | Unique identifier for the checkout process. (Required) | | **buyer** | `hash` (optional) | Information about the buyer. | | **items** | `array` (optional) | Updated array of items to be purchased. | | **fulfillment\_address** | `hash` (optional) | Updated fulfillment address. | | **fulfillment\_option\_id** | `string` (optional) | Identifier for the selected fulfillment option. | **Example request:** ```http theme={null} PUT /checkouts/:id { "items": [ { "id": "item_123", "quantity": 3 }, { "id": "item_456", "quantity": 1 } ], "fulfillment_address": { "name": "John Doe", "line_one": "456 Oak Ave", "city": "Los Angeles", "state": "CA", "country": "US", "postal_code": "90210" }, "fulfillment_option_id": "shipping_fast" } ``` ### Response The response returns the current state of the checkout from the seller. | Parameter | Type | Description | | --------------------- | ----------------- | ------------------------------------------------------------- | | **id** | `string` | Unique identifier for the Checkout Session. (Required) | | **buyer** | `hash` (optional) | Information about the buyer. | | **payment\_provider** | `hash` (optional) | Payment provider configuration and supported payment methods. | | **status** | `string` | Current status of the checkout process. (Required) | Possible values: `not_ready_for_payment` | `ready_for_payment` | `completed` | `canceled` | `in_progress` | \| **currency** | `string` | Three-letter ISO currency code, in lowercase. (Required) | \| **line\_items** | `array` | Array of line items in the checkout process. (Required) | \| **fulfillment\_address** | `hash` (optional) | Address where the order will ship. | \| **fulfillment\_options** | `array` | Available shipping and fulfillment options. (Required) | \| **fulfillment\_option\_id** | `string` (optional) | ID of the currently selected fulfillment option. | \| **totals** | `array` | Overview of charges and discounts. (Required) | \| **messages** | `array` | Array of messages or notifications related to the checkout process. (Required) | \| **links** | `array` | Array of links related to the checkout process. (Required) | **Example response:** ```json theme={null} { "id": "checkout_abc123", "buyer": { "first_name": "John", "last_name": "Doe", "email": "john.doe@example.com", "phone_number": "+1234567890" }, "payment_provider": { "provider": "stripe", "supported_payment_methods": ["card"] }, "status": "ready_for_payment", "currency": "usd", "line_items": [ { "id": "item_123", "item": { "id": "item_123", "quantity": 3 }, "base_amount": 3000, "discount": 0, "total": 3000, "subtotal": 3000, "tax": 0 }, { "id": "item_456", "item": { "id": "item_456", "quantity": 1 }, "base_amount": 500, "discount": 0, "total": 500, "subtotal": 500, "tax": 0 } ], "fulfillment_address": { "name": "John Doe", "line_one": "456 Oak Ave", "city": "Los Angeles", "state": "CA", "country": "US", "postal_code": "90210" }, "fulfillment_options": [ { "type": "shipping", "id": "shipping_fast", "title": "Express Shipping", "subtitle": "2-3 business days", "carrier": "Shipping Co", "subtotal": 150, "tax": 0, "total": 150 } ], "fulfillment_option_id": "shipping_fast", "totals": [ { "type": "subtotal", "display_text": "Subtotal", "amount": 3500 }, { "type": "fulfillment", "display_text": "Shipping", "amount": 150 }, { "type": "tax", "display_text": "Tax", "amount": 100 }, { "type": "total", "display_text": "Total", "amount": 3750 } ], "messages": [], "links": [] } ``` ## Complete a Checkout You can complete the checkout process by processing the payment and creating an order. ### Request Specify the parameters required for your request. | Parameter | Type | Description | | ----------------- | ----------------- | ----------------------------------------------------------------- | | **id** | `string` | Unique identifier for the checkout process. (Required) | | **buyer** | `hash` (optional) | Information about the buyer. | | **payment\_data** | `hash` | Payment method details for processing the transaction. (Required) | **Example request:** ```http theme={null} POST /checkouts/:id/complete { "payment_data": { "token": "spt_123", "provider": "stripe", "billing_address": { "name": "John Doe", "line_one": "123 Main St", "line_two": "Apt 4B", "city": "San Francisco", "state": "CA", "country": "US", "postal_code": "94105" } } } ``` ### Response The response returns the current state of the checkout from the seller. | Parameter | Type | Description | | --------------------- | ----------------- | ------------------------------------------------------------- | | **id** | `string` | Unique identifier for the Checkout Session. (Required) | | **buyer** | `hash` (optional) | Information about the buyer. | | **payment\_provider** | `hash` (optional) | Payment provider configuration and supported payment methods. | | **status** | `string` | Current status of the checkout process. (Required) | Possible values: `not_ready_for_payment` | `ready_for_payment` | `completed` | `canceled` | `in_progress` | \| **currency** | `string` | Three-letter ISO currency code, in lowercase. (Required) | \| **line\_items** | `array` | Array of line items in the checkout process. (Required) | \| **fulfillment\_address** | `hash` (optional) | Address where the order will ship. | \| **fulfillment\_options** | `array` | Available shipping and fulfillment options. (Required) | \| **fulfillment\_option\_id** | `string` (optional) | ID of the currently selected fulfillment option. | \| **totals** | `array` | Overview of charges and discounts. (Required) | \| **messages** | `array` | Array of messages or notifications related to the checkout process. (Required) | \| **links** | `array` | Array of links related to the checkout process. (Required) | **Example response:** ```json theme={null} { "id": "checkout_abc123", "buyer": { "first_name": "John", "last_name": "Doe", "email": "john.doe@example.com", "phone_number": "+1234567890" }, "status": "completed", "currency": "usd", "line_items": [ { "id": "item_123", "item": { "id": "item_123", "quantity": 3 }, "base_amount": 3000, "discount": 0, "total": 3000, "subtotal": 3000, "tax": 0 }, { "id": "item_456", "item": { "id": "item_456", "quantity": 1 }, "base_amount": 500, "discount": 0, "total": 500, "subtotal": 500, "tax": 0 } ], "fulfillment_address": { "name": "John Doe", "line_one": "456 Oak Ave", "city": "Los Angeles", "state": "CA", "country": "US", "postal_code": "90210" }, "fulfillment_options": [ { "type": "shipping", "id": "shipping_fast", "title": "Express Shipping", "subtitle": "2-3 business days", "carrier": "Shipping Co", "subtotal": 150, "tax": 0, "total": 150 } ], "fulfillment_option_id": "shipping_fast", "totals": [ { "type": "subtotal", "display_text": "Subtotal", "amount": 3500 }, { "type": "fulfillment", "display_text": "Shipping", "amount": 150 }, { "type": "tax", "display_text": "Tax", "amount": 100 }, { "type": "total", "display_text": "Total", "amount": 3750 } ], "messages": [], "links": [] } ``` ## Cancel a Checkout You can cancel an existing Checkout Session if necessary. ### Request Specify the parameters required for your request. | Parameter | Type | Description | | --------- | -------- | ------------------------------------------------------ | | **id** | `string` | Unique identifier for the checkout process. (Required) | **Example request:** ```http theme={null} POST /checkouts/:id/cancel {} ``` ### Response The response returns the current state of the checkout from the seller. | Parameter | Type | Description | | --------------------- | ----------------- | ------------------------------------------------------------- | | **id** | `string` | Unique identifier for the Checkout Session. (Required) | | **buyer** | `hash` (optional) | Information about the buyer. | | **payment\_provider** | `hash` (optional) | Payment provider configuration and supported payment methods. | | **status** | `string` | Current status of the checkout process. (Required) | Possible values: `not_ready_for_payment` | `ready_for_payment` | `completed` | `canceled` | `in_progress` | \| **currency** | `string` | Three-letter ISO currency code, in lowercase. (Required) | \| **line\_items** | `array` | Array of line items in the checkout process. (Required) | \| **fulfillment\_address** | `hash` (optional) | Address where the order will ship. | \| **fulfillment\_options** | `array` | Available shipping and fulfillment options. (Required) | \| **fulfillment\_option\_id** | `string` (optional) | ID of the currently selected fulfillment option. | \| **totals** | `array` | Overview of charges and discounts. (Required) | \| **messages** | `array` | Array of messages or notifications related to the checkout process. (Required) | \| **links** | `array` | Array of links related to the checkout process. (Required) | **Example response:** ```json theme={null} { "id": "checkout_abc123", "buyer": { "first_name": "John", "last_name": "Doe", "email": "john.doe@example.com", "phone_number": "+1234567890" }, "status": "canceled", "currency": "usd", "line_items": [ { "id": "item_123", "item": { "id": "item_123", "quantity": 3 }, "base_amount": 3000, "discount": 0, "total": 3000, "subtotal": 3000, "tax": 0 }, { "id": "item_456", "item": { "id": "item_456", "quantity": 1 }, "base_amount": 500, "discount": 0, "total": 500, "subtotal": 500, "tax": 0 } ], "fulfillment_address": { "name": "John Doe", "line_one": "456 Oak Ave", "city": "Los Angeles", "state": "CA", "country": "US", "postal_code": "90210" }, "fulfillment_options": [ { "type": "shipping", "id": "shipping_fast", "title": "Express Shipping", "subtitle": "2-3 business days", "carrier": "Shipping Co", "subtotal": 150, "tax": 0, "total": 150 } ], "fulfillment_option_id": "shipping_fast", "totals": [ { "type": "subtotal", "display_text": "Subtotal", "amount": 3500 }, { "type": "fulfillment", "display_text": "Shipping", "amount": 150 }, { "type": "tax", "display_text": "Tax", "amount": 100 }, { "type": "total", "display_text": "Total", "amount": 3750 } ], "messages": [ { "type": "info", "content_type": "plain", "content": "Checkout cancelled: Customer changed their mind" } ], "links": [] } ``` ## Data structures This section outlines the data structures involved in the checkout process. ### Buyer The buyer is an individual who initiates the purchase. | Parameter | Type | Description | | ----------------- | ------------------- | --------------------------------------- | | **first\_name** | `string` | The first name of the buyer. (Required) | | **last\_name** | `string` | The last name of the buyer. (Required) | | **email** | `string` | The email of the buyer. **Required** | | **phone\_number** | `string` (optional) | The phone number of the buyer. | ### Item The Item is a product or service that the buyer requests to purchase, along with its quantity. | Parameter | Type | Description | | ------------ | --------- | ---------------------------------------------------------------- | | **id** | `string` | Unique identifier for the item. (Required) | | **quantity** | `integer` | The requested quantity of the item for this checkout. (Required) | ### LineItem The LineItem includes provides about the item added to the checkout, including the amount. | Parameter | Type | Description | | ---------------- | --------- | ------------------------------------------------ | | **id** | `string` | Unique identifier for the line item. (Required) | | **item** | `hash` | The item details. (Required) | | **base\_amount** | `integer` | The base amount of the line item. (Required) | | **discount** | `integer` | The discount amount of the line item. (Required) | | **total** | `integer` | The total amount of the line item. (Required) | | **subtotal** | `integer` | The subtotal amount of the line item. (Required) | | **tax** | `integer` | The tax amount of the line item. (Required) | ### Address The Address provides the buyer’s shipping or billing address. | Parameter | Type | Description | | ---------------- | ------------------- | ------------------------------------------------------------------ | | **name** | `string` | Name of the person to whom the items are fulfilled. (Required) | | **line\_one** | `string` | Address line 1 (e.g., street, PO Box, or company name). (Required) | | **line\_two** | `string` (optional) | Address line 2 (e.g., apartment, suite, unit, or building). | | **city** | `string` | City, district, suburb, town, or village. (Required) | | **state** | `string` | State, county, province, or region. (Required) | | **country** | `string` | Two-letter country code (ISO 3166-1 alpha-2). (Required) | | **postal\_code** | `string` | ZIP or postal code. (Required) | ### PaymentData The PaymentData provides the buyer’s payment details, including the tokenized value and the payment provider. | Parameter | Type | Description | | -------------------- | ----------------- | ------------------------------------------------------------- | | **token** | `string` | The secure reference to a payment credential. (Required) | | **provider** | `string` | The payment provider that the payment data is for. (Required) | | **billing\_address** | `hash` (optional) | Billing address for the payment method. | ### Total The Total provides a summary of the overall total. | Parameter | Type | Description | | --------- | ------ | ----------------------------- | | **type** | `enum` | The type of total. (Required) | Possible values: `items_base_amount` | `items_discount` | `subtotal` | `discount` | `fulfillment` | `tax` | `fee` | `total` | \| **display\_text** | `string` | The display text for the total. (Required) | \| **amount** | `integer` | The amount of the total. (Required) | ### FulfillmentOption Fulfillment options are either shipping or digital. See [ShippingFulfillmentOption](https://docs.stripe.com/agentic-commerce/protocol/specification.md#shipping-fulfillment-option) and [DigitalFulfillmentOption](https://docs.stripe.com/agentic-commerce/protocol/specification.md#digital-fulfillment-option) for specific implementations. ### ShippingFulfillmentOption The ShippingFulfillmentOption defines the parameters for shipping fulfillment options, including the carrier information and delivery times. | Parameter | Type | Description | | --------- | ------ | ------------------------------------------ | | **type** | `enum` | The type of fulfillment option. (Required) | Possible values: `shipping` | \| **id** | `string` | Unique identifier for the shipping fulfillment option. (Required) | \| **title** | `string` | The title of the shipping fulfillment option. (Required) | \| **subtitle** | `string` (optional) | The subtitle of the shipping fulfillment option. | \| **carrier** | `string` (optional) | The carrier of the shipping fulfillment option. | \| **earliest\_delivery\_time** | `string` (optional) | The earliest delivery time of the shipping fulfillment option (ISO 8601 format). | \| **latest\_delivery\_time** | `string` (optional) | The latest delivery time of the shipping fulfillment option (ISO 8601 format). | \| **subtotal** | `integer` | The subtotal of the shipping fulfillment option. (Required) | \| **tax** | `integer` | The tax of the shipping fulfillment option. (Required) | \| **total** | `integer` | The total of the shipping fulfillment option. (Required) | ### DigitalFulfillmentOption The DigitalFulfillmentOption defines the parameters for digital fulfillment options, including the title and pricing information. | Parameter | Type | Description | | --------- | ------ | ------------------------------------------ | | **type** | `enum` | The type of fulfillment option. (Required) | Possible values: `digital` | \| **id** | `string` | Unique identifier for the digital fulfillment option. (Required) | \| **title** | `string` | The title of the digital fulfillment option. (Required) | \| **subtitle** | `string` (optional) | The subtitle of the digital fulfillment option. | \| **subtotal** | `integer` | The subtotal of the digital fulfillment option. (Required) | \| **tax** | `integer` | The tax of the digital fulfillment option. (Required) | \| **total** | `integer` | The total of the digital fulfillment option. (Required) | ### PaymentProvider The PaymentProvider defines the seller’s supported payment provider and available methods. | Parameter | Type | Description | | ------------ | -------- | ----------------------------------------- | | **provider** | `string` | The seller’s payment provider. (Required) | Possible values: `stripe` | \| **supported\_payment\_methods** | `array` | The payment methods allowed by the seller. (Required) Possible values: `card` | ### Message Messages are either informational or error messages. #### InfoMessage The InfoMessage represents informational messages, detailing the type and content. | Parameter | Type | Description | | --------- | ------ | --------------------------------------- | | **type** | `enum` | String value representing message type. | Possible values: `info` | \| **param** | `string` (optional) | RFC 9535 JSONPath to the component of the Checkout Session that the message references. | \| **content\_type** | `enum` (optional) | The type of content of the message. Possible values: `plain` | `markdown` | \| **content** | `string` | The content of the message. | #### ErrorMessage The ErrorMessage represents error messages, detailing the type and code. | Parameter | Type | Description | | --------- | ------ | --------------------------------------- | | **type** | `enum` | String value representing message type. | Possible values: `error` | \| **code** | `enum` | The code of the error. Possible values: `missing` | `invalid` | `out_of_stock` | `payment_declined` | `requires_sign_in` | `requires_3ds` | \| **param** | `string` (optional) | RFC 9535 JSONPath to the component of the Checkout Session that the message references. | \| **content\_type** | `enum` (optional) | The type of content of the message. Possible values: `plain` | `markdown` | \| **content** | `string` | The content of the message. | ### Error The Error defines the parameters related to errors occurring during the checkout process. | Parameter | Type | Description | | --------- | ------ | ----------------------------- | | **type** | `enum` | The type of error. (Required) | Possible values: `invalid_request` | `request_not_idempotent` | `processing_error` | `service_unavailable` | \| **code** | `string` | The implementation-defined error code. (Required) | \| **message** | `string` | The message of the error. (Required) | \| **param** | `string` (optional) | RFC 9535 JSONPath to the component of the Checkout Session that the message references. | ### Link The Link defines the parameters for links related to policies and agreements. | Parameter | Type | Description | | --------- | ------ | ------------------------------------------------------ | | **type** | `enum` | String value representing the type of link. (Required) | Possible values: `terms_of_use` | `privacy_policy` | `seller_shop_policies` | \| **url** | `string` | The URL of the link. (Required) | ### Order The Order provides the result of the checkout process and offers details to the buyer for order lookup. | Parameter | Type | Description | | ------------------------- | -------- | ---------------------------------------------------------------------------- | | **id** | `string` | Unique identifier for the order. (Required) | | **checkout\_session\_id** | `string` | Reference to the Checkout Session that the order originated from. (Required) | | **permalink\_url** | `string` | The permalink URL for the order. (Required) | ### Event The Event defines the parameters for events related to order creation and updates. | Parameter | Type | Description | | --------- | ------ | ----------------------------- | | **type** | `enum` | The type of event. (Required) | Possible values: `order_created` | `order_updated` | \| **data** | `hash` | Event data containing order information. (Required) | ### OrderEventData The OrderEventData includes data related to order events. | Parameter | Type | Description | | ------------------------- | -------- | --------------------------------------------------------------------------------------------------------------------------- | | **type** | `string` | The string value represents the type of event data. For order data, use the value `order`. (Required) | | **checkout\_session\_id** | `string` | ID that identifies the Checkout Session that created this order. (Required) | | **permalink\_url** | `string` | The URL points to the order. Customers can visit this URL and provide their email address to view order details. (Required) | | **status** | `enum` | String representing the latest status of the order. (Required) | Possible values: `created` | `manual_review` | `confirmed` | `canceled` | `shipped` | `fulfilled` | \| **refunds** | `array` | List of refunds that have been issued for the order. (Required) | ### Refund The Refund defines the parameters for managing refunds associated with completed orders. | Parameter | Type | Description | | --------- | ------ | ------------------------------ | | **type** | `enum` | The type of refund. (Required) | Possible values: `store_credit` | `original_payment` | \| **amount** | `integer` | The amount of the refund. (Required) | ## Next steps The protocol behind these calls. Where a completed checkout's order is sent. The conversation-to-fulfilment flow this sits inside. Serving these endpoints yourself. # Delete a delete agent agents agent ID Source: https://docs.stateset.com/api-reference/agents/agents/agents-by-delete DELETE http://localhost:8000/agents/{agent_id} Delete an agent and all associated conversations. Delete an agent and all associated conversations. ### Path parameters ### Response No response body. Agent deleted successfully. ### Status codes | Code | Meaning | | ----- | -------------------------- | | `204` | Agent deleted successfully | | `401` | Authentication required | | `404` | Agent not found | | `422` | Validation Error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request DELETE \ --url 'http://localhost:8000/agents/{agent_id}' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` # Get Agent Details Source: https://docs.stateset.com/api-reference/agents/agents/agents-by-get GET http://localhost:8000/agents/{agent_id} Get detailed information about a specific agent. Get detailed information about a specific agent. ### Path parameters ### Response `AgentDetailResponse` Agent identifier Model name Creation timestamp Number of conversations Total tokens used Agent configuration Agent status ### Status codes | Code | Meaning | | ----- | ----------------------- | | `200` | Agent details | | `401` | Authentication required | | `404` | Agent not found | | `422` | Validation Error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request GET \ --url 'http://localhost:8000/agents/{agent_id}' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` ```json 200 theme={null} { "agent_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "model_name": "string", "created_at": "2026-08-31T14:22:05Z", "conversation_count": 0, "total_tokens_used": 0, "config": {}, "status": "active" } ``` # Create Agent Source: https://docs.stateset.com/api-reference/agents/agents/agents-create POST http://localhost:8000/agents Create a new AI agent with the specified configuration. Create a new AI agent with the specified configuration. ### Request body `AgentConfigRequest` Name of the model to use Maximum tokens to generate Sampling temperature Top-p sampling parameter Top-k sampling parameter System prompt for the agent Whether to use chat template Enable long-term planning Planning configuration overrides ### Response `AgentCreatedResponse` Unique identifier for the created agent Creation timestamp Agent configuration Name of the model to use Maximum tokens to generate Sampling temperature Top-p sampling parameter Top-k sampling parameter System prompt for the agent Whether to use chat template Enable long-term planning Planning configuration overrides Status message ### Status codes | Code | Meaning | | ----- | -------------------------- | | `200` | Successful response | | `201` | Agent created successfully | | `400` | Invalid configuration | | `401` | Authentication required | | `422` | Validation Error | | `429` | Rate limit exceeded | | `500` | Internal server error | ```bash cURL theme={null} curl --request POST \ --url 'http://localhost:8000/agents' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" \ --header 'Content-Type: application/json' \ --data '{ "enable_planning": true, "max_new_tokens": 256, "model_name": "gpt2", "planning_config": { "max_steps": 4 }, "system_prompt": "You are a helpful AI assistant.", "temperature": 0.7, "top_p": 0.9 }' ``` ```json 200 theme={null} { "agent_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "created_at": "2026-08-31T14:22:05Z", "config": { "enable_planning": true, "max_new_tokens": 256, "model_name": "gpt2", "planning_config": { "max_steps": 4 }, "system_prompt": "You are a helpful AI assistant.", "temperature": 0.7, "top_p": 0.9 }, "message": "Agent created successfully" } ``` # List Agents Source: https://docs.stateset.com/api-reference/agents/agents/agents-list GET http://localhost:8000/agents Get a paginated list of all agents. Get a paginated list of all agents. ### Query parameters Page number Items per page Filter by status ### Response `AgentListResponse-Input` Request tracking ID Response timestamp Total number of items Current page number Items per page Whether there are more pages Whether there are previous pages List of agents Agent identifier Model name Creation timestamp Number of conversations Total tokens used Agent configuration Agent status ### Status codes | Code | Meaning | | ----- | ----------------------- | | `200` | List of agents | | `401` | Authentication required | | `422` | Validation Error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request GET \ --url 'http://localhost:8000/agents' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` ```json 200 theme={null} { "request_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "timestamp": "2026-08-31T14:22:05Z", "total": 102, "page": 1, "page_size": 20, "has_next": false, "has_prev": false, "items": [ { "agent_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "model_name": "string", "created_at": "2026-08-31T14:22:05Z", "conversation_count": 0, "total_tokens_used": 0, "config": {}, "status": "active" } ] } ``` # Delete a delete conversation conversations conversation ID Source: https://docs.stateset.com/api-reference/agents/conversations/conversations-by-delete DELETE http://localhost:8000/conversations/{conversation_id} Delete a specific conversation. Delete a specific conversation. ### Path parameters ### Response No response body. Conversation deleted successfully. ### Status codes | Code | Meaning | | ----- | --------------------------------- | | `204` | Conversation deleted successfully | | `401` | Authentication required | | `404` | Conversation not found | | `422` | Validation Error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request DELETE \ --url 'http://localhost:8000/conversations/{conversation_id}' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` # Get a get conversation conversations conversation ID Source: https://docs.stateset.com/api-reference/agents/conversations/conversations-by-get GET http://localhost:8000/conversations/{conversation_id} Get details of a specific conversation. Get details of a specific conversation. ### Path parameters ### Response `object` ### Status codes | Code | Meaning | | ----- | ----------------------- | | `200` | Successful response | | `401` | Authentication required | | `404` | Conversation not found | | `422` | Validation Error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request GET \ --url 'http://localhost:8000/conversations/{conversation_id}' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` ```json 200 theme={null} { "conversation_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "agent_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "messages": [ {} ], "created_at": "2026-08-31T14:22:05Z", "last_message_at": "2026-08-31T14:22:05Z", "total_tokens": 1, "metadata": {} } ``` # Chat with Agent Source: https://docs.stateset.com/api-reference/agents/conversations/conversations-create POST http://localhost:8000/conversations Send a message to an agent and get a response. Send a message to an agent and get a response. ### Request body `ConversationRequest` List of conversation messages (append-only when conversation\_id is set) Conversation identifier User identifier Maximum tokens in response Response temperature Whether to stream the response Additional context ### Response `ConversationResponse` Agent's response Conversation identifier Number of tokens used Processing time in seconds Additional metadata ### Status codes | Code | Meaning | | ----- | ----------------------- | | `200` | Successful response | | `400` | Invalid input | | `401` | Authentication required | | `404` | Agent not found | | `422` | Validation Error | | `429` | Rate limit exceeded | | `500` | Internal server error | ```bash cURL theme={null} curl --request POST \ --url 'http://localhost:8000/conversations' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" \ --header 'Content-Type: application/json' \ --data '{ "context": { "goal": "Plan a 4-day trip to Kyoto", "plan_goal": "Plan a 4-day trip to Osaka", "plan_update": { "action": "advance" } }, "conversation_id": "demo-trip", "max_tokens": 256, "messages": [ { "content": "You are a helpful assistant.", "role": "system" }, { "content": "Hello! How can you help me?", "role": "user" } ], "stream": false, "temperature": 0.7 }' ``` ```json 200 theme={null} { "response": "string", "conversation_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "tokens_used": 1, "processing_time": 1.5, "metadata": {} } ``` # List Conversations Source: https://docs.stateset.com/api-reference/agents/conversations/conversations-list GET http://localhost:8000/conversations Get a paginated list of conversations. Get a paginated list of conversations. ### Query parameters Page number Items per page Filter by agent ID ### Response `ConversationListResponse-Input` Request tracking ID Response timestamp Total number of items Current page number Items per page Whether there are more pages Whether there are previous pages List of conversations Conversation identifier Associated agent ID Number of messages Creation timestamp Last message timestamp Total tokens used ### Status codes | Code | Meaning | | ----- | ----------------------- | | `200` | List of conversations | | `401` | Authentication required | | `422` | Validation Error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request GET \ --url 'http://localhost:8000/conversations' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` ```json 200 theme={null} { "request_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "timestamp": "2026-08-31T14:22:05Z", "total": 102, "page": 1, "page_size": 20, "has_next": false, "has_prev": false, "items": [ { "conversation_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "agent_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "message_count": 0, "created_at": "2026-08-31T14:22:05Z", "last_message_at": "2026-08-31T14:22:05Z", "total_tokens": 0 } ] } ``` # API Changelog Source: https://docs.stateset.com/api-reference/agents/documentation/api-docs-changelog-list GET http://localhost:8000/api/docs/changelog Get the API changelog with version history. Get the API changelog with version history. ### Response `object` ### Status codes | Code | Meaning | | ----- | ----------------------- | | `200` | Successful response | | `401` | Authentication required | | `422` | Validation error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request GET \ --url 'http://localhost:8000/api/docs/changelog' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` ```json 200 theme={null} { "versions": [ { "version": "2026-08-01", "date": "2026-08-31", "changes": [] } ] } ``` # Error Codes Reference Source: https://docs.stateset.com/api-reference/agents/documentation/api-docs-errors-list GET http://localhost:8000/api/docs/errors Get a reference of all possible error codes. Get a reference of all possible error codes. ### Response `object` Keyed by code — `BAD_REQUEST`, `UNAUTHORIZED`, `FORBIDDEN`, `NOT_FOUND`, … ### Status codes | Code | Meaning | | ----- | ----------------------- | | `200` | Successful response | | `401` | Authentication required | | `422` | Validation error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request GET \ --url 'http://localhost:8000/api/docs/errors' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` ```json 200 theme={null} { "error_codes": {} } ``` # Rate Limits Source: https://docs.stateset.com/api-reference/agents/documentation/api-docs-rate-limits-list GET http://localhost:8000/api/docs/rate-limits Get rate limit information. Get rate limit information. ### Response `object` The `X-RateLimit-*` headers and what each means ### Status codes | Code | Meaning | | ----- | ----------------------- | | `200` | Successful response | | `401` | Authentication required | | `422` | Validation error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request GET \ --url 'http://localhost:8000/api/docs/rate-limits' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` ```json 200 theme={null} { "rate_limits": { "default": { "requests_per_minute": 1, "description": "Two-person tent, green — replacement for damaged pole set." }, "unauthenticated": { "requests_per_minute": 1, "description": "Two-person tent, green — replacement for damaged pole set." }, "training": { "concurrent_jobs": 1, "description": "Two-person tent, green — replacement for damaged pole set." } }, "headers": {} } ``` # Liveness Check Source: https://docs.stateset.com/api-reference/agents/health/live-list GET http://localhost:8000/live Check if the API is alive. Check if the API is alive. ### Response `object` ### Status codes | Code | Meaning | | ----- | ----------------------- | | `200` | Successful response | | `401` | Authentication required | | `422` | Validation error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request GET \ --url 'http://localhost:8000/live' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` ```json 200 theme={null} { "status": "alive", "timestamp": "2026-08-31T14:22:05Z" } ``` # Readiness Check Source: https://docs.stateset.com/api-reference/agents/health/ready-list GET http://localhost:8000/ready Check if the API is ready to receive traffic. Check if the API is ready to receive traffic. ### Response `object` ### Status codes | Code | Meaning | | ----- | ----------------------- | | `200` | Successful response | | `401` | Authentication required | | `422` | Validation error | | `429` | Rate limit exceeded | | `503` | Not ready | ```bash cURL theme={null} curl --request GET \ --url 'http://localhost:8000/ready' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` ```json 200 theme={null} { "status": "ready" } ``` # API Root Source: https://docs.stateset.com/api-reference/agents/health/root-list GET http://localhost:8000/ Welcome endpoint with API information and available endpoints. Welcome endpoint with API information and available endpoints. ### Response `object` Name → path for every top-level surface ### Status codes | Code | Meaning | | ----- | ----------------------- | | `200` | Successful response | | `401` | Authentication required | | `422` | Validation error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request GET \ --url 'http://localhost:8000/' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` ```json 200 theme={null} { "message": "Two-person tent, green — replacement for damaged pole set.", "version": "2026-08-01", "documentation": "string", "openapi": "string", "health": "string", "endpoints": {} } ``` # Agents API history Source: https://docs.stateset.com/api-reference/agents/history Operations added or removed in the Agents API surface, derived from the spec's own revision history — the API's changelog, distinct from the docs changelog. Derived from the revision history of `spec/agents.json` (`node scripts/api-history.mjs`). An operation listed as **removed** was retired from the published surface — a caller still using it must migrate, not merely re-read the docs. Added operations link to their reference pages. ## 2026-08-31 Initial publication — **35 operations**, generated from the service's own OpenAPI. Highlights: * [`POST /agents`](/api-reference/agents/agents/agents-create) * [`GET /agents`](/api-reference/agents/agents/agents-list) * [`GET /agents/{agent_id}`](/api-reference/agents/agents/agents-by-get) * [`DELETE /agents/{agent_id}`](/api-reference/agents/agents/agents-by-delete) * [`POST /conversations`](/api-reference/agents/conversations/conversations-create) * [`GET /conversations`](/api-reference/agents/conversations/conversations-list) # Create Message Source: https://docs.stateset.com/api-reference/agents/messages/v1-messages-create POST http://localhost:8000/v1/messages Anthropic-compatible Messages endpoint. Supports OpenAI-style messages as input and can return OpenAI-style responses when response_format='openai' is provided. Anthropic-compatible Messages endpoint. Supports OpenAI-style messages as input and can return OpenAI-style responses when `response_format='openai'` is provided. ### Request body `MessagesRequest` Model identifier Conversation messages System prompt (string or content blocks) Maximum tokens to generate Sampling temperature Top-p Top-k Stop sequences (Anthropic style) Stream response tokens Tool definitions (Anthropic or OpenAI format) Tool choice configuration Arbitrary metadata Response format preference ### Response `MessagesResponse` Message identifier Response type Assistant role Model identifier Response content blocks Stop reason Stop sequence Token usage Prompt tokens used Completion tokens used ### Status codes | Code | Meaning | | ----- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | | `200` | Anthropic-shaped reply. With `response_format: 'openai'` the body is an OpenAI chat completion instead; with `stream: true` it is a `text/event-stream`. | | `401` | Authentication required | | `422` | Validation Error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request POST \ --url 'http://localhost:8000/v1/messages' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" \ --header 'Content-Type: application/json' \ --data '{ "model": "string", "messages": [ { "role": "system", "content": "Two-person tent, green — replacement for damaged pole set.", "name": "Two-Person Tent", "tool_call_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "tool_calls": [] } ], "system": "string", "max_tokens": 1, "temperature": 1.5, "top_p": 1.5, "top_k": 1, "stop_sequences": [ "string" ], "stream": false, "tools": [ {} ], "tool_choice": "string", "metadata": {}, "response_format": "anthropic" }' ``` ```json 200 theme={null} { "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "type": "message", "role": "assistant", "model": "string", "content": [ {} ], "stop_reason": "string", "stop_sequence": "string", "usage": { "input_tokens": 1, "output_tokens": 1 } } ``` # Circuit Breaker Status Source: https://docs.stateset.com/api-reference/agents/metrics/circuits-list GET http://localhost:8000/circuits Get the status of all circuit breakers. Get the status of all circuit breakers. ### Response `object` Per-circuit breaker state and counters ### Status codes | Code | Meaning | | ----- | ----------------------- | | `200` | Successful response | | `401` | Authentication required | | `422` | Validation error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request GET \ --url 'http://localhost:8000/circuits' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` ```json 200 theme={null} { "circuits": {}, "timestamp": "2026-08-31T14:22:05Z" } ``` # Health Check Source: https://docs.stateset.com/api-reference/agents/observability/health-list GET http://localhost:8000/health Check the health status of the API service and its components. Check the health status of the API service and its components. ### Response `RouterHealthResponse` Overall service status Response timestamp API version Service uptime in seconds Component statuses keyed by component name ### Status codes | Code | Meaning | | ----- | ----------------------- | | `200` | Service is healthy | | `401` | Authentication required | | `422` | Validation error | | `429` | Rate limit exceeded | | `503` | Service is unhealthy | ```bash cURL theme={null} curl --request GET \ --url 'http://localhost:8000/health' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` ```json 200 theme={null} { "status": "pending", "timestamp": "2026-08-31T14:22:05Z", "version": "2026-08-01", "uptime": 1.5, "components": {} } ``` # Kubernetes Health Probe Source: https://docs.stateset.com/api-reference/agents/observability/healthz-list GET http://localhost:8000/healthz Simple health probe for Kubernetes liveness checks. Simple health probe for Kubernetes liveness checks. ### Response `object` ### Status codes | Code | Meaning | | ----- | ----------------------- | | `200` | Successful response | | `401` | Authentication required | | `422` | Validation error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request GET \ --url 'http://localhost:8000/healthz' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` ```json 200 theme={null} { "status": "ok" } ``` # Cache Metrics Source: https://docs.stateset.com/api-reference/agents/observability/metrics-cache-list GET http://localhost:8000/metrics/cache Get cache statistics. Get cache statistics. ### Response `object` Hit, miss and size counters ### Status codes | Code | Meaning | | ----- | ----------------------- | | `200` | Successful response | | `401` | Authentication required | | `422` | Validation error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request GET \ --url 'http://localhost:8000/metrics/cache' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` ```json 200 theme={null} { "timestamp": "2026-08-31T14:22:05Z", "cache": {} } ``` # Get get metrics metrics jsons Source: https://docs.stateset.com/api-reference/agents/observability/metrics-json-list GET http://localhost:8000/metrics/json Get comprehensive system and performance metrics. Requires admin role. Get comprehensive system and performance metrics. Requires admin role. ### Response `DetailedMetricsResponse` Metrics timestamp System metrics API metrics Performance metrics Security metrics Cache metrics ### Status codes | Code | Meaning | | ----- | ------------------------------ | | `200` | Metrics retrieved successfully | | `401` | Authentication required | | `403` | Insufficient permissions | | `422` | Validation error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request GET \ --url 'http://localhost:8000/metrics/json' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` ```json 200 theme={null} { "timestamp": "2026-08-31T14:22:05Z", "system_metrics": {}, "api_metrics": {}, "performance_metrics": {}, "security_metrics": {}, "cache_metrics": {} } ``` # Prometheus Metrics Source: https://docs.stateset.com/api-reference/agents/observability/metrics-list GET http://localhost:8000/metrics Prometheus text exposition format metrics. Prometheus text exposition format metrics. ### Response Returns `text/plain` — Prometheus exposition format. ### Status codes | Code | Meaning | | ----- | ---------------------------- | | `200` | Prometheus exposition format | | `401` | Authentication required | | `404` | Prometheus not installed | | `422` | Validation error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request GET \ --url 'http://localhost:8000/metrics' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` ```json 200 theme={null} "string" ``` # Security Metrics Source: https://docs.stateset.com/api-reference/agents/observability/metrics-security-list GET http://localhost:8000/metrics/security Get security-specific metrics. Requires admin role. Get security-specific metrics. Requires admin role. ### Response `object` Counters from the security monitor ### Status codes | Code | Meaning | | ----- | ------------------------ | | `200` | Successful response | | `401` | Authentication required | | `403` | Insufficient permissions | | `422` | Validation error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request GET \ --url 'http://localhost:8000/metrics/security' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` ```json 200 theme={null} { "timestamp": "2026-08-31T14:22:05Z", "statistics": {}, "recent_events": [ { "type": "standard", "threat_level": "string", "blocked": false, "path": "string", "timestamp": "2026-08-31T14:22:05Z" } ] } ``` # Chat Completions (OpenAI Compatible) Source: https://docs.stateset.com/api-reference/agents/openai/v1-chat-completions-create POST http://localhost:8000/v1/chat/completions Successful Response ### Request body `OpenAIChatCompletionRequest` Model identifier Chat messages Max tokens to generate Sampling temperature Top-p sampling Stream partial responses Stop sequence(s) Tool definitions Tool choice ### Response `OpenAIChatCompletionResponse` ### Status codes | Code | Meaning | | ----- | ----------------------- | | `200` | Successful Response | | `401` | Authentication required | | `422` | Validation Error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request POST \ --url 'http://localhost:8000/v1/chat/completions' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" \ --header 'Content-Type: application/json' \ --data '{ "model": "string", "messages": [ { "role": "system", "content": "Two-person tent, green — replacement for damaged pole set.", "name": "Two-Person Tent", "tool_call_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "tool_calls": [] } ], "max_tokens": 1, "temperature": 1.5, "top_p": 1.5, "stream": false, "stop": "string", "tools": [ {} ], "tool_choice": "string" }' ``` ```json 200 theme={null} { "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "object": "string", "created": 1, "model": "string", "choices": [ {} ], "usage": {} } ``` # Models (OpenAI Compatible) Source: https://docs.stateset.com/api-reference/agents/openai/v1-models-list GET http://localhost:8000/v1/models OpenAI-style model list. With a live vLLM backend the payload is the backend's own /v1/models response passed through untouched… ### Response `object` `list` `model` Unix timestamp (seconds). ### Status codes | Code | Meaning | | ----- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `200` | OpenAI-style model list. With a live vLLM backend the payload is the backend's own `/v1/models` response passed through untouched; with a configured model map, the stub backend, or an unreachable backend, the gateway builds the list itself from its configured model names. | | `401` | Authentication required | | `422` | Validation error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request GET \ --url 'http://localhost:8000/v1/models' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` ```json 200 theme={null} { "object": "string", "data": [ { "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "object": "string", "created": 1, "owned_by": "string" } ] } ``` # Agents API Source: https://docs.stateset.com/api-reference/agents/overview The REST gateway of the stateset-agents framework — agents, conversations, training jobs, and Anthropic- and OpenAI-compatible inference endpoints — generated from the server's own OpenAPI. [StateSet Agents](/stateset-agents/overview) is a Python framework, and most of it is driven from Python or the [CLI](/stateset-agents/cli). It also ships a FastAPI gateway, and this tab is that gateway's contract: create agents, chat with them, start and watch training jobs, and serve a trained model behind endpoints that Anthropic and OpenAI clients already know how to call. | | | | -------- | -------------------------------------------------------------------------------------------------- | | Run it | `stateset-agents serve` — the same as `uvicorn stateset_agents.api.main:app` | | Base URL | `http://localhost:8000` on every page; there is no hosted deployment of this API | | Auth | `Authorization: Bearer ` or `X-API-Key: `; configured with `API_KEYS` / `API_JWT_SECRET` | | Spec | `GET /openapi.json` from a running server — this tab is generated from it | | Version | 2.0.0 (`/api/v1/*` is the versioned surface; the unprefixed routes are the original one) | The server refuses to start with authentication required and no credential source configured. Set `API_KEYS` (comma-separated) or `API_JWT_SECRET`, or `API_REQUIRE_AUTH=false` for local work. Rate limits default to 60 requests a minute per key and 30 unauthenticated, reported in `X-RateLimit-Limit`, `X-RateLimit-Remaining` and `X-RateLimit-Reset`. ## What is here Create, list, inspect and delete agent configurations. Multi-turn chat with an agent, with the transcript kept server-side. Start a GRPO or other RL training job, poll its status, cancel it. `POST /v1/messages` against the configured inference backend; OpenAI-style input accepted. `POST /v1/chat/completions` and `GET /v1/models` — point an OpenAI SDK at the server. Health, readiness and liveness probes, Prometheus metrics, circuit-breaker state. ## Serving a trained model The `/v1/messages` and `/v1/chat/completions` routes are thin: they forward to whatever `INFERENCE_BACKEND` points at — a vLLM server carrying the model you trained — and translate the request and reply into the shape the caller expects. ```bash theme={null} export INFERENCE_BACKEND=vllm export INFERENCE_BACKEND_URL=http://localhost:8001 export INFERENCE_DEFAULT_MODEL=moonshotai/Kimi-K2.5 curl http://localhost:8000/v1/messages \ --header "X-API-Key: $STATESET_AGENTS_API_KEY" \ --header "Content-Type: application/json" \ --data '{ "model": "moonshotai/Kimi-K2.5", "max_tokens": 128, "messages": [{ "role": "user", "content": "Hello" }] }' ``` Add `"stream": true` to either endpoint for server-sent chunks; set `INFERENCE_STREAM_INCLUDE_USAGE=true` to have the backend report token usage in the stream when it supports it. ## Errors and versioning Every error is one envelope: ```json theme={null} { "error": { "code": "ERROR_CODE", "message": "Human-readable message", "details": [] }, "request_id": "uuid", "timestamp": "ISO-8601", "path": "/api/endpoint" } ``` `GET /api/docs/errors` lists the codes a running server can return, and `GET /api/docs/changelog` what changed between its versions. Deprecated routes answer with a `Deprecation` header carrying the sunset date. ## Not in this reference * The **Training Lab** router (`/api/lab/*`) — the backend of the separately published dashboard and mobile apps. It is behind `API_ENABLE_TRAINING_LAB`, off by default, and has no deployment path today. * The **MCP server** and **CLI**, which wrap the framework rather than this gateway — see [MCP server](/stateset-agents/mcp-server) and [CLI](/stateset-agents/cli). The server's own OpenAPI leaves twenty-two responses untyped. Nineteen of them are typed here from the handler that produces them (`spec/overlays/agents.json` in the docs repository); the three that remain — the two `DELETE`s and `GET /v1/models` — say so on their pages. # Cancel Training Source: https://docs.stateset.com/api-reference/agents/training/training-by-delete DELETE http://localhost:8000/training/{training_id} Cancel a running training job. Cancel a running training job. ### Path parameters ### Response `TrainingCancelResponse` Training job identifier New status (cancelled) Status message ### Status codes | Code | Meaning | | ----- | ------------------------ | | `200` | Training cancelled | | `401` | Authentication required | | `403` | Insufficient permissions | | `404` | Training job not found | | `422` | Validation Error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request DELETE \ --url 'http://localhost:8000/training/{training_id}' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` ```json 200 theme={null} { "training_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "status": "pending", "message": "Two-person tent, green — replacement for damaged pole set." } ``` # Get Training Status Source: https://docs.stateset.com/api-reference/agents/training/training-by-get GET http://localhost:8000/training/{training_id} Get detailed status of a specific training job. Get detailed status of a specific training job. ### Path parameters ### Response `TrainingJobDetail` Training job identifier Current status Creation timestamp Start timestamp Completion timestamp Progress percentage Current training episode Total episodes Training metrics Error message if failed Training configuration ### Status codes | Code | Meaning | | ----- | ----------------------- | | `200` | Training job details | | `401` | Authentication required | | `404` | Training job not found | | `422` | Validation Error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request GET \ --url 'http://localhost:8000/training/{training_id}' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` ```json 200 theme={null} { "training_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "status": "pending", "created_at": "2026-08-31T14:22:05Z", "started_at": "2026-08-31T14:22:05Z", "completed_at": "2026-08-31T14:22:05Z", "progress": 0, "current_episode": 0, "total_episodes": 0, "metrics": {}, "error": "string", "config": {} } ``` # Start Training Source: https://docs.stateset.com/api-reference/agents/training/training-create POST http://localhost:8000/training Start a new training job with the specified configuration. Start a new training job with the specified configuration. ### Request body `TrainingRequest` Training scenarios Reward function configuration Number of training episodes Training profile Checkpoint path to resume training from Optional training configuration overrides ### Response `TrainingResponse` Training job identifier Training status Estimated completion time Status message ### Status codes | Code | Meaning | | ----- | ----------------------------- | | `200` | Successful response | | `202` | Training started successfully | | `400` | Invalid configuration | | `401` | Authentication required | | `403` | Insufficient permissions | | `422` | Validation Error | | `429` | Rate limit exceeded | | `500` | Internal server error | ```bash cURL theme={null} curl --request POST \ --url 'http://localhost:8000/training' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" \ --header 'Content-Type: application/json' \ --data '{ "agent_config": { "enable_planning": false, "max_new_tokens": 256, "model_name": "gpt2", "temperature": 0.7 }, "environment_scenarios": [ { "id": "customer_support", "topic": "support", "user_responses": [ "I need help", "Thank you" ] } ], "num_episodes": 50, "profile": "balanced", "resume_from_checkpoint": "./outputs/checkpoint-100", "reward_config": { "helpfulness_weight": 0.7, "safety_weight": 0.3 }, "training_config_overrides": { "continual_strategy": "replay_lwf", "replay_ratio": 0.3 } }' ``` ```json 200 theme={null} { "training_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "status": "pending", "estimated_time": 1.5, "message": "Two-person tent, green — replacement for damaged pole set." } ``` # List Training Jobs Source: https://docs.stateset.com/api-reference/agents/training/training-list GET http://localhost:8000/training Get a paginated list of training jobs. Get a paginated list of training jobs. ### Query parameters Page number Items per page Filter by status ### Response `TrainingJobListResponse-Input` Request tracking ID Response timestamp Total number of items Current page number Items per page Whether there are more pages Whether there are previous pages List of training jobs Training job identifier Current status Creation timestamp Start timestamp Completion timestamp Progress percentage Current training episode Total episodes Training metrics Error message if failed Training configuration ### Status codes | Code | Meaning | | ----- | ----------------------- | | `200` | List of training jobs | | `401` | Authentication required | | `422` | Validation Error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request GET \ --url 'http://localhost:8000/training' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` ```json 200 theme={null} { "request_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "timestamp": "2026-08-31T14:22:05Z", "total": 102, "page": 1, "page_size": 20, "has_next": false, "has_prev": false, "items": [ { "training_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "status": "pending", "created_at": "2026-08-31T14:22:05Z", "started_at": "2026-08-31T14:22:05Z", "completed_at": "2026-08-31T14:22:05Z", "progress": 0, "current_episode": 0, "total_episodes": 0, "metrics": {}, "error": "string", "config": {} } ] } ``` # End Conversation (v1) Source: https://docs.stateset.com/api-reference/agents/v1/conversations-by-delete DELETE http://localhost:8000/api/v1/conversations/{conversation_id} End a v1 conversation. End a v1 conversation. ### Path parameters ### Response `object` ### Status codes | Code | Meaning | | ----- | ----------------------- | | `200` | Successful response | | `401` | Authentication required | | `422` | Validation Error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request DELETE \ --url 'http://localhost:8000/api/v1/conversations/{conversation_id}' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` ```json 200 theme={null} { "status": "ended", "conversation_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6" } ``` # Get Conversation (v1) Source: https://docs.stateset.com/api-reference/agents/v1/conversations-by-get GET http://localhost:8000/api/v1/conversations/{conversation_id} Get a v1 conversation transcript. Get a v1 conversation transcript. ### Path parameters ### Response `object` ### Status codes | Code | Meaning | | ----- | ----------------------- | | `200` | Successful response | | `401` | Authentication required | | `422` | Validation Error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request GET \ --url 'http://localhost:8000/api/v1/conversations/{conversation_id}' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` ```json 200 theme={null} { "conversation_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "messages": [ {} ] } ``` # Chat (v1) Source: https://docs.stateset.com/api-reference/agents/v1/conversations-create POST http://localhost:8000/api/v1/conversations Create or continue a v1 conversation. Create or continue a v1 conversation. ### Request body `ConversationRequestV1` ### Response `ConversationResponseV1` ### Status codes | Code | Meaning | | ----- | ----------------------- | | `200` | Successful Response | | `401` | Authentication required | | `422` | Validation Error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request POST \ --url 'http://localhost:8000/api/v1/conversations' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" \ --header 'Content-Type: application/json' \ --data '{ "message": "Two-person tent, green — replacement for damaged pole set.", "messages": [ { "role": "system", "content": "Two-person tent, green — replacement for damaged pole set." } ], "conversation_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "model": "string", "temperature": 1.5, "max_tokens": 1 }' ``` ```json 200 theme={null} { "conversation_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "response": "string", "tokens_used": 1, "processing_time_ms": 250 } ``` # Health Check (v1) Source: https://docs.stateset.com/api-reference/agents/v1/health-list GET http://localhost:8000/api/v1/health Health check endpoint for API v1. Health check endpoint for API v1. ### Response `object` `api`, `auth` or `rate_limit` `healthy`, `enabled` or `disabled` ### Status codes | Code | Meaning | | ----- | ----------------------- | | `200` | Successful response | | `401` | Authentication required | | `422` | Validation error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request GET \ --url 'http://localhost:8000/api/v1/health' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` ```json 200 theme={null} { "status": "pending", "version": "2026-08-01", "uptime_seconds": 250, "components": [ { "name": "Two-Person Tent", "status": "pending" } ], "timestamp": "2026-08-31T14:22:05Z" } ``` # Cancel Training (v1) Source: https://docs.stateset.com/api-reference/agents/v1/training-by-delete DELETE http://localhost:8000/api/v1/training/{job_id} Cancel a v1 training job. Cancel a v1 training job. ### Path parameters ### Response `TrainingCancelResponseV1` ### Status codes | Code | Meaning | | ----- | ----------------------- | | `200` | Successful Response | | `401` | Authentication required | | `422` | Validation Error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request DELETE \ --url 'http://localhost:8000/api/v1/training/{job_id}' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` ```json 200 theme={null} { "job_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "status": "pending", "message": "Two-person tent, green — replacement for damaged pole set." } ``` # Get Training Status (v1) Source: https://docs.stateset.com/api-reference/agents/v1/training-by-get GET http://localhost:8000/api/v1/training/{job_id} Get v1 training job status. Get v1 training job status. ### Path parameters ### Response `TrainingStatusResponseV1` ### Status codes | Code | Meaning | | ----- | ----------------------- | | `200` | Successful Response | | `401` | Authentication required | | `422` | Validation Error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request GET \ --url 'http://localhost:8000/api/v1/training/{job_id}' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" ``` ```json 200 theme={null} { "job_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "status": "pending", "metrics": {} } ``` # Start Training (v1) Source: https://docs.stateset.com/api-reference/agents/v1/training-create POST http://localhost:8000/api/v1/training Start a new v1 training job. Start a new v1 training job. ### Request body `TrainingRequestV1` Training prompts Training strategy Number of training iterations Optional idempotency key for safe retries ### Response `TrainingCreateResponseV1` ### Status codes | Code | Meaning | | ----- | ----------------------- | | `200` | Successful response | | `202` | Successful Response | | `401` | Authentication required | | `422` | Validation Error | | `429` | Rate limit exceeded | ```bash cURL theme={null} curl --request POST \ --url 'http://localhost:8000/api/v1/training' \ --header "Authorization: Bearer $STATESET_AGENTS_API_KEY" \ --header 'Content-Type: application/json' \ --data '{ "prompts": [ "string" ], "strategy": "computational", "num_iterations": 1, "idempotency_key": "req_01J9X4Q8M2ZK7V3N" }' ``` ```json 200 theme={null} { "job_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "status": "pending" } ``` # API Design Guidelines Source: https://docs.stateset.com/api-reference/api-design-guidelines Best practices and standards for StateSet API development # StateSet API Design Guidelines This document outlines the design principles, patterns, and best practices for the StateSet API. Following these guidelines ensures consistency, usability, and maintainability across all API endpoints. ## Core Principles ### 1. RESTful Design * Use standard HTTP methods appropriately: * `GET` for retrieving resources * `POST` for creating resources * `PUT/PATCH` for updating resources * `DELETE` for removing resources ### 2. Resource-Oriented URLs ```sql theme={null} Good: GET /v1/orders GET /v1/orders/{id} POST /v1/orders PUT /v1/orders/{id} DELETE /v1/orders/{id} Avoid: GET /v1/getOrders POST /v1/createOrder POST /v1/orders/update ``` ### 3. Consistent Naming Conventions * Use lowercase with hyphens for URLs: `/v1/work-orders` * Use camelCase for JSON properties: `firstName`, `createdAt` * Use snake\_case for query parameters: `created_after`, `sort_by` * Pluralize collection endpoints: `/orders` not `/order` ## Request Standards ### Headers Required headers for all requests: ```http theme={null} Authorization: Bearer YOUR_API_KEY Content-Type: application/json Accept: application/json ``` Optional headers: ```http theme={null} X-API-Version: 2024-01-01 X-Idempotency-Key: unique-request-id X-Request-ID: client-generated-id Accept-Language: en-US ``` ### Query Parameters #### Pagination All list endpoints must support: ```http theme={null} GET /v1/orders?limit=20&offset=0 GET /v1/orders?limit=20&cursor=eyJpZCI6MTAwfQ== ``` Default pagination: * `limit`: 20 (max: 100) * `offset`: 0 #### Filtering Use consistent filter patterns: ```http theme={null} # Exact match GET /v1/orders?status=shipped # Multiple values GET /v1/orders?status_in=shipped,delivered # Range queries GET /v1/orders?created_after=2024-01-01T00:00:00Z GET /v1/orders?created_before=2024-12-31T23:59:59Z GET /v1/orders?amount_gte=10000 GET /v1/orders?amount_lte=50000 # Search GET /v1/orders?search=john+doe GET /v1/orders?customer_email=john@example.com ``` #### Sorting ```http theme={null} GET /v1/orders?sort=created_at&order=desc GET /v1/orders?sort=-created_at # Alternative: prefix with - for desc ``` ### Request Body #### Required Fields Clearly mark required fields in documentation: ```json theme={null} { "customer": { // required "email": "...", // required "name": "..." // optional } } ``` #### Nested Objects Use nested objects for logical grouping: ```json theme={null} { "customer": { "email": "john@example.com", "first_name": "John", "last_name": "Doe" }, "shipping_address": { "line1": "123 Main St", "city": "San Francisco", "state": "CA", "postal_code": "94105", "country": "US" } } ``` ## Response Standards ### Success Responses #### Single Resource ```jsonc theme={null} { "id": "ord_1a2b3c4d", "object": "order", "created_at": "2024-01-15T10:30:00Z", "updated_at": "2024-01-15T10:30:00Z", // ... resource fields } ``` #### Collection ```jsonc theme={null} { "object": "list", "data": [ { "id": "ord_1a2b3c4d", "object": "order", // ... resource fields } ], "has_more": true, "total_count": 150, "url": "/v1/orders" } ``` #### With Metadata ```jsonc theme={null} { "data": { "id": "ord_1a2b3c4d", "object": "order", // ... resource fields }, "meta": { "request_id": "req_xyz789", "version": "v1", "timestamp": "2024-01-15T10:30:00Z" } } ``` ### Error Responses #### Standard Error Format ```json theme={null} { "error": { "type": "validation_error", "code": "VALIDATION_ERROR", "message": "Invalid request parameters", "details": { "field_errors": { "email": "Invalid email format", "quantity": "Must be a positive integer" } }, "documentation_url": "https://docs.stateset.com/errors/VALIDATION_ERROR", "request_id": "req_xyz789" } } ``` #### HTTP Status Codes | Status | Usage | | ------------------------- | --------------------------------------- | | 200 OK | Successful GET, PUT, PATCH | | 201 Created | Successful POST creating resource | | 202 Accepted | Request accepted for async processing | | 204 No Content | Successful DELETE | | 400 Bad Request | Invalid request parameters | | 401 Unauthorized | Missing or invalid authentication | | 403 Forbidden | Valid auth but insufficient permissions | | 404 Not Found | Resource doesn't exist | | 409 Conflict | Resource conflict (e.g., duplicate) | | 422 Unprocessable Entity | Validation errors | | 429 Too Many Requests | Rate limit exceeded | | 500 Internal Server Error | Server error | | 503 Service Unavailable | Temporary unavailability | ## Data Types and Formats ### Timestamps Always use ISO 8601 format with timezone: ```json theme={null} { "created_at": "2024-01-15T10:30:00Z", "scheduled_for": "2024-01-20T14:00:00-08:00" } ``` ### Money/Currency Store monetary values in smallest currency unit (cents): ```jsonc theme={null} { "amount": 1999, // $19.99 "currency": "USD" } ``` ### Phone Numbers Use E.164 format: ```json theme={null} { "phone": "+14155551234" } ``` ### Countries and States Use ISO standards: * Countries: ISO 3166-1 alpha-2 (US, CA, GB) * States/Provinces: ISO 3166-2 (CA-ON, US-NY) ```json theme={null} { "country": "US", "state": "CA" } ``` ## Versioning ### URL Versioning Primary versioning method: ``` https://api.stateset.com/api/v1/orders https://api.stateset.com/v2/orders ``` ### Header Versioning For minor versions: ```http theme={null} X-API-Version: 2024-01-01 ``` ### Deprecation Process 1. Announce deprecation with 90-day notice 2. Add deprecation headers: ```http theme={null} Sunset: Sat, 31 Dec 2024 23:59:59 GMT Deprecation: true Link: ; rel="successor-version" ``` 3. Maintain deprecated version for minimum 12 months ## Idempotency ### Implementation Support idempotency for all POST, PUT, PATCH requests: ```http theme={null} POST /v1/orders X-Idempotency-Key: unique-request-id-123 ``` Response includes: ```http theme={null} X-Idempotency-Key: unique-request-id-123 X-Idempotent-Replayed: true ``` An idempotency key must be derived from the operation, not generated per attempt. A client that calls `crypto.randomUUID()` inside its retry loop sends a different key each time and gets exactly the duplicate the header exists to prevent. Derive it from the thing being created — the cart id, the order number, the conversation — and reuse it for every retry of that operation. ## Webhooks ### Event Naming Use dot notation for event types: ``` order.created order.updated order.shipped order.delivered return.requested return.approved payment.succeeded payment.failed ``` ### Webhook Payload ```jsonc theme={null} { "id": "evt_1a2b3c4d", "type": "order.created", "created_at": "2024-01-15T10:30:00Z", "data": { "object": { "id": "ord_xyz789", "object": "order", // ... full object }, "previous_attributes": { // ... for update events } }, "request": { "id": "req_abc123", "idempotency_key": "unique-key-123" } } ``` ### Security Always include signature header: ```http theme={null} X-Stateset-Signature: sha256=3f3b3c4d... ``` ## Performance Guidelines ### Response Times Target response times: * Simple reads: \< 200ms * Complex queries: \< 500ms * Writes: \< 1000ms ### Payload Size * Limit response size to 1MB * Use pagination for large collections * Support field filtering: ```http theme={null} GET /v1/orders?fields=id,status,customer ``` ### Caching Include cache headers: ```http theme={null} Cache-Control: private, max-age=300 ETag: "33a64df551" Last-Modified: Wed, 15 Jan 2024 10:30:00 GMT ``` ## GraphQL Guidelines ### Query Naming ```graphql theme={null} # Good query GetOrder($id: ID!) { order(id: $id) { id status } } # Avoid query fetchOrderData($id: ID!) { getOrderById(id: $id) { id status } } ``` ### Mutations ```graphql theme={null} mutation CreateOrder($input: OrderCreateInput!) { orderCreate(input: $input) { order { id status } userErrors { field message } } } ``` ### Error Handling Return errors in userErrors field: ```json theme={null} { "data": { "orderCreate": { "order": null, "userErrors": [ { "field": "customer.email", "message": "Email is required" } ] } } } ``` ## Testing ### Test Coverage All endpoints must include: * Success path tests * Error handling tests * Edge case tests * Performance tests ### Example Test Cases ```javascript theme={null} describe('POST /v1/orders', () => { test('creates order successfully', async () => { const response = await api.post('/v1/orders', validOrderData); expect(response.status).toBe(201); expect(response.body.object).toBe('order'); }); test('returns 400 for invalid data', async () => { const response = await api.post('/v1/orders', invalidOrderData); expect(response.status).toBe(400); expect(response.body.error.code).toBe('VALIDATION_ERROR'); }); test('handles idempotency', async () => { const key = 'test-idempotency-key'; const response1 = await api.post('/v1/orders', data, { headers: { 'X-Idempotency-Key': key } }); const response2 = await api.post('/v1/orders', data, { headers: { 'X-Idempotency-Key': key } }); expect(response1.body.id).toBe(response2.body.id); }); }); ``` ## Documentation Requirements Every endpoint must document: 1. **Description**: Clear explanation of what the endpoint does 2. **Authentication**: Required permissions 3. **Parameters**: All query params, headers, and body fields 4. **Response**: Success and error response formats 5. **Examples**: Working code examples in multiple languages 6. **Rate Limits**: Specific limits if different from defaults 7. **Webhooks**: Related webhook events 8. **See Also**: Links to related endpoints ## Security Best Practices 1. **Always use HTTPS** 2. **Validate all inputs** - Never trust client data 3. **Rate limit all endpoints** 4. **Log security events** - Failed auth, permission denials 5. **Sanitize outputs** - Prevent XSS in responses 6. **Use secure headers**: ```http theme={null} X-Content-Type-Options: nosniff X-Frame-Options: DENY X-XSS-Protection: 1; mode=block ``` ## Monitoring and Observability Include correlation IDs in all requests: ```http theme={null} X-Request-ID: client-generated-uuid X-Correlation-ID: server-generated-uuid ``` Log format: ```json theme={null} { "timestamp": "2024-01-15T10:30:00Z", "request_id": "req_abc123", "method": "POST", "path": "/v1/orders", "status": 201, "duration_ms": 145, "user_id": "usr_xyz789" } ``` ## Change Management 1. **Backwards Compatibility**: Never break existing integrations 2. **Additive Changes**: New fields are safe to add 3. **Deprecation Notices**: Minimum 90 days 4. **Migration Guides**: Provide clear upgrade paths 5. **Changelog**: Maintain detailed changelog ## Contact For API design questions or to propose changes to these guidelines: * Email: [api-team@stateset.com](mailto:api-team@stateset.com) * Slack: #api-design * GitHub: stateset/api-guidelines ## Next steps The two API surfaces these guidelines apply to. Event delivery, signatures and retry behaviour. The limits a well-behaved client is designed around. The error envelope these standards describe. # Authentication Source: https://docs.stateset.com/api-reference/authentication Complete guide to authenticating with the Stateset API **Quick Start:** All API requests require authentication via API key in the `Authorization` header: ```bash theme={null} Authorization: Bearer sk_live_your_api_key_here ``` Stateset provides multiple authentication methods to secure your API access, including API keys, JWT tokens, and OAuth 2.0 for different use cases. ## Authentication Overview All data in Stateset is private by default, requiring authentication for every API request. We support multiple authentication methods: ### Supported Authentication Methods Best for server-to-server communication and backend integrations Ideal for user-specific access and session management Perfect for third-party integrations and partner access Secure webhook delivery with HMAC signatures ## API Key Authentication ### Key Types and Permissions | Key Type | Prefix | Use Case | Permissions | | ------------------- | ---------- | ---------------------- | --------------------- | | **Secret Key** | `sk_live_` | Server-side operations | Full API access | | **Restricted Key** | `rk_live_` | Limited scope access | Custom permissions | | **Publishable Key** | `pk_live_` | Client-side operations | Read-only public data | | **Test Key** | `sk_test_` | Development & testing | Sandbox environment | ### Creating API Keys 1. Navigate to **Settings → API Keys** in your dashboard 2. Click **Create New Key** 3. Select key type and permissions 4. Copy and securely store your key API keys are shown only once. Store them securely and never expose secret keys in client-side code. ### Using API Keys ```bash cURL theme={null} curl https://api.stateset.com/api/v1/orders \ -H "Authorization: Bearer sk_live_your_api_key_here" \ -H "Content-Type: application/json" ``` ```javascript Node.js theme={null} // Using environment variables (recommended) const headers = { 'Authorization': `Bearer ${process.env.STATESET_API_KEY}`, 'Content-Type': 'application/json' }; const response = await fetch('https://api.stateset.com/api/v1/orders', { headers }); ``` ```python Python theme={null} import os import requests headers = { 'Authorization': f'Bearer {os.getenv("STATESET_API_KEY")}', 'Content-Type': 'application/json' } response = requests.get( 'https://api.stateset.com/api/v1/orders', headers=headers ) ``` ### Restricted API Keys Create keys with specific permissions for enhanced security: Create the key from the StateSet Dashboard and select only the permissions it needs — for example `orders:read` and `customers:read` for a read-only integration — along with an expiry date. *JS SDK support for programmatic API-key management has not shipped yet.* ## JWT Token Authentication JWT tokens provide secure, stateless authentication for user sessions. ### JWT Token Structure ```json theme={null} { "header": { "alg": "RS256", "typ": "JWT", "kid": "key_id_123" }, "payload": { "sub": "user_123", "org_id": "org_456", "role": "admin", "permissions": ["orders:*", "customers:*"], "iat": 1704067200, "exp": 1704070800, "iss": "https://api.stateset.com" }, "signature": "..." } ``` ## GraphQL API Authentication ### GraphQL Endpoint Access Stateset GraphQL API requires authentication via HTTP headers: ```http theme={null} # GraphQL endpoint https://api.stateset.com/graphql # Required headers Authorization: Bearer sk_live_your_api_key Content-Type: application/json ``` ### GraphQL Request Example ```javascript Apollo Client theme={null} import { ApolloClient, InMemoryCache, createHttpLink } from '@apollo/client'; import { setContext } from '@apollo/client/link/context'; const httpLink = createHttpLink({ uri: 'https://api.stateset.com/graphql', }); const authLink = setContext((_, { headers }) => { const token = process.env.STATESET_API_KEY; return { headers: { ...headers, authorization: token ? `Bearer ${token}` : "", } } }); const client = new ApolloClient({ link: authLink.concat(httpLink), cache: new InMemoryCache() }); ``` ```python Python GraphQL theme={null} import requests query = """ query GetOrders { orders(limit: 10) { id status total } } """ response = requests.post( 'https://api.stateset.com/graphql', json={'query': query}, headers={ 'Authorization': f'Bearer {api_key}', 'Content-Type': 'application/json' } ) ``` ## OAuth 2.0 Authentication For third-party integrations and partner access, we support OAuth 2.0: ### OAuth Flow ```bash theme={null} GET https://api.stateset.com/oauth/authorize? client_id=YOUR_CLIENT_ID& redirect_uri=YOUR_REDIRECT_URI& response_type=code& scope=orders:read customers:read& state=RANDOM_STATE ``` ```bash theme={null} POST https://api.stateset.com/oauth/token Content-Type: application/x-www-form-urlencoded grant_type=authorization_code& code=AUTH_CODE& client_id=YOUR_CLIENT_ID& client_secret=YOUR_CLIENT_SECRET& redirect_uri=YOUR_REDIRECT_URI ``` ```bash theme={null} GET https://api.stateset.com/api/v1/orders Authorization: Bearer ACCESS_TOKEN ``` ### Available Scopes | Scope | Description | | ----------------- | ------------------------ | | `orders:read` | Read order data | | `orders:write` | Create and update orders | | `customers:read` | Read customer data | | `customers:write` | Manage customers | | `inventory:read` | View inventory levels | | `inventory:write` | Update inventory | | `returns:*` | Full returns access | | `admin` | Full API access | ## Role-Based Access Control (RBAC) ### User Roles and Permissions Stateset implements fine-grained permissions using role-based access control: | Role | Description | Default Permissions | API Key Prefix | | ---------------- | -------------------- | -------------------------- | -------------- | | **anonymous** | Unauthenticated user | Public read-only endpoints | N/A | | **viewer** | Read-only access | All read operations | `rk_view_` | | **operator** | Standard user | CRUD on assigned resources | `sk_live_` | | **manager** | Team manager | CRUD on team resources | `sk_mgr_` | | **admin** | Full access | All operations | `sk_admin_` | | **super\_admin** | System admin | System configuration | `sk_super_` | ### Permission Matrix | Resource | Anonymous | Viewer | Operator | Manager | Admin | | --------- | --------- | ------ | ---------- | ------------- | ---------- | | Orders | ❌ | Read | CRUD (own) | CRUD (team) | CRUD (all) | | Customers | ❌ | Read | Read (own) | CRUD (team) | CRUD (all) | | Inventory | Read | Read | Read | CRUD | CRUD | | Returns | ❌ | Read | CRUD (own) | CRUD (team) | CRUD (all) | | Reports | ❌ | Read | Read (own) | Read (team) | CRUD | | Settings | ❌ | ❌ | Read (own) | Update (team) | CRUD | ### Custom Permissions with Session Variables Implement fine-grained access control using session variables: #### JWT Claims Structure ```json theme={null} { "https://hasura.io/jwt/claims": { "x-hasura-org-id": "org_123", "x-hasura-user-id": "user_456", "x-hasura-default-role": "operator", "x-hasura-allowed-roles": ["viewer", "operator", "manager"], "x-hasura-team-id": "team_789", "x-hasura-permissions": [ "orders:read", "orders:write", "customers:read" ] }, "iat": 1704067200, "exp": 1704070800 } ``` #### Permission Checks ```javascript theme={null} // Middleware for permission checking function requirePermission(permission) { return (req, res, next) => { const token = req.headers.authorization?.split(' ')[1]; const decoded = jwt.verify(token, process.env.JWT_SECRET); const claims = decoded['https://hasura.io/jwt/claims']; const permissions = claims['x-hasura-permissions'] || []; if (!permissions.includes(permission)) { return res.status(403).json({ error: 'INSUFFICIENT_PERMISSIONS', message: `Missing required permission: ${permission}` }); } req.user = { id: claims['x-hasura-user-id'], org_id: claims['x-hasura-org-id'], role: claims['x-hasura-default-role'], permissions }; next(); }; } // Usage app.post('/api/orders', requirePermission('orders:write'), createOrderHandler ); ``` ## Webhook Authentication ### Webhook Signature Verification All webhooks from Stateset are signed for security: ```javascript theme={null} const crypto = require('crypto'); class WebhookVerifier { constructor(secret) { this.secret = secret; } verify(payload, signature) { // Parse signature header const elements = signature.split(' '); const timestamp = elements.find(e => e.startsWith('t=')).slice(2); const signatures = elements .filter(e => e.startsWith('v1=')) .map(e => e.slice(3)); // Check timestamp (5 minute tolerance) const currentTime = Math.floor(Date.now() / 1000); if (currentTime - parseInt(timestamp) > 300) { throw new Error('Webhook timestamp expired'); } // Compute expected signature const signedPayload = `${timestamp}.${payload}`; const expectedSig = crypto .createHmac('sha256', this.secret) .update(signedPayload) .digest('hex'); // Timing-safe comparison const valid = signatures.some(sig => crypto.timingSafeEqual( Buffer.from(sig), Buffer.from(expectedSig) ) ); if (!valid) { throw new Error('Invalid webhook signature'); } return JSON.parse(payload); } } // Express middleware const webhookAuth = (req, res, next) => { const verifier = new WebhookVerifier(process.env.WEBHOOK_SECRET); try { req.body = verifier.verify( req.rawBody, req.headers['stateset-signature'] ); next(); } catch (error) { res.status(401).json({ error: 'Webhook verification failed' }); } }; ``` ## Security Best Practices * Store keys in environment variables, never in code * Use different keys for different environments * Rotate keys regularly (every 90 days recommended) * Use restricted keys with minimal permissions * Monitor key usage for anomalies * Implement short token lifetimes (15-30 minutes) * Use refresh tokens for long-lived sessions * Store tokens securely (httpOnly cookies) * Implement token revocation * Log all authentication events * Always use HTTPS for API calls * Implement IP allowlisting for production * Use VPN or private networks when possible * Enable CORS with specific origins * Implement rate limiting per API key ## Authentication Examples ### React Hook with Authentication ```jsx theme={null} import { useState, useEffect } from 'react'; import { useAuth } from '@clerk/nextjs'; /** * Custom hook for authenticated Stateset API calls */ export const useStatesetAPI = () => { const { getToken, isSignedIn } = useAuth(); const [loading, setLoading] = useState(false); const [error, setError] = useState(null); const apiCall = async (endpoint, options = {}) => { if (!isSignedIn) { throw new Error('User not authenticated'); } setLoading(true); setError(null); try { const token = await getToken({ template: 'stateset' }); const response = await fetch(`https://api.stateset.com/v1${endpoint}`, { ...options, headers: { 'Authorization': `Bearer ${token}`, 'Content-Type': 'application/json', ...options.headers } }); if (!response.ok) { const error = await response.json(); throw new Error(error.message || 'API request failed'); } const data = await response.json(); return data; } catch (err) { setError(err); throw err; } finally { setLoading(false); } }; return { apiCall, loading, error }; }; // Usage function OrderList() { const { apiCall } = useStatesetAPI(); const [orders, setOrders] = useState([]); useEffect(() => { apiCall('/orders') .then(data => setOrders(data.orders)) .catch(console.error); }, []); return (
{orders.map(order => ( ))}
); } ``` ### Python Authentication Manager ```python theme={null} import os import time import requests from typing import Optional, Dict, Any from functools import wraps class StatesetAuth: def __init__(self, api_key: Optional[str] = None): self.api_key = api_key or os.getenv('STATESET_API_KEY') self.base_url = 'https://api.stateset.com/v1' self.session = requests.Session() self.session.headers.update({ 'Authorization': f'Bearer {self.api_key}', 'Content-Type': 'application/json' }) def validate_key(self) -> bool: """Validate API key is active""" try: response = self.session.get(f'{self.base_url}/auth/validate') return response.status_code == 200 except: return False def with_retry(self, max_retries: int = 3): """Decorator for automatic retry with exponential backoff""" def decorator(func): @wraps(func) def wrapper(*args, **kwargs): for attempt in range(max_retries): try: return func(*args, **kwargs) except requests.exceptions.RequestException as e: if attempt == max_retries - 1: raise wait_time = 2 ** attempt time.sleep(wait_time) return None return wrapper return decorator @with_retry(max_retries=3) def api_request( self, method: str, endpoint: str, data: Optional[Dict[Any, Any]] = None ) -> Dict[Any, Any]: """Make authenticated API request""" url = f'{self.base_url}{endpoint}' response = self.session.request(method, url, json=data) response.raise_for_status() return response.json() # Usage auth = StatesetAuth() if auth.validate_key(): orders = auth.api_request('GET', '/orders') print(f"Found {len(orders['data'])} orders") ``` ## Troubleshooting Authentication ### Common Issues and Solutions | Issue | Cause | Solution | | ----------------------- | ---------------------------- | ------------------------------- | | `401 Unauthorized` | Invalid or expired API key | Check key validity in dashboard | | `403 Forbidden` | Insufficient permissions | Verify key has required scopes | | `429 Too Many Requests` | Rate limit exceeded | Implement exponential backoff | | `CORS Error` | Cross-origin request blocked | Use server-side proxy or SDK | | `Signature Mismatch` | Invalid webhook secret | Verify webhook secret matches | ### Debug Authentication ```bash theme={null} # Test API key validity curl -I https://api.stateset.com/v1/auth/validate \ -H "Authorization: Bearer YOUR_API_KEY" # Check key permissions curl https://api.stateset.com/v1/auth/permissions \ -H "Authorization: Bearer YOUR_API_KEY" ``` *** **Next Steps:** [Create your first API request →](/api-reference/commerce/overview) # StateSet API Source: https://docs.stateset.com/api-reference/backend-framework A high-performance, scalable backend framework built with Rust, Axum, and SeaORM for enterprise-grade applications. # StateSet API: A Robust Backend Framework StateSet API is a modern, scalable, and reliable backend system designed for enterprise-grade web services. Built with Rust, it leverages cutting-edge web technologies and best practices to deliver a high-performance infrastructure solution. It is particularly well-suited for e-commerce and manufacturing businesses, adept at handling order management, inventory control, returns processing, warranty management, shipment tracking, and work order handling. ## Core Features StateSet API is built to handle complex business operations. Here's a breakdown of its key features: * **Order Management:** Full CRUD operations for orders, support for complex workflows and statuses. * **Inventory Control:** Real-time tracking across multiple locations, automated reorder point notifications. * **Returns Processing:** Streamlined return authorization, processing, and integration with refund and exchange systems. * **Warranty Management:** Track and manage product warranties, handle automated claim processing. * **Shipment Tracking:** Real-time integration with major carriers, custom shipment status notifications. * **Manufacturing & Production:** Supplier management, BOM tracking, and version control. * **Work Order Handling:** Create and manage work orders, track progress, and manage resource allocation. ## Tech Stack: The Foundation of Performance Our carefully selected tech stack ensures high performance, scalability, and maintainability. ### Core Technologies * **Language:** [Rust](https://www.rust-lang.org/) (for performance, safety, and concurrency) * **Web Framework:** [Axum](https://github.com/tokio-rs/axum/) (a lightweight, async, and fast web framework) * **Database:** [PostgreSQL](https://www.postgresql.org/) (for robust, reliable data storage) with [SQLx](https://github.com/launchbadge/sqlx) (for async operations). ### ORM and Query Building * [SeaORM](https://www.sea-ql.org/SeaORM) (a powerful async ORM for Rust). ### API Protocols and Services * **REST:** Implemented natively with Axum. * **GraphQL:** Implemented using [Async-GraphQL](https://async-graphql.github.io/) for a high-performance GraphQL API. * **gRPC:** Built with [Tonic](https://github.com/hyperium/tonic) for efficient, type-safe gRPC support. ### Caching and Messaging * **Caching:** [Redis](https://redis.io/) for high-speed data caching. * **Message Queue:** [RabbitMQ](https://www.rabbitmq.com/) for reliable asynchronous processing. ### Observability * **Metrics:** [Prometheus](https://prometheus.io/) for detailed system monitoring. * **Tracing:** [OpenTelemetry](https://opentelemetry.io/) with [Jaeger](https://www.jaegertracing.io/) for distributed tracing. * **Logging:** [slog](https://docs.rs/slog/latest/slog/) for structured, efficient logging. ### Technological Advantages: * **Rust's Performance & Safety:** Guarantees memory safety and prevents common programming errors, leading to a highly reliable and performant API. * **Asynchronous Operations:** Utilizing Rust's async capabilities for high-throughput and non-blocking processing. * **Comprehensive Observability:** Enables proactive maintenance and rapid issue resolution. * **Flexible API Protocols:** Supports various API protocols, ensuring integration with different client applications and services. ### Use Cases: * **E-commerce Platforms:** Efficiently manage orders, inventory, shipments, and customer interactions. * **Manufacturing Systems:** Streamline production workflows, inventory control, supplier management, and quality assurance. * **Enterprise Solutions:** Provides a scalable backend infrastructure for a wide variety of business operations and services. ## Architecture: Modular and Event-Driven StateSet API adopts a modular, asynchronous, event-driven architecture designed for scalability and maintainability. ```mermaid theme={null} flowchart TB Client[Client] subgraph API["StateSet API"] direction TB Handlers[Handlers] Commands[Commands] Queries[Queries] Services[Services] Models[Models] Middleware[Middleware] end subgraph Core["Core Technologies"] direction LR Rust[Rust] Axum[Axum] PostgreSQL[PostgreSQL] end subgraph Data["Data Layer"] direction LR SeaORM[SeaORM] SQLx[SQLx] end subgraph Protocols["API Protocols"] direction TB REST[REST] GraphQL[Async-GraphQL] gRPC[Tonic gRPC] end subgraph Cache["Caching & Messaging"] direction LR Redis[Redis] RabbitMQ[RabbitMQ] end subgraph Observability["Observability"] direction TB Prometheus[Prometheus] OpenTelemetry[OpenTelemetry] Jaeger[Jaeger] Slog[Slog] end Client --> Protocols Protocols --> API API --> Core API --> Data API --> Cache API --> Observability Core --> PostgreSQL Data --> PostgreSQL ``` ### Key Components * **Services:** Implement the core business logic for specific functionalities. * **Handlers:** Process incoming HTTP requests, routing them to the appropriate logic. * **Commands:** Handle operations that modify data or state in the system (writes). * **Queries:** Handle read operations, retrieving data without modifying it. * **Events:** Enable asynchronous communication and processing through event triggers. * **Models:** Represent domain entities, providing a structured data layer. * **Middleware:** Handle cross-cutting concerns like authentication, rate limiting, and logging. ## Performance and Reliability StateSet API scales horizontally: throughput is added by adding nodes rather than by vertical scaling, and the stateless request path means a node can be added or removed without draining sessions. Throughput and availability figures are deliberately not quoted here. Both depend on your workload mix, region, and plan — a number measured on someone else's traffic will not predict yours. For a contractual availability commitment, see your plan's SLA; for capacity planning, ask us to measure against a representative sample of your own traffic. ## Code Examples Here are some representative code snippets to demonstrate the implementation. ### Axum Web Service Entrypoint This code sets up the Axum web service, including routes, middleware, and integration with other services. ```rust theme={null} #[tokio::main] async fn main() -> Result<(), AppError> { dotenv().ok(); let config = Arc::new(config::load()?); let log = setup_logger(&config); info!(log, "Starting Stateset API"; "environment" => &config.environment, "version" => env!("CARGO_PKG_VERSION") ); let app_state = build_app_state(&config, &log).await?; let schema = Arc::new(graphql::create_schema( app_state.services.order_service.clone(), // ... other service clones )); setup_telemetry(&config)?; // Spawn event processing tokio::spawn(events::process_events( app_state.event_sender.subscribe(), app_state.services.clone(), log.clone(), )); // Start gRPC server (conditionally included) #[cfg(feature = "grpc")] let grpc_server = grpc_server::start(config.clone(), app_state.services.clone()).await?; let app = Router::new() .route("/health", get(health::health_check)) .nest("/orders", handlers::orders::routes()) // ... other route nests .route("/proto_endpoint", post(handle_proto_request)) .layer(Extension(app_state)) .layer(Extension(schema)) .layer(TraceLayer::new_for_http()) .layer(CompressionLayer::new()) .layer(axum::middleware::from_fn(auth::auth_middleware)) .layer(axum::middleware::from_fn(rate_limiter::rate_limit_middleware)); let addr = format!("{}:{}", config.host, config.port); info!(log, "Stateset API server running"; "address" => &addr); axum::Server::bind(&addr.parse().unwrap()) .serve(app.into_make_service()) .await .unwrap(); info!(log, "Shutting down"); Ok(()) } ``` > This shows the main entry point of the server including the config loading, setup of global services, the route definition, telemetry and the server startup ### Handlers: API Request Routing Here are some examples of how API requests are routed to the correct command. #### Create Order Handler ```rust theme={null} async fn create_order( State(order_service): State>, Json(command): Json, ) -> Result { let result = command.execute(order_service).await?; Ok(Json(result)) } ``` > This handler pulls the OrderService from the application state and calls the execute method on the CreateOrderCommand #### Close Return Handler ```rust theme={null} async fn close_return( State(return_service): State>, Path(return_id): Path, ) -> Result { let command = CloseReturnCommand { return_id }; let closed_return = command.execute(return_service).await?; Ok(Json(closed_return)) } ``` > This handler pulls the ReturnService from the application state and calls the execute method on the CloseReturnCommand ### Commands: Business Logic Execution The following shows the Create Order and Close Return commands which handle the business logic. #### Create Order Command ```rust theme={null} #[derive(Debug, Serialize, Deserialize, Validate)] pub struct CreateOrderCommand { pub customer_id: Uuid, #[validate(length(min = 1, message = "At least one item is required"))] pub items: Vec, } #[derive(Debug, Serialize, Deserialize)] pub struct OrderItem { pub product_id: Uuid, #[validate(range(min = 1))] pub quantity: i32, } #[derive(Debug, Serialize, Deserialize)] pub struct CreateOrderResult { pub id: Uuid, pub customer_id: Uuid, pub status: String, pub created_at: DateTime, pub items: Vec, } #[async_trait::async_trait] impl Command for CreateOrderCommand { type Result = CreateOrderResult; #[instrument(skip(self, db_pool, event_sender))] async fn execute( &self, db_pool: Arc, event_sender: Arc, ) -> Result { // Validates the incoming command. Returns a validation error if invalid self.validate().map_err(|e| { ORDER_CREATION_FAILURES.inc(); let msg = format!("Invalid input: {}", e); error!("{}", msg); ServiceError::ValidationError(msg) })?; let db = db_pool.as_ref(); let saved_order = self.create_order(db).await?; self.log_and_trigger_event(&event_sender, &saved_order).await?; ORDER_CREATIONS.inc(); Ok(CreateOrderResult { id: saved_order.id, customer_id: saved_order.customer_id, status: saved_order.status, created_at: saved_order.created_at.and_utc(), items: self.items.clone(), }) } } // ... create order implementation ``` > This shows the command definition, validation, and execution of the create order command. #### Close Return Command ```rust theme={null} #[derive(Debug, Serialize, Deserialize)] pub struct CloseReturnCommand { pub return_id: Uuid, } #[derive(Debug, Serialize, Deserialize)] pub struct CloseReturnResult { pub id: Uuid, pub object: String, pub completed: bool, } #[async_trait::async_trait] impl Command for CloseReturnCommand { type Result = CloseReturnResult; #[instrument(skip(self, db_pool, event_sender))] async fn execute( &self, db_pool: Arc, event_sender: Arc, ) -> Result { let db = db_pool.as_ref(); let completed_return = self.close_return(db).await?; self.log_and_trigger_event(&event_sender, &completed_return) .await?; Ok(CloseReturnResult { id: completed_return.id, object: "return".to_string(), completed: true, }) } } // ... close return implementation ``` > This shows the command definition, and execution of the close return command. ### Queries: Data Retrieval These show the two return queries used by the application. #### Get Returns By Order Query ```rust theme={null} #[derive(Debug, Serialize, Deserialize)] pub struct GetReturnsByOrderQuery { pub order_id: i32, } #[async_trait] impl Query for GetReturnsByOrderQuery { type Result = Vec; async fn execute(&self, db_pool: Arc) -> Result { let db = db_pool.get().map_err(|_| ServiceError::DatabaseError)?; Return::find() .filter(Return::Column::OrderId.eq(self.order_id)) .all(&db) .await .map_err(|_| ServiceError::DatabaseError) } } ``` > This shows how we can find the returns based on the order ID using SeaORM #### Get Returns By Date Range Query ```rust theme={null} #[derive(Debug, Serialize, Deserialize)] pub struct GetReturnsByDateRangeQuery { pub start_date: DateTime, pub end_date: DateTime, pub limit: u64, pub offset: u64, } #[async_trait] impl Query for GetReturnsByDateRangeQuery { type Result = Vec; async fn execute(&self, db_pool: Arc) -> Result { let db = db_pool.get().map_err(|_| ServiceError::DatabaseError)?; Return::find() .filter(Return::Column::CreatedAt.between(self.start_date, self.end_date)) .order_by_desc(Return::Column::CreatedAt) .limit(self.limit) .offset(self.offset) .all(&db) .await .map_err(|_| ServiceError::DatabaseError) } } ``` > This shows how we can find the returns based on a date range using SeaORM. To learn more about SeaORM, check out [this article](https://www.sea-ql.org/blog/2024-08-04-sea-orm-1.0/). ## Acknowledgments We express our gratitude to the open-source community and the creators of the following libraries: * [Axum](https://github.com/tokio-rs/axum/) for the powerful web framework. * [SeaORM](https://www.sea-ql.org/SeaORM) for providing a robust ORM. * [Tonic](https://github.com/hyperium/tonic) for enabling gRPC support. * [Async-GraphQL](https://async-graphql.github.io/) for efficient GraphQL handling. # API Changelog Source: https://docs.stateset.com/api-reference/changelog Changes to the StateSet One API that affect callers — new endpoints, new headers, and behaviour you may need to adapt to. Changes that affect you as a caller: new endpoints, new response headers, and behaviour changes. This page preserves legacy StateSet One release notes. For current service changes, use the [documentation changelog](/changelog) and each API tab’s history. This is deliberately not a complete engineering log. Internal refactors, test infrastructure, and CI changes are omitted because they change nothing about how you call the API. Absence from this historical page is not evidence that a release had no caller-visible changes. ## Unreleased **New response headers.** Three additions, all safe to ignore but useful to read: | Header | What it does | | -------------------- | ----------------------------------------------------------------------------------------------------------- | | `RateLimit-*` | Rate limit state in the documented deployment-specific form, alongside the existing `X-RateLimit-*` headers | | `X-Correlation-ID` | Propagates across service boundaries, so one ID traces a request through every internal hop | | Deprecation warnings | Versioning middleware now emits a warning header when you call a deprecated endpoint | **Bulk operations are rate limited per operation type.** Limits are configured independently per bulk operation rather than sharing one global bucket. If you drive bulk imports at a fixed rate, check `RateLimit-Remaining` rather than assuming your previous throughput still applies. A bulk endpoint may now throttle at a different threshold than the general API. **Security headers** are now sent on all API responses: `Content-Security-Policy`, `HSTS` with a one-year max-age and `preload`, `Permissions-Policy`, `X-Permitted-Cross-Domain-Policies`, `X-XSS-Protection`, and `Referrer-Policy: strict-origin-when-cross-origin`. Cache-control headers prevent caching of sensitive responses, and the `Server` header now returns a generic `Stateset-API` rather than framework details. ## 0.1.6 — 2024-10-30 **New endpoint groups:** * [Work orders](/api-reference/commerce/work_orders/work-orders-list) — manufacturing work order tracking * [Bill of materials](/api-reference/commerce/bom/boms-list) * [Advanced shipping notices](/api-reference/commerce/inbound_shipments/inbound-shipments-list) — ASN management * Analytics endpoints, with permission-based access Also: a Postman collection for the API, improved error handling, and fixes to inventory allocation edge cases and the return workflow. ## 0.1.5 — 2024-09-29 * Crypto payment integration * AI-powered checkout * Product feed specifications * Inventory management gained **lot tracking** * Improved shipment tracking and warranty claim processing * Fixed rate limiting edge cases ## 0.1.4 — 2024-08-24 * **Multi-factor authentication** support * [RBAC](/api-reference/authentication) with granular permissions * Password policy enforcement * Redis and in-memory caching, with cache warming * Fixed JWT token refresh, and concurrent inventory reservation bugs The concurrent-reservation fix matters if you built a workaround for double-allocation under concurrent load. That behaviour is corrected — a retry-and-compare workaround is now redundant, and may mask the correct result. ## 0.1.3 — 2024-08-11 * **[Request idempotency](/api-reference/overview#your-first-request)** — send an `Idempotency-Key` on writes and a retried request replays the original result instead of repeating the effect * Comprehensive [health check endpoints](/api-reference/commerce/health/health-list) * Circuit breaker pattern for upstream calls * OpenTelemetry integration * Rate limiting gained per-path policies * Improved error messages and codes Idempotency keys arriving in 0.1.3 is the single most useful change on this page for anyone integrating agents. An agent that retries on timeout will otherwise create duplicate orders. ## 0.1.2 — 2024-07-24 * **[gRPC support](/api-reference/grpc-framework)** with 30+ proto definitions * Swagger UI * Prometheus metrics endpoint * `stateset-cli` for local development * Fixed timeout handling on external service calls and memory usage on large result sets ## 0.1.1 — 2024-07-18 First functional surface: [order management](/api-reference/commerce/orders/orders-list), [inventory control](/api-reference/commerce/inventory/inventory-list) with allocations, [returns processing](/api-reference/commerce/returns/returns-list), [warranties](/api-reference/commerce/warranties/warranties-list), and [shipment tracking](/api-reference/commerce/shipments/shipments-by-get). ## 0.1.0 — 2024-07-01 Initial release. CRUD for orders, inventory, and returns; JWT authentication. ## Related * [Overview](/api-reference/overview) — base URL, auth, errors, rate limits * [Rate limiting](/api-reference/rate-limiting) — every header and what to do with it * [Errors](/api-reference/errors) — the full error code catalogue # Revoke an admin key Source: https://docs.stateset.com/api-reference/chat-widget/admin/api-admin-keys-by-delete DELETE https://api.example.com/api/admin/keys/{id} Revoke an admin key ### Path parameters Key id ### Response `Ok` ### Status codes | Code | Meaning | | ----- | ------- | | `200` | OK | | `401` | Error | | `404` | Error | ```bash cURL theme={null} curl --request DELETE \ --url 'https://api.example.com/api/admin/keys/{id}' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ```json 200 theme={null} { "ok": true } ``` # Create an admin key Source: https://docs.stateset.com/api-reference/chat-widget/admin/api-admin-keys-create POST https://api.example.com/api/admin/keys With a bearer key for your own org, or — once, for a new org — bootstrapSecret (the server's ADMIN_BOOTSTRAP_SECRET) and no bearer. The key is returned once. With a bearer key for your own org, or — once, for a new org — `bootstrapSecret` (the server's `ADMIN_BOOTSTRAP_SECRET`) and no bearer. The key is returned once. ### Request body `AdminKeyCreate` Only when minting the first key for a new org, with no bearer ### Response `AdminKey` Returned once, on creation ### Status codes | Code | Meaning | | ----- | ------- | | `200` | OK | | `400` | Error | | `401` | Error | ```bash cURL theme={null} curl --request POST \ --url 'https://api.example.com/api/admin/keys' \ --header 'Content-Type: application/json' \ --data '{ "orgId": "string", "label": "Replacement tent pole set", "bootstrapSecret": "YOUR_API_KEY" }' ``` ```json 200 theme={null} { "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "key": "string", "orgId": "string", "label": "Replacement tent pole set" } ``` # List admin keys Source: https://docs.stateset.com/api-reference/chat-widget/admin/api-admin-keys-list GET https://api.example.com/api/admin/keys Array of keys (never the secret) ### Response `object[]` — each element: ### Status codes | Code | Meaning | | ----- | -------------------------------- | | `200` | Array of keys (never the secret) | | `401` | Error | ```bash cURL theme={null} curl --request GET \ --url 'https://api.example.com/api/admin/keys' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ```json 200 theme={null} [ { "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "label": "Replacement tent pole set", "createdAt": "2026-08-31T14:22:05Z", "revokedAt": "2026-08-31T14:22:05Z", "lastUsedAt": "2026-08-31T14:22:05Z" } ] ``` # Revoke an OAuth grant Source: https://docs.stateset.com/api-reference/chat-widget/admin/api-admin-oauth-grants-by-delete DELETE https://api.example.com/api/admin/oauth/grants/{id} Revoke an OAuth grant ### Path parameters Grant id ### Response `Ok` ### Status codes | Code | Meaning | | ----- | ------- | | `200` | OK | | `401` | Error | | `404` | Error | ```bash cURL theme={null} curl --request DELETE \ --url 'https://api.example.com/api/admin/oauth/grants/{id}' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ```json 200 theme={null} { "ok": true } ``` # List OAuth grants (MCP connector) Source: https://docs.stateset.com/api-reference/chat-widget/admin/api-admin-oauth-grants-list GET https://api.example.com/api/admin/oauth/grants Array of grants ### Response `object[]` — each element: ### Status codes | Code | Meaning | | ----- | --------------- | | `200` | Array of grants | | `401` | Error | ```bash cURL theme={null} curl --request GET \ --url 'https://api.example.com/api/admin/oauth/grants' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ```json 200 theme={null} [ { "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "clientId": "string", "createdAt": "2026-08-31T14:22:05Z" } ] ``` # Org record and counts Source: https://docs.stateset.com/api-reference/chat-widget/admin/api-admin-org-list GET https://api.example.com/api/admin/org The org record plus counts ### Response `object` Row counts for the org ### Status codes | Code | Meaning | | ----- | -------------------------- | | `200` | The org record plus counts | | `401` | Error | ```bash cURL theme={null} curl --request GET \ --url 'https://api.example.com/api/admin/org' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ```json 200 theme={null} { "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "name": "Two-Person Tent", "createdAt": "2026-08-31T14:22:05Z", "counts": { "sites": 1, "keys": 1 } } ``` # Site-scoped thread analytics Source: https://docs.stateset.com/api-reference/chat-widget/admin/api-admin-sites-by-analytics-get GET https://api.example.com/api/admin/sites/{id}/analytics Site-scoped thread analytics ### Path parameters Site id ### Response `object` `null` for an org-wide scope `null` unless a range was requested The thread-analytics report (same shape as the ResponseCX thread-analytics `analysis`) ### Status codes | Code | Meaning | | ----- | ---------------------------- | | `200` | Site-scoped thread analytics | | `401` | Error | | `404` | Error | ```bash cURL theme={null} curl --request GET \ --url 'https://api.example.com/api/admin/sites/{id}/analytics' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ```json 200 theme={null} { "skipped": true, "orgId": "string", "siteId": "string", "scope": "string", "note": "Two-person tent, green — replacement for damaged pole set.", "range": {}, "limit": 20, "analysis": {} } ``` # Draft and published config Source: https://docs.stateset.com/api-reference/chat-widget/admin/api-admin-sites-by-config-get GET https://api.example.com/api/admin/sites/{id}/config Draft and published config ### Path parameters Site id ### Response `SiteConfig` ### Status codes | Code | Meaning | | ----- | ------- | | `200` | OK | | `401` | Error | | `404` | Error | ```bash cURL theme={null} curl --request GET \ --url 'https://api.example.com/api/admin/sites/{id}/config' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ```json 200 theme={null} { "draft": {}, "published": {} } ``` # Publish the draft Source: https://docs.stateset.com/api-reference/chat-widget/admin/api-admin-sites-by-config-publish-create POST https://api.example.com/api/admin/sites/{id}/config/publish 400 no_draft when nothing is saved. `400 no_draft` when nothing is saved. ### Path parameters Site id ### Response `object` The site's widget settings — free-form, whatever the editor saved `null` until published ### Status codes | Code | Meaning | | ----- | -------------------- | | `200` | The published config | | `400` | Error | | `401` | Error | | `404` | Error | ```bash cURL theme={null} curl --request POST \ --url 'https://api.example.com/api/admin/sites/{id}/config/publish' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ```json 200 theme={null} { "version": 1, "config": {}, "publishedAt": "2026-08-31T14:22:05Z" } ``` # Roll back to a version Source: https://docs.stateset.com/api-reference/chat-widget/admin/api-admin-sites-by-config-rollback-create POST https://api.example.com/api/admin/sites/{id}/config/rollback Body: { version }. Body: `{ version }`. ### Path parameters Site id ### Response `object` The site's widget settings — free-form, whatever the editor saved `null` until published ### Status codes | Code | Meaning | | ----- | ------------------- | | `200` | The restored config | | `400` | Error | | `401` | Error | | `404` | Error | ```bash cURL theme={null} curl --request POST \ --url 'https://api.example.com/api/admin/sites/{id}/config/rollback' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ```json 200 theme={null} { "version": 1, "config": {}, "publishedAt": "2026-08-31T14:22:05Z" } ``` # Save the draft config Source: https://docs.stateset.com/api-reference/chat-widget/admin/api-admin-sites-by-config-update PUT https://api.example.com/api/admin/sites/{id}/config Body: { config }. Body: `{ config }`. ### Path parameters Site id ### Response `object` The site's widget settings — free-form, whatever the editor saved `null` until published ### Status codes | Code | Meaning | | ----- | --------------- | | `200` | The saved draft | | `400` | Error | | `401` | Error | | `404` | Error | ```bash cURL theme={null} curl --request PUT \ --url 'https://api.example.com/api/admin/sites/{id}/config' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ```json 200 theme={null} { "version": 1, "config": {}, "publishedAt": "2026-08-31T14:22:05Z" } ``` # Read one config version Source: https://docs.stateset.com/api-reference/chat-widget/admin/api-admin-sites-by-config-versions-by-get GET https://api.example.com/api/admin/sites/{id}/config/versions/{version} One config version ### Path parameters Site id Version number ### Response `object` The site's widget settings — free-form, whatever the editor saved `null` until published ### Status codes | Code | Meaning | | ----- | ------------------ | | `200` | One config version | | `401` | Error | | `404` | Error | ```bash cURL theme={null} curl --request GET \ --url 'https://api.example.com/api/admin/sites/{id}/config/versions/{version}' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ```json 200 theme={null} { "version": 1, "config": {}, "publishedAt": "2026-08-31T14:22:05Z" } ``` # List config versions Source: https://docs.stateset.com/api-reference/chat-widget/admin/api-admin-sites-by-config-versions-get GET https://api.example.com/api/admin/sites/{id}/config/versions All versions, newest first ### Path parameters Site id ### Response `object[]` — each element: The site's widget settings — free-form, whatever the editor saved `null` until published ### Status codes | Code | Meaning | | ----- | -------------------------- | | `200` | All versions, newest first | | `401` | Error | | `404` | Error | ```bash cURL theme={null} curl --request GET \ --url 'https://api.example.com/api/admin/sites/{id}/config/versions' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ```json 200 theme={null} [ { "version": 1, "config": {}, "publishedAt": "2026-08-31T14:22:05Z" } ] ``` # Delete a site Source: https://docs.stateset.com/api-reference/chat-widget/admin/api-admin-sites-by-delete DELETE https://api.example.com/api/admin/sites/{id} Delete a site ### Path parameters Site id ### Response `Ok` ### Status codes | Code | Meaning | | ----- | ------- | | `200` | OK | | `401` | Error | | `404` | Error | ```bash cURL theme={null} curl --request DELETE \ --url 'https://api.example.com/api/admin/sites/{id}' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ```json 200 theme={null} { "ok": true } ``` # Get a site Source: https://docs.stateset.com/api-reference/chat-widget/admin/api-admin-sites-by-get GET https://api.example.com/api/admin/sites/{id} Get a site ### Path parameters Site id ### Response `Site` ### Status codes | Code | Meaning | | ----- | ------- | | `200` | OK | | `401` | Error | | `404` | Error | ```bash cURL theme={null} curl --request GET \ --url 'https://api.example.com/api/admin/sites/{id}' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ```json 200 theme={null} { "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "platform": "string", "domain": "string", "createdAt": "2026-08-31T14:22:05Z" } ``` # Install on a Shopify store Source: https://docs.stateset.com/api-reference/chat-widget/admin/api-admin-sites-by-install-shopify-create POST https://api.example.com/api/admin/sites/{id}/install/shopify Uses the site's stored accessToken. Uses the site's stored `accessToken`. ### Path parameters Site id ### Response `object` On failure (`ok: false`): `unsafe_domain`, `unsafe_address`, `shopify_unreachable`, or the Shopify API error ### Status codes | Code | Meaning | | ----- | -------------- | | `200` | Install result | | `401` | Error | | `404` | Error | ```bash cURL theme={null} curl --request POST \ --url 'https://api.example.com/api/admin/sites/{id}/install/shopify' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ```json 200 theme={null} { "ok": true, "method": "script_tag", "scriptTagId": 1, "note": "Two-person tent, green — replacement for damaged pole set.", "error": "string" } ``` # Generate the install snippet Source: https://docs.stateset.com/api-reference/chat-widget/admin/api-admin-sites-by-install-snippet-create POST https://api.example.com/api/admin/sites/{id}/install/snippet Body: { platform }. Returns { snippet, instructions }. Body: `{ platform }`. Returns `{ snippet, instructions }`. ### Path parameters Site id ### Response `object` The \ ``` Replace `YOUR_WIDGET_ID` with the permanent widget ID from the builder and place the script before the closing body tag. The hosted loader initializes the widget; this path does not require the separate React/UMD mounting example. Do not put a private service API key in the script. If your site uses a content security policy, allow the loader origin in `script-src` and the API origin in `connect-src`. Test the actual published page on an allowed origin. If using a tag manager or cached site, confirm the snippet reached the live page and was not inserted twice. ## 3. Follow a message into the desk
VISITORHosted widget

Sends a message with its session and thread context.

RESPONSECXAgent and transcript

Handles the message and persists the conversation for the organization.

OPERATORChat Desk

Reads the thread and saves a reply for the same visitor conversation.

Send a unique test message, such as “Widget handoff test: shipping policy.” In the desk's **Chat** view, find that text and compare the thread identity and agent. The widget runtime persists chat threads and messages and also records response evidence. The desk combines native chat threads with eligible response-backed records; a response row alone is not proof that an editable native thread exists. The visitor's open widget receives thread updates through streaming with polling fallback. An operator reply must be saved to the same thread and retrieved by that visitor session to appear. Desk event updates and widget updates are separate connections. ## 4. Verify a human handoff 1. In Chat Desk, **Claim** the test conversation and choose **Pause AI / take over**. 2. Send another message from the same visitor widget. When the widget backend detects human takeover, it saves the customer's message and returns a handoff acknowledgement instead of generating a new AI answer. 3. Use **Send reply** in the desk. Confirm the result says the transcript was saved, then watch the reply appear in the visitor widget. 4. Add a clearly marked test **internal note** in the desk. Confirm the team can see it and the visitor cannot. Public widget transcript reads filter internal-note messages. 5. If returning the conversation to automation, clear the relevant takeover state and verify the next test message. Releasing assignment alone does not clear every takeover signal. Takeover is checked before generation using thread and latest-response state. The current public chat handler continues with AI if that state lookup fails. Treat the desk pause as conversation coordination, not a fail-closed emergency stop for the whole deployment. ## Completion checklist * The installed widget uses the intended widget ID, agent, organization, and allowed origin. * A real visitor message is visible in the desk under the matching conversation. * Human takeover is observed on a subsequent visitor message. * An operator reply appears in that same visitor session. * An internal note remains absent from the public transcript. Keep the thread ID and the observed results in your [onboarding record](/guides/platform-onboarding#keep-a-record-you-can-resume). ## Troubleshooting | Symptom | Check | | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | | Builder preview works, website does not | Widget publication, exact allowed origin, installed script, permanent widget ID, and browser network errors | | AI responds but desk lacks messages | Transcript persistence and organization/thread mapping; inspect server errors rather than assuming every AI response was stored | | Reply saved but visitor sees nothing | Same thread/session, open widget, blocked streaming/polling, and reply versus note audience | | AI responds during a handoff | Actual takeover state and backend lookup failures; assignment alone is not enough | | History or updates are rejected | Origin/session/thread authorization; keep the loader's returned thread context intact | Continue with [desk operations](/stateset-response/chat-desk) or [channel coverage](/stateset-response/chat-desk-channels). Implementation checked against `response-one-prod` on 2026-09-20: `pages/widget-builder.js`, `public/widget/v2/loader.js`, `pages/api/public/widget/chat.js`, public widget v2 thread routes, `lib/widget/publicChatThreads.ts`, and `lib/chat-desk/thread-state.ts`. # Response Chat Widget Source: https://docs.stateset.com/stateset-response/response-chat-widget Configuration, revenue capture, deployment, and hardening for the onsite chat widget. Using the hosted ResponseCX Widget Builder and Chat Desk? Follow [Widget to desk](/stateset-response/chat-desk-widget) for the hosted v2 loader, human handoff, and transcript verification. This page covers the standalone widget integration; its custom backend is not automatically connected to the desk. The reference for the onsite chat widget. To get it running first, start with the [Onsite Chat Quickstart](/stateset-response/response-chat-widget-quickstart). ## What it ships with * **AI ready** — OpenAI, Anthropic, and Gemini integration with streaming, tool feedback, and source citations. * **Rich UI** — Markdown with syntax highlighting, reactions, exports, forms, and programmable actions. * **Mobile ready** — responsive fullscreen with safe-area handling for modern iPhone Safari. * **Operations ready** — built-in record views for orders, products, subscriptions, returns, and exchanges. * **Global ready** — locale packs, translation overrides, automatic RTL layout. * **Developer friendly** — UMD bundle, type definitions, mock and AI dev servers, Docker image. ## Hosting options | Approach | When to use | Notes | | ------------------- | ---------------------------------------------- | ------------------------------------------------------------------------ | | **Static CDN** | You already have APIs and just need the bundle | Serve `chat-widget.css` and `chat-widget.bundle.js` with long cache TTLs | | **Docker + Nginx** | Easiest full-stack demo | `docker build -t chat-widget . && docker run -p 8080:80 chat-widget` | | **Edge function** | Injecting the snippet across multiple sites | Host the bundle on a CDN and lazy-load via script tag | | **React / Vue app** | Embedding inside an SPA | Use the UMD API (`window.ChatWidget`) or wrap it in a component | ### Production CDN The recommended production setup is Cloud Storage plus Cloud CDN with versioned assets. One-time setup creates the bucket, CDN backend, URL map, HTTPS proxy, and global IP: ```bash theme={null} scripts/setup-cdn-gcp.sh \ --project stateset-network \ --bucket stateset-chat-cdn \ --domain stateset.chat \ --location US ``` Then in DNS for `stateset.chat`, create an `A` record pointing at the global IP the script prints, and wait for the managed SSL certificate to become ACTIVE — that can take 15–60 minutes. Publish a version: ```bash theme={null} npm run build scripts/publish-cdn.sh --bucket stateset-chat-cdn --version 1.0.0 ``` Embed the versioned assets: ```html theme={null} ``` Versioned filenames are what make long cache TTLs safe. Publish a new version rather than overwriting one — then you rarely need cache invalidation at all: ```bash theme={null} gcloud compute url-maps invalidate-cdn-cache stateset-chat-cdn-map --path "/*" ``` ## Backend options | Server | Suitable for | | ---------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Mock server** (`dev-server.js`) | Demos and UI work. No auth, no persistence | | **AI server** (`server-openai.js`) | OpenAI/Anthropic/Gemini completions | | **Your API** | Production — expose `POST /api/chat` plus optional thread CRUD; the full contract is the [Chat Widget API reference](/api-reference/chat-widget/overview) | The bundled AI server has **no authentication, logging, or database**. Add all three before putting it in front of customers — it exists to prove the wiring, not to run production traffic. ### Hardening * Put your API behind OAuth/JWT or signed session cookies. * Rate limit inbound chat requests per IP and session at the edge. * Validate and sanitize user uploads before echoing them in the UI. * Sanitize AI-generated markdown — the widget uses DOMPurify by default. * Set Content-Security-Policy headers in `nginx.conf` or at your hosting platform. ### Secrets and runtime Keep `.env` out of version control, hold provider keys in a secret manager (AWS Secrets Manager, Doppler, Vault), and pin Node via `.nvmrc` or your host's runtime setting — the widget requires **20.17+**. ## Revenue capture v2 For the ResponseCX lead-qualifier rollout, the bundle exports helpers that turn a public config response into mountable widget props, so configuration lives server-side rather than in the embed snippet. ```html theme={null} ``` Helpers: * `window.ChatWidget.buildRevenueCapturePageContext(...)` * `window.ChatWidget.createRevenueCaptureMountProps(...)` * `window.ChatWidget.mountRevenueCaptureWidget(...)` The config route is `GET /api/public/widget/v2/config?token=`. For a local end-to-end mock of the public contract, run `npm run dev` and open `/widget/v2/demo.html`. The token in the embed is a **signed public** widget token — it is visible to anyone who views source, and is meant to be. Do not put a private API key in the embed snippet. ## Related * [Onsite Chat Quickstart](/stateset-response/response-chat-widget-quickstart) * [Response Public API](/stateset-response/response-public-api) * [Response MCP Servers](/stateset-response/response-mcp) # Onsite Chat Quickstart Source: https://docs.stateset.com/stateset-response/response-chat-widget-quickstart Add the Response chat widget to your site — embed it, point it at your backend, and go live. Using the hosted ResponseCX Widget Builder and Chat Desk? Follow [Widget to desk](/stateset-response/chat-desk-widget) for the hosted v2 loader, human handoff, and transcript verification. This page covers the standalone widget integration; its custom backend is not automatically connected to the desk. The Response Chat Widget is a production-ready, AI-powered chat widget you drop into any web application. This page gets it on your site and talking to a backend. ## 1. Embed the widget The widget ships as a UMD bundle. Include React, the stylesheet, and the bundle, then mount it into any container: ```html theme={null}
``` Pin a version in the CDN URL — replace `vX.Y.Z` with a real release. Pointing at a floating "latest" means a widget upgrade ships to your production site without you deploying anything. `mount()` returns a control surface: ```js theme={null} api.open(); api.close(); api.toggle(); ``` `provider` accepts `openai`, `anthropic`, or `gemini`. ## 2. Wire up your backend With `messageEndpoint` set, the widget POSTs JSON in this shape: ```json theme={null} { "subject": "Optional conversation subject", "message": "Customer message text", "attachments": [{ "name": "screenshot.png", "type": "image/png", "size": 12345 }], "meta": { "department": "Support", "customFields": {} }, "sessionId": "optional-session", "source": "response-chat-widget", "channel": "web", "threadId": "abc123", "provider": "openai" } ``` Your endpoint should return an object the widget can render. If your API has a different shape, you don't need to change it — override both directions: | Hook | Purpose | | ----------------------------------------------- | ----------------------------------------------------------------------------- | | `buildMessagePayload(payload, outgoingMessage)` | Reshape the outgoing request | | `parseResponse(response)` | Map your response to `{ text, suggestions, threadId?, toolCalls?, sources? }` | ### Threads Provide `threadEndpoint` and `threadId` to load history. The widget issues `GET {endpoint}?threadId={id}` with optional bearer auth. Full CRUD is available via `threadCreateEndpoint`, `threadUpdateEndpoint`, and `threadDeleteEndpoint`, with `buildThreadRequest`, `buildThreadUrl`, `buildThreadUpdateUrl`, and `buildThreadDeleteUrl` for custom URL shapes. ## 3. Run it locally first ```bash theme={null} npm install cp env.example .env # set OPENAI_API_KEY / ANTHROPIC_API_KEY / GEMINI_API_KEY npm run dev # mock backend + demo widget → http://localhost:3000 npm run dev:ai # talk to real models (needs an API key) ``` Node.js **20.17.0+** is required and enforced in CI. | Command | Purpose | | ------------------ | -------------------------------------------------- | | `npm run dev` | Mock server with streaming and ChatKit simulations | | `npm run dev:ai` | AI-powered server (OpenAI/Anthropic/Gemini) | | `npm run build` | Produces `public/chat-widget.bundle.js` and CSS | | `npm test` | Jest suite (jsdom + Testing Library) | | `npm run validate` | Format check → lint → test, as CI runs it | Start against the mock server. It exercises streaming, tool visualization, and citations without spending API credits or needing a working backend. ## 4. Show commerce records The widget header carries tabs for **Chat**, **Orders**, **Products**, **Subscriptions**, **Returns**, and **Exchanges**, driven by the `records` prop: ```js theme={null} window.ChatWidget.mount({ container: document.getElementById('chat'), props: { records: { products: [{ id: 'SKU-44819', name: 'Luna Smart Bottle', status: 'In Stock', price: '$89.00', stock: 182, rating: 4.8, badges: ['Top seller'], }], orders: [{ id: 'ORD-9001', status: 'Processing', amount: '$199.00', customer: 'Jordan Lee', summary: 'Home gym starter kit', metrics: [{ label: 'Items', value: '4' }], nextAction: 'Confirm inventory with warehouse', }], returns: [], subscriptions: [], exchanges: [], }, }, }); ``` Each category renders status badges, metrics, and summary cards automatically. Orders, returns, exchanges, and subscriptions have a **List** mode; products support **List** and **Carousel**. Omit `records` entirely and sample data is shown, so the UI never looks empty in a demo. Make sure you pass real data before going to production — otherwise customers see the samples. ## Feature flags | Prop | Effect | | ------------------------- | ------------------------------------------------- | | `enableStreaming` | Token-by-token streaming responses | | `enableToolVisualization` | Shows tool calls as the agent makes them | | `enableSourceCitations` | Renders source citations under answers | | `enableRichMessages` | Markdown, syntax highlighting, forms, and actions | Other capabilities: reactions, transcript export, programmable actions, responsive fullscreen with iPhone safe-area handling, and built-in locale packs with translation overrides and automatic RTL layout. ## Next steps Configuration, revenue capture v2, and deployment. The contract your `messageEndpoint` implements — request, reply, and the SSE event stream. Build the agent behind the chat. # Response MCP Servers Source: https://docs.stateset.com/stateset-response/response-mcp The remote agent-building MCP server at POST /api/mcp, and the local stdio analytics/reporting server. New to MCP? Follow [Manage Agents & Workflows with MCP](/guides/manage-platform-with-mcp) for connection checks and first-task prompts. ResponseCX ships two Model Context Protocol servers with different jobs. | Server | Transport | For | | ---------------------- | ---------------------------------- | ----------------------------------------------------------- | | **ResponseCX remote** | Streamable HTTP at `POST /api/mcp` | An external agent **building and tuning** ResponseCX agents | | **Response analytics** | Local stdio | Claude Desktop reading **analytics and reporting** data | *** ## Remote agent-building server Connect an external assistant to `https://response.stateset.com/api/mcp` using the deployment's OAuth flow, a scoped `rcx_` API key, or an enabled StateSet workspace-key connection. Workspace keys are verified against the onboarding service and require that integration to be configured on the Response deployment. ```http theme={null} POST /api/mcp Authorization: Bearer ``` Authentication establishes the organization and available scopes. Inspect `tools/list` for the toolset your connection exposes; do not rely on a fixed tool count. Tool scopes apply on calls as well as discovery. A workspace key may permit writes, so a read-only prompt is not a substitute for scoped credentials and host approvals. The server is **stateless**: each JSON-RPC POST builds a fresh server and transport, so there is no cross-request session state to leak between tenants. Tool output is capped at 60,000 characters and truncated with a marker beyond that. ### Tools **Discovery** — so an agent can orient before it writes anything: | Tool | Purpose | | --------------------- | ------------------------------------------------------------------------- | | `describe_workspace` | What exists in this workspace | | `describe_vocabulary` | The legal terms for rule conditions — stops the model guessing at grammar | **Building:** | Tool | Purpose | | ------------------------------------------------------------- | ----------------------------------------------------------------------------------- | | `provision_agent` | Build a complete agent — settings, rules, functions, knowledge — in one atomic call | | `build_support_agent` | Opinionated support-agent scaffold | | `create_agent` / `update_agent` / `get_agent` / `list_agents` | Agent lifecycle | | `configure_agent` / `get_agent_settings` | Model and behavior settings | | `add_rule` / `update_rule` / `list_rules` | Rules | | `add_function` / `update_function` / `list_functions` | Functions | | `add_knowledge` / `search_knowledge` | Knowledge base | **Verifying:** | Tool | Purpose | | ------------- | --------------------------------------------------- | | `test_agent` | Exercise the agent — closes the build → verify loop | | `audit_agent` | Review an agent's configuration | **Reading:** | Tool | Purpose | | --------------------------------- | -------------------------------------------------------------------------------------------------- | | `list_responses` / `get_response` | Response history | | `analytics_summary` | Aggregate analytics | | `agent_analytics` | Per-agent volume and human-handled rate — finds *which* agent needs attention | | `list_conversations` | Recent threads with channel, status, escalation flag and rating | | `get_conversation` | One conversation in full, transcript in order — what the customer asked and how the agent answered | **Grounding it in your stack** — an agent grounded in nothing answers from nothing: | Tool | Purpose | | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | | `describe_integrations` | Whether Shopify and Gorgias are connected, with identifiers but never tokens | | `connect_shopify` | Store a custom-app Admin API token, verified against the store live before it is saved | | `connect_gorgias` | Same for a Gorgias domain, agent email and REST key | | `import_shopify_knowledge` | Pull products, collections and policies into the knowledge base; near-duplicates update in place, so re-running refreshes rather than duplicates | Follow [Your First Agent Evaluation](/guides/first-agent-evaluation) for exact tool inputs, case read-back, grading outcomes, and partial-suite handling. **The eval loop** — how a review becomes a durable fix rather than a note: | Tool | Purpose | | ------------------------- | --------------------------------------------------------------------------------------------------------------------- | | `create_eval` | Record what the agent *should* have said against a real `response_id`; becomes a regression case and fine-tuning data | | `run_eval` | Replay one eval and grade on substance, not wording. `passed` is `null`, never `false`, when grading could not run | | `run_evals` | Run the suite for a pass/fail count. Each costs a real generation, so runs are sequential and capped at 20 | | `list_evals` / `get_eval` | What has already been captured — check before writing a new one | **Maintaining what you built:** | Tool | Purpose | | --------------------------- | ------------------------------------------------------------------------------------------------------------------ | | `get_rule` / `get_function` | Read one by id before patching, so an update starts from current state | | `test_function` | Probe a tool: reachable, credentials still valid, answers in time. A broken tool fails *silently* mid-conversation | | `update_knowledge` | Correct a document in place — adding a correction instead leaves the agent two contradictory sources | | `delete_knowledge` | Remove an obsolete document. The only delete in the API, and not reversible | | `list_changes` | The configuration change log: what changed, when, by whom — including changes made through this server | | `switch_organization` | OAuth connections only: change which organization subsequent calls act on | ### Resources Re-readable context a client can attach without spending a tool call — notably `vocabulary` and `workspace`. ### The build → verify loop The intended flow for an agent building an agent: 1. `describe_workspace` and `describe_vocabulary` to orient. 2. `provision_agent` to create the whole thing atomically. 3. `test_agent` to exercise it. 4. `audit_agent`, then `update_rule` / `update_function` to correct. Errors come back as safe, model-readable messages — internals are never leaked. *** ## Local analytics server A stdio MCP server that lets Claude Desktop call the Response analytics and reporting API through a small authenticated bridge. ### Run it ```bash theme={null} RESPONSE_API_BASE_URL=http://localhost:3000 \ RESPONSE_API_TOKEN= \ RESPONSE_ORG_ID= \ npm run mcp:response-api ``` Optional: * `RESPONSE_INTERNAL_SECRET` (or `INTERNAL_API_SECRET`) adds `x-internal-secret` for internal-only routes. * `RESPONSE_MCP_MAX_RESPONSE_CHARS` caps large tool responses. Default `60000`. ### Claude Desktop ```json theme={null} { "mcpServers": { "response-analytics": { "command": "node", "args": ["/path/to/scripts/response-api-mcp-server.mjs"], "env": { "RESPONSE_API_BASE_URL": "https://your-response-host", "RESPONSE_API_TOKEN": "", "RESPONSE_ORG_ID": "" } } } } ``` ### Tools | Tool | Returns | | ------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `response_analytics_overview` | The comprehensive reporting payload: response volume, ratings, channels, AI resolution rate, widget activity, commerce signals, question themes, support performance, funnel steps, response times | | `response_widget_analytics` | The widget/chat subset: conversations, messages, widget events, funnel, handoff/containment, response time, commerce activity | | `response_dashboard_stats` | Compact dashboard stats | | `response_responses_created` | Recently created responses, filterable by date range, channel, rating, function-call status, and search | | `response_response_analytics_export` | Response rows for analytics and export | | `response_response_count` | Total response count for the authenticated organization | | `response_chat_conversations` | Chat/widget conversations with manager reporting fields — latest customer/agent messages, escalation and review state, status, counts, linked response signals | | `response_chat_thread` | One conversation transcript with recent messages, counts, attachments, and response history | | `response_cx_reporting_bundle` | The core CX reporting dataset in one call | | `response_api_health` | Checks `/api/health` | | `response_reporting_api_request` | Operator/debug escape hatch for allowlisted `/api/...` routes | | `response_mcp_servers` | Available MCP integrations for the organization | ### CX reporting flow For a broad report, start with `response_cx_reporting_bundle`. For deeper evidence, use `response_responses_created` to pull filtered response rows and `response_chat_conversations` to find relevant threads. Use `response_chat_thread` when you need the actual transcript. The analytics server only accepts `/api/...` paths and never writes logs to stdout, so the MCP JSON-RPC channel stays clean. ## Related * [Response Public API (v1)](/stateset-response/response-public-api) * [MCP Integration Guide](/guides/mcp-integration-guide) * [All MCP servers](/mcp-servers) # Response Public API (v1) Source: https://docs.stateset.com/stateset-response/response-public-api The API-key-authenticated /api/v1 surface — agents, rules, functions, settings, knowledge, responses, and analytics. A public, API-key-authenticated REST surface for building and tuning ResponseCX agents programmatically. It is the same surface the [remote MCP server](/stateset-response/response-mcp) exposes as tools. ## Authentication Pass an `rcx_` API key as a Bearer token: ```bash theme={null} curl https://your-response-host/api/v1/agents \ -H "Authorization: Bearer $RESPONSE_API_KEY" ``` Identity, organization, and scopes are derived **strictly from the key** — every route is gated on scopes and rate-limited. Existing session auth still works and falls through the guard. ### Scopes | Scope | Grants | | ----------------- | ---------------------------------------------------- | | `agents:read` | Read agents, settings, rules, functions | | `agents:write` | Create and update agents, rules, functions, settings | | `responses:read` | Read responses | | `responses:write` | Write responses | | `knowledge:read` | Read and search knowledge | | `knowledge:write` | Add knowledge | | `analytics:read` | Read analytics summaries | ## Endpoints ### Agents | Method | Path | Purpose | | ------------- | ------------------------------- | ---------------------------------------------- | | `GET` | `/api/v1/agents` | List agents | | `POST` | `/api/v1/agents` | Create an agent | | `GET` | `/api/v1/agents/{id}` | Get one agent | | `PATCH` | `/api/v1/agents/{id}` | Update an agent | | `POST` | `/api/v1/agents/provision` | Build a complete agent in one atomic call | | `GET`/`PATCH` | `/api/v1/agents/{id}/settings` | Read and configure model and behavior settings | | `GET`/`POST` | `/api/v1/agents/{id}/rules` | Read back and add an agent's rules | | `GET`/`POST` | `/api/v1/agents/{id}/functions` | Read back and add an agent's functions | ### Rules and functions | Method | Path | Purpose | | ---------------- | ------------------------ | --------------------------- | | `PATCH`/`DELETE` | `/api/v1/rules/{id}` | Update or retire a rule | | `PATCH`/`DELETE` | `/api/v1/functions/{id}` | Update or retire a function | ### Knowledge | Method | Path | Purpose | | ------ | -------------------------- | --------------------- | | `POST` | `/api/v1/knowledge` | Add knowledge, scoped | | `POST` | `/api/v1/knowledge/search` | Semantic search | ### Responses and analytics | Method | Path | Purpose | | ------ | --------------------------- | ---------------------- | | `GET` | `/api/v1/responses` | List responses | | `GET` | `/api/v1/responses/{id}` | Get one response | | `GET` | `/api/v1/analytics/summary` | Analytics summary | | `GET` | `/api/v1/workspace` | Describe the workspace | ## Provisioning an agent in one call `POST /api/v1/agents/provision` creates an agent together with its settings, rules, functions, and knowledge atomically — either the whole agent exists or none of it does. This is the path to prefer when an external system or agent is building a ResponseCX agent from scratch, since it avoids leaving a half-configured agent behind on a failed step. ## Validation errors that teach Every route returns **per-field issues** rather than a single opaque message, so a caller — especially an automated one — can correct the request without guessing: ```json theme={null} { "error": "validation_failed", "issues": [ { "path": "settings.model", "message": "unknown model 'gpt-9'" }, { "path": "rules[0].condition", "message": "expected a vocabulary term" } ] } ``` ## Discovering the vocabulary Rules and functions are written against a controlled vocabulary. Rather than guessing at the grammar, call: * `GET /api/v1/workspace` — what exists in this workspace. * The `describe_vocabulary` tool or `vocabulary` resource on the [MCP server](/stateset-response/response-mcp) — the legal terms for rule conditions. ## Related * [Response MCP Server](/stateset-response/response-mcp) * [ResponseCX](/stateset-response/stateset-responsecx) # Autonomy dial Source: https://docs.stateset.com/stateset-response/responsecx-autonomy The one control for how far an agent acts on its own — draft, suggest or autopilot — and exactly what each level gates. Every agent has an autonomy level. It is the single user-owned control for how far the agent acts without a person, and it is read in three places: the automation engine, the tool-call gate, and the sandbox. | Level | The agent | Confirmation required for | | ----------- | -------------------------------------------------------------- | ------------------------------------------------------------ | | `draft` | Writes replies; a human sends them | **Every** non-read tool | | `suggest` | **Default.** Sends and closes at high confidence; runs lookups | Money-moving and irreversible tools | | `autopilot` | Acts without host-side confirmation | Nothing host-side — the in-sandbox policy gate still applies | ## What each level sets The level maps to an engine policy rather than being interpreted ad hoc: | Level | Auto-send | Auto-close | Minimum confidence | | ----------- | --------- | ---------- | ------------------ | | `draft` | no | off | — | | `suggest` | yes | on | 0.80 | | `autopilot` | yes | on | 0.60 | `autopilot` does not remove the confidence floor; it lowers it. An agent on autopilot still declines to close a conversation it is not reasonably sure about. ## Setting it Autonomy lives on the agent, in `metadata.autonomy`: ```bash theme={null} curl --request PUT "https://api.stateset.com/v1/agents/$AGENT_ID" \ --header "Authorization: Bearer $STATESET_API_KEY" \ --header "Content-Type: application/json" \ --data '{ "metadata": { "autonomy": "draft" } }' ``` An unrecognised value does not fail — it silently becomes `suggest`. So `"autonomy": "manual"`, `"off"` or a typo does not put the agent in draft; it puts the agent in the **middle** setting, sending replies and running lookups on its own. Read the value back after setting it, and treat "the dial did nothing" as evidence the value was rejected rather than applied. Read it back to confirm it took: ```bash theme={null} curl "https://api.stateset.com/v1/agents/$AGENT_ID" \ --header "Authorization: Bearer $STATESET_API_KEY" ``` ```json Response theme={null} { "id": "agent_8Kx2mN4pQr", "agent_name": "Alex", "metadata": { "autonomy": "draft" } } ``` ## How a tool is judged money-moving Under `suggest`, the confirmation gate fires for money-moving and irreversible tools. A tool qualifies if it is on the platform's canonical destructive-tool list, **or** if its name classifies as financial, destructive or bulk. Read tools never qualify. Classification is by substring on the tool name: | Facet | Matches names containing | | ----------- | ------------------------------------------------------------------------------- | | read | `get_` `list_` `search_` `find_` `fetch_` `_info` `_details` `_stats` `_status` | | bulk | `batch_` `bulk_` `_all` `mass_` | | destructive | `delete_` `remove_` `cancel_` `close_` `deactivate_` `disable_` `terminate_` | | financial | `refund` `capture` `payment` `charge` `gift_card` `discount` `credit` | | write | `create_` `update_` `add_` `set_` `modify_` `change_` `apply_` `assign_` | **A custom tool whose name matches none of these is treated as not money-moving.** `void_invoice` contains no listed pattern, so under the default `suggest` level it runs with no confirmation — the gate never sees it as risky. If you register your own tools, either name them so they classify (`cancel_invoice`, `refund_order`) or put the agent on `draft`, where every non-read tool is gated regardless of its name. On `autopilot` the host-side gate is off, but the sandbox's own policy gate still runs. Autopilot means "no confirmation prompts", not "no guardrails" — and if you want a proof that each call was authorized rather than a policy check you have to trust, put an [Agent Gate](/stateset-nsr-agent-gate) in front of the tool server. ## The approvals queue On `suggest`, a money-moving or irreversible action does not fail and does not wait in a chat thread — it lands in the **approvals queue** and the agent moves on. A human resolves it there. | | | | -------------------------------- | ------------------------------------------------------------------------------------------------------------------ | | `GET /api/approvals/list?limit=` | Pending approvals for the caller's organization, with a suggestion for each. Any org key; 120 requests per minute. | | `POST /api/approvals/resolve` | `{ "approvalId": "…", "decision": "approved" \| "denied" }`. **Admin only**; 60 per minute. | Approving is the moment the money moves: the tool call executes through the same MCP handlers the agent sandbox uses, with the same guardrails. Denying records the decision and the agent's next turn sees it. The queue is not a table of its own. It is a view over the append-only `audit_events` ledger, so an approval, its resolution, and who resolved it are one immutable record — which is what makes the queue auditable after the fact rather than merely convenient before it. ## Choosing a level Start on `draft` while you are still reading what the agent writes. Move to `suggest` once its replies are ones you would have sent — that is the level most teams stay on, and the money-moving gate is what makes it safe to. Reach for `autopilot` only where a wrong action is cheap to reverse, or where an Agent Gate is proving each call. ## Next steps The platform this control governs. Hard constraints, for behaviour no autonomy level should permit. Requiring a verified proof before a tool call runs. Where the engine policy this sets is published from. # ResponseCX Platform Source: https://docs.stateset.com/stateset-response/responsecx-platform The AI Workforce platform — architecture, outcome model, Workflow Studio, org provisioning, and API conventions. **The AI Workforce platform.** Outcome-priced AI agents that resolve customer work end to end across commerce and CX — cancel orders, process returns, update subscriptions, issue refunds — with audit trails and human escalation. The platform ships as a multi-tenant Next.js app hosted on GKE. **ResponseCX** is the customer-facing product surface; StateSet is the company and category brand. ## Architecture ``` Next.js 15 (React 18) + TailwindCSS 4 | |-- Auth: WorkOS (multi-tenant orgs, PKCE OAuth, sealed sessions) |-- Database: Hasura GraphQL (PostgreSQL) |-- Vector DB: Qdrant (Gemini Embedding 2, 3072-dim), per-org collections |-- AI: Claude, OpenAI, Groq, Gemini, xAI, mem0 |-- Workflow: Temporal (Rust orchestration engine) | Ingest → Classify → Context → Generate → Review → Dispatch |-- Billing: Stripe metered subscriptions + customer balance transactions |-- Email: Gmail / Outlook OAuth (inbound email processing) |-- Infra: Docker + GKE + nginx ingress + cert-manager ``` Node **22.x** is required and enforced on preinstall, predev, and prebuild. ```bash theme={null} npm install --legacy-peer-deps # mem0ai peers conflict with groq-sdk cp .env.example .env.local npm run dev # http://localhost:3000 ``` ## Primary surfaces | Route | What it does | | ------------------ | ----------------------------------------------------------------------------- | | `/` | Marketing landing with an interactive write-access proof widget | | `/onboarding` | 3-step self-serve onboarding; trial defaults to no-card | | `/setup` | 6-step product wizard. `?demo=1` auto-provisions a support agent | | `/chat-desk` | Operator chat queue — virtualized, 3-pane, TipTap composer | | `/workflow-studio` | Configure the 15-phase Temporal pipeline. `?view=graph` for the visual editor | | `/outcomes` | Outcome Accounting Dashboard — SKUs, revenue, dispute ledger | ## Outcome model The platform sells **outcomes**, not seats. Each outcome is a verified, end-to-end resolution priced against BPO labor. The canonical contract is the SKU catalog in `lib/outcomes/sku-catalog.js`. Two SKUs are live today — Triage Outcome ($0.50) and Resolved Customer Contact ($2.00) — with the rest in development. See [StateSet Billing](/stateset-billing) for the catalog. **High-Value Actions**, by default anything at or above \$100, require human approval before the engine executes them. This is configured per brand in Workflow Studio's Review Gate phase. Disputes are persisted in Stripe customer balance transactions, where a negative amount is a credit on the next invoice. Customers can flag an outcome within 14 days from the Outcome Accounting Dashboard. ## Workflow Studio The Studio configures a **15-phase Temporal pipeline per brand**. New brands can start from a template that pre-fills 14+ phases: | Template | Motion | | --------------------------------------------------------------------------- | ------------------------------------- | | `support_control_tower` | Omnichannel CX with human-in-the-loop | | `ecommerce` / `subscription` | Shopify and Recharge motions | | `order_status`, `return_refund`, `subscription_pause`, `order_cancellation` | Workflow-narrow templates | The editor supports a List ↔ Graph view toggle, a `⌘K` phase jumper, `⌘S` save, per-phase `?` explainers, and a **pending-changes diff against the last saved config** — so you can see what you're about to change before saving. ## Org provisioning Creating an org runs all of this in one request: 1. WorkOS org created, creator set as `org:admin` 2. Hasura `organizations` and `access_tokens` rows created 3. Qdrant KB collection created (`{slug}_customer_support`) 4. Default AI agent created 5. Stripe customer created 6. Workflow engine tenant registered 7. Default workflow brand created, from a template if one is specified 8. All IDs written back to `access_tokens` ## API conventions There are roughly **600 API routes**, all behind `applyApiGuards`. Every route follows the same shape: ```js theme={null} import { z } from 'zod'; import { applyApiGuards } from '@/lib/api/guard'; import { respondError } from '@/lib/api/errors'; import { parseBody, respondValidationError } from '@/lib/api/validation'; const schema = z.object({ name: z.string().min(1).max(200) }); export default async function handler(req, res) { const guard = await applyApiGuards(req, res, { route: 'my-route', methods: ['POST'], auth: { requireOrg: true }, rateLimit: { keyPrefix: 'my-route', windowMs: 60_000, max: 30 }, maxBodySize: 100_000, }); if (!guard.ok) return; const parsed = parseBody(schema, req.body); if (!parsed.ok) return respondValidationError(res, parsed.error); try { return res.status(200).json({ ok: true }); } catch (error) { guard.logger.error('my-route.failed', { error: error?.message }); return respondError(res, 500, 'Operation failed'); } } ``` The guard provides rate limiting, method validation, WorkOS auth, body-size limits, request timing, a structured logger, and request-id propagation. **Auth options:** `auth: true` (user required), `auth: { requireOrg: true }`, `auth: { requireAdmin: true }` (implies `requireOrg`), and `auth: { allowInternal: true }`. ### Security rules These are enforced project-wide: * **Never return `error.message` to clients.** Use `respondError(res, status, 'Safe message')` and log the detail server-side. * **Validate external URLs** with `assertSafeUrl()` from `lib/security/ssrf.js`, or pass `ssrfCheck: true` to `fetchWithTimeout`. * **All MCP tool schemas need bounded `.max()` on arrays.** * **Admin-only routes use `auth: { requireAdmin: true }`** — the guard enforces it, so the handler must not re-check. ## Deployment Auto-deploys on push to `master`. The workflow builds via Cloud Build, runs `kubectl set image`, watches the rollout, smoke-checks `/api/health`, then **commits the new tag back to the deployment manifest** so git tracks live state. Required GitHub secrets: `GCP_SA_KEY`, `GCP_PROJECT_ID`, `GKE_CLUSTER`, `GKE_REGION`, `GKE_NAMESPACE`, `GKE_DEPLOYMENT`, `GAR_IMAGE_PATH`. ## Related Build and tune agents programmatically. Agent-building and analytics over MCP. Embed the chat widget. Outcome SKUs, plans, and disputes. # Website Onboarding Source: https://docs.stateset.com/stateset-response/responsecx-website-onboarding Generate rules, attributes, and knowledge base entries automatically from your website content. Website onboarding bootstraps an agent from content you already have. Point it at your site and it generates rules, attributes, and knowledge base entries — so a new agent starts with your FAQs, product details, and tone rather than an empty configuration. ## Crawling modes Discovers and crawls pages starting from your homepage. Respects `robots.txt` and crawl limits. Default maximum 20 pages, configurable up to 100. Best for complete website analysis. Processes only the URLs you provide — no crawling or discovery. Best for targeted extraction from pages you already know matter. ## What it generates Choose any combination: | Output | Purpose | | ------------------ | --------------------------------------------- | | **Rules** | Automated response rules for common inquiries | | **Attributes** | Agent behavior and knowledge settings | | **Knowledge Base** | Training data for AI responses | ## AI enhancement | Mode | Behavior | | ------------ | ------------------------------------------------------------------------------------------------------------ | | **Standard** | Basic content extraction and analysis | | **Enhanced** | Deeper analysis of FAQs and common questions, product information, business details, and communication style | ## Running it 1. **Select your agent** — choose which agent receives the generated content. 2. **Choose a crawling mode** — full crawl or specific pages. 3. **Enter URLs** — your homepage for a full crawl, or each page individually. 4. **Configure** — maximum pages, what to generate, and whether to enable AI analysis. 5. **Process** — start and wait. Duration depends on page count, page complexity, and AI settings. **Always review generated content before activating rules.** Onboarding produces a starting point from whatever your site happens to say — it can pick up outdated policies or marketing copy that shouldn't become an automated response. A generated rule looks like this before you edit it — a starting point drawn from a returns page, not a finished policy: ```json theme={null} { "name": "Returns window", "source_url": "https://example.com/policy/returns", "condition": "intent == 'return_request'", "response": "You can return any unworn item within 30 days of delivery.", "status": "draft" } ``` Generated rules land as `draft` for a reason. A site that still advertises a 60-day window it no longer honours will produce a rule promising 60 days, and activating it turns a stale marketing page into a commitment your agent makes to every customer who asks. Read each rule against the policy you actually operate, not against the site. ## Getting good results * **Start small.** Run 10–20 pages first and check the output before scaling up. * **Target high-value pages** in specific-pages mode: FAQs, product catalogs, About and Contact, terms of service, and pricing. * **Enable AI analysis** when you want better FAQ extraction, product information parsing, and business context. ## Troubleshooting | Problem | Try | | ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------- | | **No content extracted** | The site may block scrapers. Switch to specific pages, and verify the URLs are publicly reachable. | | **Timeout errors** | Reduce the page count, switch to specific-pages mode, or check the site's response time. | | **Poor quality results** | Enable AI analysis, target high-value pages, and check the pages have real content structure rather than being JS-rendered shells. | ## How it works * Respects `robots.txt`. * Falls back to alternative scrapers for JavaScript-heavy sites. * Processes concurrently with rate limiting. * Handles a range of content types and page structures. ## Related * [ResponseCX Platform](/stateset-response/responsecx-platform) * [Workflow Studio](/stateset-response/responsecx-workflow-studio) * [Public API — knowledge endpoints](/stateset-response/response-public-api) # Workflow Studio Source: https://docs.stateset.com/stateset-response/responsecx-workflow-studio Configure the per-brand Temporal pipeline — templates, context sources, review gates, and escalation. Workflow Studio configures the **15-phase Temporal pipeline** that runs per brand. It is where you decide what the agent looks at, what it is allowed to do on its own, and when a human gets involved. ## The pipeline ``` Ingest → Classify → Context → Generate → Review → Dispatch ``` | Stage | What happens | | ------------ | ---------------------------------------------------- | | **Ingest** | The inbound message arrives from a channel | | **Classify** | Contact reason and response motion are determined | | **Context** | Data is gathered from the configured sources | | **Generate** | The response and any actions are produced | | **Review** | The review gate decides: dispatch, hold, or escalate | | **Dispatch** | The response and actions are executed | ## Templates New brands start from a template that pre-fills 14+ phases, rather than configuring the pipeline from scratch: | Template | Motion | | ----------------------- | ------------------------------------- | | `support_control_tower` | Omnichannel CX with human-in-the-loop | | `sales_concierge` | Sales-led conversations | | `voice_receptionist` | Voice-first reception | | `ecommerce` | Shopify commerce motion | | `subscription` | Recharge subscription motion | | `knowledge_base` | Knowledge-led answering | | `order_status` | Workflow-narrow: order status | | `return_refund` | Workflow-narrow: returns and refunds | | `subscription_pause` | Workflow-narrow: pause a subscription | | `order_cancellation` | Workflow-narrow: cancel an order | The workflow-narrow templates are useful when you want an agent that does exactly one job well, rather than a general assistant. What a template produces is an `AutomationConfig` — the same declarative contract the engine executes, so anything Studio can express is also editable as JSON and reviewable in a pull request: ```json theme={null} { "skip_rules": { "business_hours": { "timezone": "America/Los_Angeles" } }, "context_sources": ["shopify_customer", "knowledge_base"], "review_gate": { "min_confidence": 0.75, "escalation_always_review": true }, "dispatch": { "add_tags": ["ai-handled"], "set_status": "open" } } ``` Studio and the control plane can disagree about which version is live — Studio may show a draft the engine has not been given yet. Check the published version before concluding that an edit had no effect; a change that looks ignored is usually a change that was never published. ## Context sources The Context phase gathers from whichever sources you enable: | Source | Provides | | ------------------------ | ------------------------------ | | `response` | Prior Response history | | `knowledge_base` | The org's Qdrant KB collection | | `stateset_agent_context` | Agent-level context | | `gorgias_ticket` | The originating Gorgias ticket | | `shopify_customer` | Shopify customer record | | `shopify_orders` | Shopify order history | | `recharge_customer` | Recharge customer record | | `recharge_subscriptions` | Recharge subscriptions | | `loop_subscriptions` | Loop subscriptions | ### Gather strategy `context_gather_strategy` controls what happens when a source is slow or unavailable: | Strategy | Behavior | | --------------- | ----------------------------------------------------- | | `require_all` | Every configured source must return before generating | | `critical_only` | Only sources marked critical must return | | `best_effort` | Generate with whatever arrived | `best_effort` keeps the pipeline moving when an integration is down, but the agent will answer from partial context — it may not know about a recent order. For flows that act on money or orders, prefer `require_all` or `critical_only` and let the gate escalate rather than answering on incomplete data. ## Review gate The Review phase is the safety boundary between what the agent generated and what the customer actually receives. Configured actions include: | Action | Effect | | ------------------- | --------------------------------------- | | `auto_approve` | Dispatch without human review | | `flag_and_dispatch` | Dispatch, but flag for later inspection | | `tag_and_review` | Tag and route into review | | `block` | Do not dispatch | | `escalate` | Hand off to a human | | `reject` | Discard the generated response | `review_timeout_secs` bounds how long a decision can pend, and `escalation_always_review` forces review on escalation paths regardless of other rules. ### High-Value Actions By default any action valued at **\$100 or more** requires human approval before the engine executes it. This is configured per brand in the Review Gate phase. This threshold is the main lever between autonomy and control. Lower it while you build trust in a new brand's configuration, then raise it once the escalation and dispute rates look right. ## Editing The Studio editor supports: * **List ↔ Graph** view toggle (`?view=graph` for the visual editor) * **`⌘K`** phase jumper * **`⌘S`** save * Per-phase **`?` explainers** * A **pending-changes diff** against the last saved config Read the pending-changes diff before saving. Because a template pre-fills 14+ phases, it is easy to change more than you intended — the diff is the only place the full effect is visible. ## Rollout Workflow Studio is designed to be adopted in stages rather than switched on wholesale: 1. **Operator-led private beta** — operators drive, the pipeline shadows. 2. **Assisted onboarding** — configuration with support. 3. **Self-serve production** — the customer configures their own brands. An `operator_shadow` phase supports running the pipeline alongside a human without dispatching, so you can compare what the agent *would* have done against what the operator did. ## Related * [ResponseCX Platform](/stateset-response/responsecx-platform) * [ResponseCX Workflows](/stateset-response/stateset-responsecx-workflows) * [Billing & outcomes](/stateset-billing) ## Next steps Every AutomationConfig field, with the environment it runs in. The workflow a published config drives. Publishing and syncing configs across brands. Seeding a new brand's rules from its own site. # Response ChatGPT App Guide Source: https://docs.stateset.com/stateset-response/stateset-response-chatgpt-guide Step-by-step setup for ChatGPT Developer Mode. # Response ChatGPT App Runs the StateSet Response workspace **inside ChatGPT** — a React widget rendered by ChatGPT Apps, backed by a Node MCP server that exposes the `stateset-response-chat` tool. | Piece | Role | | ------------ | ----------------------------------------- | | React widget | The workspace UI, rendered in ChatGPT | | MCP server | Exposes `stateset-response-chat` over SSE | | Connector | Registered in ChatGPT Developer Mode | This guide builds the widget, runs the server, and connects it. ChatGPT reaches the MCP server over **SSE**, so the endpoint has to be publicly resolvable — a localhost URL won't work. Step 3 covers exposing it. ## 1) Build the widget assets ```bash theme={null} git clone https://github.com/stateset/stateset-response-chatgpt-app.git cd stateset-response-chatgpt-app pnpm install pnpm run build ``` The build emits `stateset-response.html/js/css` inside `assets/`. ## 2) Run the MCP server ```bash theme={null} cd server pnpm install STATESET_API_BASE_URL=https://api.stateset.com \ STATESET_API_KEY=sk-... \ pnpm start ``` Without credentials, the server uses sample data for local testing. ## 3) Expose the MCP server Use a tunnel to expose the SSE endpoint: ```bash theme={null} ngrok http 3030 ``` The endpoint becomes `/mcp`. Confirm the server is actually serving MCP before wiring ChatGPT to it — a connector that fails here gives no useful error: ```bash theme={null} # Must return an SSE stream, not JSON curl -sS -N -H 'Accept: text/event-stream' "$MCP_URL/mcp" | head -c 200 ``` An `event:` line means the server is ready. JSON or a `404` means the MCP route is not mounted — usually the server is running but on a different path or port than the one you exposed. ## 4) Add the connector in ChatGPT 1. Open ChatGPT → **Settings → Developer Mode → Connectors**. 2. Click **Add connector**, choose **MCP via SSE**. 3. Name it `Stateset Response`. 4. Paste `/mcp` as the endpoint. 5. Save and confirm the connector is online. ## 5) Test the app ```text theme={null} @Stateset Response open the customer support workspace for org 1234. ``` ## Troubleshooting * Re-run `pnpm run build` if assets are missing. * Verify the tunnel is reachable and SSE is stable. * Confirm API credentials when using live data. ## Related Documentation * App README ## Next steps Hosting this on managed Kubernetes instead of locally. The wider set of servers StateSet publishes. Requiring a proof before a tool call runs. The platform the widget talks to. # Set up ResponseCX Source: https://docs.stateset.com/stateset-response/stateset-responsecx Connect integrations, configure an agent, and get a first workflow resolving safely. The path from an empty workspace to an agent resolving real tickets. The ordering below is deliberate — each step de-risks the next. ## Prerequisites * A StateSet account with API access * ResponseCX enabled for your workspace * Credentials for the commerce and CX systems you want the agent to reach ## 1. Connect integrations The agent can only act on systems it can see. Connect your commerce platform (orders, inventory) and your CX tool (ticket ingestion) first. ```bash theme={null} curl "https://api.stateset.com/v1/tenants/$TENANT/integrations" \ -H "x-stateset-api-key: $SYNC_API_KEY" ``` See [Integrations](/api-reference/integrations) — 186 are available natively. Verify a read works before configuring anything else. If the agent can't retrieve an order, no amount of policy tuning will help — and the symptom looks like an agent problem rather than a credentials problem. ## 2. Configure the agent Set the model, and — more importantly — the boundaries: | Setting | Why it matters first | | ------------------------- | -------------------------------------------------------------------------- | | `allowed_intents` | An **allow-list**. Keep it narrow; the agent acts only on what you approve | | `agent_takeover_phrases` | Phrases that hand control to a human immediately | | `health_concern_keywords` | Forces escalation on medical or safety language | | `escalation_team_id` | Who receives escalations | | `temperature` | Low for anything quoting policy or amounts | Full field reference: [Settings](/api-reference/responsecx/agents-by-settings-get). ## 3. Start read-only Run a workflow that answers rather than acts. **WISMO** is the right first choice — high volume, and the resolution is information rather than a consequential action. See [Workflows](/stateset-response/stateset-responsecx-workflows). ## 4. Add write actions behind the gate Once reads look right, enable actions — with the [review gate](/stateset-response/responsecx-workflow-studio) in front of anything consequential. Refunds at or above the High-Value Action threshold (**\$100** by default) require human approval. Don't enable refunds and remove the review gate in the same change. If something goes wrong you won't know which caused it — and one of the two moves money. ## 5. Verify and watch Confirm the agent retrieves customer context and responds in a test conversation. Then watch real traffic before widening `allowed_intents`. | What to watch | Where | | ------------------------------ | ------------------------------------------------------------------------ | | Outcomes produced and disputed | [Outcome Accounting Dashboard](/stateset-billing) | | Escalation rate | Rising means the agent is correctly refusing, or the policy is too tight | | Approval rate at the gate | Falling means the agent is proposing things it shouldn't | A rising escalation rate isn't automatically bad. An agent that escalates instead of guessing is working — and triage is itself a billable outcome. Compare it against the *dispute* rate to tell caution apart from a policy that's too narrow. ## Troubleshooting | Symptom | First check | | ---------------------------------- | ------------------------------------------------------------ | | No context returned | Integration credentials and permissions | | Agent errors | Rate limits and policy configuration | | Agent stays silent | Skip rules — `skip_tags`, `skip_channels`, `allowed_intents` | | Action proposed but never executed | Awaiting review-gate approval, not failed | | Right answer, wrong tone | `temperature`, and the system prompt | ## Related * [ResponseCX Platform](/stateset-response/responsecx-platform) * [Workflows](/stateset-response/stateset-responsecx-workflows) * [Workflow Studio](/stateset-response/responsecx-workflow-studio) * [Public API](/stateset-response/response-public-api) # ResponseCX Workflows Source: https://docs.stateset.com/stateset-response/stateset-responsecx-workflows The workflows teams automate first — and which billable outcome each one produces. These are the flows teams automate first. Each one maps to a **billable outcome**, so the workflow and the unit you're paid for are the same thing — which is why "did it resolve?" has a definite answer rather than a judgement call. | Workflow | Outcome produced | | ------------------- | ------------------------- | | WISMO | WISMO Resolution | | Returns and refunds | Resolved Return / RMA | | Order changes | Resolved Customer Contact | | Escalation | Triage Outcome | See [Billing](/stateset-billing) for which are live today. ## WISMO — where is my order The highest-volume ticket type in most stores, and the easiest to resolve without a human. 1. Pull the latest fulfilment and carrier status. 2. Detect exceptions or delays. 3. Answer, or offer resolution options if the news is bad. Automate this first. It's high volume, low risk, and the resolution is almost always "here is the information" rather than an action with consequences — so it builds confidence in the agent before you point it at refunds. ## Returns and refunds 1. Validate eligibility against policy — window, condition, item. 2. Create the return authorisation. 3. Trigger the refund or exchange. 4. Notify the customer with return tracking. Step 3 is where money moves, so it belongs behind the [review gate](/stateset-response/responsecx-workflow-studio). Refunds at or above the High-Value Action threshold — **\$100 by default** — require human approval before execution. Also: approving a return doesn't restock anything. Receipt does. See the [returns workflow](/stateset-icommerce/stateset-icommerce-returns-workflow) for the state machine. ## Order changes 1. Verify order state and payment status. 2. Apply the update — address, quantity, substitution. 3. Recalculate totals and validate constraints. 4. Confirm with the customer. Step 1 matters more than it looks: a shipped order can't have its address changed, and attempting it produces a confusing failure rather than a clean refusal. Check state before promising anything. ## Escalation Escalation isn't a failure path — it's a **first-class outcome**, billed as a Triage Outcome, and the correct result when the agent shouldn't decide. 1. Detect a low-confidence outcome, a policy exception, or a [refused decision](/stateset-nsr-decisions). 2. Route to human review **with full context** — the conversation, the order, and what the agent was about to do. 3. Resume automation after approval. The design point: a refusal is the right answer, not a broken one. An agent that escalates rather than guessing is working correctly, and you're paid for the triage. That's what makes refuse-by-default commercially viable instead of a cost. ## What runs underneath These workflows execute as durable [Temporal workflows](/next-temporal/workflow) through a 15-phase pipeline you configure per brand in [Workflow Studio](/stateset-response/responsecx-workflow-studio). Because each step is a replayable activity, an escalation can wait hours or days for a human without holding a connection open. ## Configuring them | Concern | Where | | --------------------------------------- | ---------------------------------------------------------------- | | Which phases run, and skip rules | [Workflow Studio](/stateset-response/responsecx-workflow-studio) | | Model, escalation phrases, takeover | [Settings](/api-reference/responsecx/agents-by-settings-get) | | Named policy constraints | [Rules](/api-reference/responsecx/agents-by-rules-get) | | Authorisation for consequential actions | [Decision gate](/next-temporal/policy-engine) | ## Related * [ResponseCX Platform](/stateset-response/responsecx-platform) — architecture and outcome model * [Workflow Studio](/stateset-response/responsecx-workflow-studio) * [Billing](/stateset-billing) — outcome SKUs and disputes # StateSet ResponseCX CLI Source: https://docs.stateset.com/stateset-responsecx-cli AI-powered CLI for managing the StateSet ResponseCX platform. AI-powered CLI for managing the [StateSet ResponseCX](https://response.cx) platform. Chat with an AI agent that can manage your agents, rules, skills, knowledge base, channels, messages, and more — all from the terminal. Includes optional WhatsApp and Slack gateways for connecting your agent to messaging platforms. ## Install ```bash theme={null} npm install -g stateset-response-cli ``` Or clone and build locally: ```bash theme={null} git clone https://github.com/stateset/stateset-response-cli.git cd stateset-response-cli npm install npm run build ``` ## Quick Start ```bash theme={null} # Authenticate with your Stateset organization response auth login # Start an interactive chat session response chat ``` ## Authentication The CLI supports two authentication methods: **Browser / Device Code (recommended)** ```bash theme={null} response auth login ``` Follow the prompts to authenticate via your browser. The CLI will receive a scoped token automatically. **Manual Setup** During `response auth login`, you can provide your GraphQL endpoint and admin secret directly. Credentials are stored in `~/.stateset/config.json` with restricted file permissions (600). ### Multiple Organizations ```bash theme={null} # Switch between configured organizations response auth switch # View current auth status response auth status ``` `response auth login` stores a scoped token; the manual path stores your GraphQL admin secret. Both land in `~/.stateset/config.json` at mode 600, but an admin secret is not scoped and cannot be revoked individually — prefer the browser flow anywhere the machine is shared, and on CI use a scoped token from an environment variable rather than a config file. ## Usage ### Interactive Chat ```bash theme={null} response chat response chat --model haiku response chat --model opus ``` The agent understands natural language. Ask it to list your agents, create rules, search the knowledge base, etc. **Session commands:** | Command | Description | | ---------- | ---------------------------------- | | `/help` | Show available commands | | `/clear` | Reset conversation history | | `/history` | Show conversation turn count | | `/model` | Switch model (sonnet, haiku, opus) | | `exit` | End the session | Multi-line input is supported — end a line with `\` to continue on the next line. Press `Ctrl+C` to cancel the current request. ### WhatsApp Gateway Bridge incoming WhatsApp messages to your StateSet Response agent. ```bash theme={null} response-whatsapp ``` On first run, scan the QR code with WhatsApp (Settings > Linked Devices > Link a Device). Auth state is persisted in `~/.stateset/whatsapp-auth/`. **Options:** ``` --model Model to use (sonnet, haiku, opus) --allow Comma-separated allowlist of phone numbers --groups Allow messages from group chats --auth-dir WhatsApp auth credential directory --reset Clear stored auth and re-scan QR --verbose, -v Enable debug logging ``` **Examples:** ```bash theme={null} response-whatsapp --model haiku response-whatsapp --allow 14155551234,14155559999 response-whatsapp --reset ``` ### Slack Gateway Bridge Slack messages to your StateSet Response agent via Socket Mode. ```bash theme={null} response-slack ``` **Setup:** 1. Create a Slack app at [https://api.slack.com/apps](https://api.slack.com/apps) 2. Enable Socket Mode (Settings > Socket Mode) 3. Generate an app-level token (`xapp-...`) with `connections:write` scope 4. Add Bot Token Scopes: `chat:write`, `app_mentions:read`, `im:history`, `channels:history` 5. Install the app to your workspace 6. Set environment variables: ```bash theme={null} export SLACK_BOT_TOKEN=xoxb-... export SLACK_APP_TOKEN=xapp-... ``` **Behavior:** * In DMs: responds to all messages * In channels: responds when @mentioned or in threads the bot has participated in **Options:** ``` --model Model to use (sonnet, haiku, opus) --allow Comma-separated allowlist of Slack user IDs --verbose, -v Enable debug logging ``` ## Environment Variables | Variable | Required | Description | | --------------------------- | -------- | -------------------------------------------- | | `ANTHROPIC_API_KEY` | Yes | Anthropic API key for Claude | | `STATESET_INSTANCE_URL` | No | StateSet ResponseCX instance URL | | `STATESET_GRAPHQL_ENDPOINT` | No | GraphQL API endpoint | | `STATESET_KB_HOST` | No | Knowledge base (Qdrant) host URL | | `SLACK_BOT_TOKEN` | Slack | Bot User OAuth Token (`xoxb-...`) | | `SLACK_APP_TOKEN` | Slack | App-level token for Socket Mode (`xapp-...`) | | `OPENAI_API_KEY` | KB | OpenAI API key for knowledge base embeddings | ## Available Tools The AI agent has access to 80+ tools organized by resource type: ### Agents `list_agents` `get_agent` `create_agent` `update_agent` `delete_agent` `bootstrap_agent` `export_agent` ### Rules `list_rules` `get_agent_rules` `create_rule` `update_rule` `delete_rule` `import_rules` `bulk_update_rule_status` `bulk_assign_rules_to_agent` `bulk_delete_rules` ### Skills `list_skills` `get_agent_skills` `create_skill` `update_skill` `delete_skill` `import_skills` `bulk_update_skill_status` `bulk_delete_skills` ### Attributes `list_attributes` `create_attribute` `update_attribute` `delete_attribute` `import_attributes` ### Examples `list_examples` `create_example` `update_example` `delete_example` `import_examples` ### Evaluations `list_evals` `create_eval` `update_eval` `delete_eval` `export_evals_for_finetuning` ### Datasets `list_datasets` `get_dataset` `create_dataset` `update_dataset` `delete_dataset` `add_dataset_entry` `delete_dataset_entry` ### Functions `list_functions` `create_function` `update_function` `delete_function` `import_functions` ### Responses `list_responses` `get_response` `get_response_count` `bulk_update_response_ratings` `search_responses` ### Knowledge Base `kb_search` `kb_upsert` `kb_update` `kb_delete` `kb_get_collection_info` `kb_scroll` ### Channels `list_channels` `get_channel` `get_channel_with_messages` `create_channel` `update_channel` `delete_channel` `get_channel_count` ### Messages `list_messages` `get_message` `create_message` `update_message` `delete_message` `search_messages` `get_message_count` ### Settings `list_agent_settings` `get_agent_settings` `update_agent_settings` `get_channel_settings` ### Organizations `get_organization` `get_organization_overview` `update_organization` ## Architecture The CLI uses the [Model Context Protocol (MCP)](https://modelcontextprotocol.io) to expose platform tools to Claude. On startup, the CLI spawns an MCP server as a child process over stdio. Claude calls tools through this server, which executes GraphQL queries against the StateSet backend. ``` User <--> CLI (Anthropic SDK) <--> MCP Server <--> Stateset GraphQL API <--> Qdrant Vector DB ``` The WhatsApp and Slack gateways create per-user agent sessions with the same architecture. Sessions have a 30-minute TTL and are automatically cleaned up. ## Development ```bash theme={null} # Run in development mode (no build step) npm run dev # Build TypeScript npm run build # Run production build npm start ``` ## License [MIT](LICENSE) ## Next steps What the CLI manages. The agents, rules and knowledge behind these commands. Authoring the rules the CLI deploys. Testing a change before it reaches customers. # StateSet Agents Source: https://docs.stateset.com/stateset-rl-agents Reinforcement‑learning framework for multi‑turn conversational AI agents. StateSet Agents is a production‑oriented RL stack for training and serving LLM‑backed agents that improve through **multi‑turn interaction**. The library provides: * Async‑first **agent APIs** (`MultiTurnAgent`, `ToolAgent`) with Hugging Face and stub backends. * **Environments** for conversational and task‑oriented episodes. * **Trajectories** and value/advantage utilities tailored to dialogue. * Composable **reward functions** (heuristic, domain, multi‑objective, neural). * A family of **group‑based policy‑optimization trainers** (GRPO, GSPO, GEPO, DAPO, VAPO) plus PPO and RLAIF. * **Offline RL algorithms** for learning from logged conversations (BCQ, BEAR, CQL, IQL, Decision Transformer). * **Sim‑to‑Real transfer** for training in simulation and deploying to real users (domain randomization, system identification, progressive transfer). * **Continual learning + long‑term planning** utilities (replay/LwF/EWC, plan context injection). * Optional **performance layers** (vLLM generation, Rust acceleration, distributed training, HPO, FastAPI service). If you want a framework that treats conversations as first‑class RL episodes (rather than single turns), this is it. *** ## Why group‑based optimization? Traditional RLHF/PPO trains on one sampled response at a time. In long conversations this leads to high‑variance updates and brittle behavior.\ StateSet Agents implements **group‑relative methods**: * **GRPO (Group Relative Policy Optimization)**: sample a group of trajectories per prompt, compute advantages relative to the group baseline, then apply clipped policy‑gradient updates. * **GSPO (Group Sequence Policy Optimization)**: a more stable sequence‑level variant (Alibaba Qwen team) that avoids token‑level collapse on long outputs and MoE models. The result is steadier learning for dialogue tasks. *** ## Core concepts * **Agent**: wraps a causal LM and exposes `initialize()` and `generate_response()`. * `MultiTurnAgent` handles conversation history and state. * `ToolAgent` adds function/tool calling. * **Environment**: defines episode reset/step logic and optional reward hooks. * `ConversationEnvironment` ships with scenario‑driven multi‑turn conversations. * `TaskEnvironment` is for goal‑oriented tasks. * **Trajectory**: a multi‑turn record of turns, rewards, and metadata (`MultiTurnTrajectory`). * **Rewards**: `RewardFunction` subclasses and factories; combined via `CompositeReward` or multi‑objective reward models. * **Training**: trainers in `stateset_agents.training` implement GRPO‑family updates, GAE/value heads, KL regularization, LoRA support, and optional distributed/vLLM execution. *** ## Installation ### Core (lightweight, stub‑ready) ```bash theme={null} pip install stateset-agents ``` ### Training / real models ```bash theme={null} pip install "stateset-agents[training]" ``` ### Optional extras ```bash theme={null} pip install "stateset-agents[trl]" # TRL GRPO integration + bitsandbytes pip install "stateset-agents[vllm]" # vLLM generation backend pip install "stateset-agents[hpo]" # Optuna/Ray Tune HPO pip install "stateset-agents[api]" # FastAPI service pip install "stateset-agents[distributed]"# DeepSpeed / multi‑GPU helpers pip install "stateset-agents[full]" # Most extras in one go ``` *** ## Quick start ### 1) Stub hello world (no downloads) Runs without Torch/transformers and is ideal for CI or prototyping. ```python theme={null} import asyncio from stateset_agents import MultiTurnAgent from stateset_agents.core.agent import AgentConfig async def main(): agent = MultiTurnAgent(AgentConfig(model_name="stub://demo")) await agent.initialize() reply = await agent.generate_response([{"role": "user", "content": "Hi!"}]) print(reply) asyncio.run(main()) ``` ### 2) Chat with a real model ```python theme={null} import asyncio from stateset_agents import MultiTurnAgent from stateset_agents.core.agent import AgentConfig async def main(): agent = MultiTurnAgent( AgentConfig(model_name="gpt2", max_new_tokens=128, temperature=0.7) ) await agent.initialize() messages = [{"role": "user", "content": "What is GRPO?"}] print(await agent.generate_response(messages)) asyncio.run(main()) ``` *** ## Train a multi‑turn agent with GRPO The high‑level `train(...)` helper chooses single‑turn vs multi‑turn GRPO automatically. ```python theme={null} import asyncio from stateset_agents import ( MultiTurnAgent, ConversationEnvironment, CompositeReward, HelpfulnessReward, SafetyReward, train, ) from stateset_agents.core.agent import AgentConfig async def main(): # 1) Agent agent = MultiTurnAgent(AgentConfig(model_name="gpt2")) await agent.initialize() # 2) Environment scenarios = [ { "id": "refund", "topic": "refunds", "context": "User wants a refund for a delayed order.", "user_responses": [ "My order is late.", "I'd like a refund.", "Thanks for your help.", ], } ] env = ConversationEnvironment(scenarios=scenarios, max_turns=6) # 3) Reward reward_fn = CompositeReward( [HelpfulnessReward(weight=0.7), SafetyReward(weight=0.3)] ) # 4) Train trained_agent = await train( agent=agent, environment=env, reward_fn=reward_fn, num_episodes=50, profile="balanced", save_path="./outputs/refund_agent", ) # 5) Try the trained model resp = await trained_agent.generate_response( [{"role": "user", "content": "My order was delayed, what can you do?"}] ) print(resp) asyncio.run(main()) ``` More end‑to‑end scripts live in `examples/complete_grpo_training.py` and `examples/production_ready_customer_service.py`. *** ## Continual learning + long‑term planning (optional) Enable planning context and replay/LwF in the trainer with config overrides: ```python theme={null} agent = MultiTurnAgent( AgentConfig( model_name="gpt2", enable_planning=True, planning_config={"max_steps": 4}, ) ) trained_agent = await train( agent=agent, environment=env, reward_fn=reward_fn, num_episodes=50, # resume_from_checkpoint="./outputs/checkpoint-100", config_overrides={ "continual_strategy": "replay_lwf", "continual_kl_beta": 0.1, "replay_buffer_size": 500, "replay_ratio": 0.3, "replay_sampling": "balanced", "task_id_key": "task_id", "task_schedule": ["task_a", "task_b"], "task_switch_steps": 25, }, ) context = {"conversation_id": "demo-trip", "goal": "Plan a 4-day trip to Kyoto"} resp = await trained_agent.generate_response( [{"role": "user", "content": "Can you draft a plan?"}], context=context, ) followup = await trained_agent.generate_response( [{"role": "user", "content": "Great. What should we do next?"}], context={"conversation_id": "demo-trip", "plan_update": {"action": "advance"}}, ) # To update the plan goal explicitly: # context={"conversation_id": "demo-trip", "plan_goal": "Plan a 4-day trip to Osaka"} ``` *** ## Other training algorithms All algorithms are available under `stateset_agents.training` when training deps are installed: * **GSPO**: stable sequence‑level GRPO variant (`GSPOTrainer`, `GSPOConfig`, `train_with_gspo`) * **GEPO**: expectation‑based group optimization for heterogeneous/distributed setups * **DAPO**: decoupled clip + dynamic sampling for reasoning‑heavy tasks * **VAPO**: value‑augmented group optimization (strong for math/reasoning) * **PPO baseline**: standard PPO trainer for comparison * **RLAIF**: RL from AI feedback via judge/reward models Minimal GSPO sketch: ```python theme={null} from stateset_agents.training import get_config_for_task, GSPOConfig, train_with_gspo from stateset_agents.rewards.multi_objective_reward import create_customer_service_reward base_cfg = get_config_for_task("customer_service", model_name="gpt2") gspo_cfg = GSPOConfig.from_training_config(base_cfg, num_outer_iterations=5) trained_agent = await train_with_gspo( config=gspo_cfg, agent=agent, environment=env, reward_model=create_customer_service_reward(), ) ``` See `docs/GSPO_GUIDE.md`, `docs/ADVANCED_RL_ALGORITHMS.md`, and `examples/train_with_gspo.py` for full configs. *** ## Offline RL: Learn from logged conversations Train agents from historical conversation logs without online interaction. Useful when: * You have existing customer service transcripts * Online training is expensive or risky * You want to bootstrap before online fine‑tuning ### Available Algorithms | Algorithm | Best For | Key Innovation | | ------------------------ | --------------------- | ------------------------------- | | **BCQ** | Conservative learning | VAE‑constrained action space | | **BEAR** | Distribution matching | MMD kernel regularization | | **CQL** | Pessimistic Q‑values | Conservative Q‑function penalty | | **IQL** | Expectile regression | Implicit value learning | | **Decision Transformer** | Sequence modeling | Return‑conditioned generation | ### Quick Start ```python theme={null} from stateset_agents.data import ConversationDataset, ConversationDatasetConfig from stateset_agents.training import BCQTrainer, BCQConfig # Load historical conversations config = ConversationDatasetConfig(quality_threshold=0.7) dataset = ConversationDataset.from_jsonl("conversations.jsonl", config) # Train with BCQ bcq_config = BCQConfig( hidden_dim=256, latent_dim=64, num_epochs=100, ) trainer = BCQTrainer(bcq_config) await trainer.train(dataset) ``` ### Hybrid Offline + Online Training Combine offline pretraining with online GRPO fine‑tuning: ```python theme={null} from stateset_agents.training import OfflineGRPOTrainer, OfflineGRPOConfig config = OfflineGRPOConfig( offline_algorithm="cql", offline_pretrain_steps=1000, online_ratio=0.3, # 30% online, 70% offline ) trainer = OfflineGRPOTrainer(config) trained = await trainer.train(agent, env, reward_fn, offline_dataset=dataset) ``` See `docs/OFFLINE_RL_SIM_TO_REAL_GUIDE.md` for complete documentation. *** ## Sim‑to‑Real Transfer Train in simulation, deploy to real users. The framework provides: ### Domain Randomization Generate diverse training scenarios with randomized user personas: ```python theme={null} from stateset_agents.training import DomainRandomizer, DomainRandomizationConfig config = DomainRandomizationConfig( persona_variation=0.3, topic_variation=0.2, style_variation=0.2, ) randomizer = DomainRandomizer(config) # Randomize during training persona = randomizer.sample_persona() scenario = randomizer.sample_scenario(topic="returns") ``` ### Conversation Simulator Calibratable simulator with adjustable realism: ```python theme={null} from stateset_agents.environments import ConversationSimulator, ConversationSimulatorConfig simulator = ConversationSimulator(ConversationSimulatorConfig( base_model="gpt2", realism_level=0.8, )) # Calibrate to real data await simulator.calibrate(real_conversations) # Measure sim‑to‑real gap gap = simulator.compute_sim_real_gap(real_data, sim_data) ``` ### Progressive Transfer Gradually transition from simulation to real interactions: ```python theme={null} from stateset_agents.training import SimToRealTransfer, SimToRealConfig transfer = SimToRealTransfer(SimToRealConfig( transfer_schedule="cosine", # linear, exponential, step warmup_steps=100, total_steps=1000, )) # Get current sim/real mixing ratio sim_ratio = transfer.get_sim_ratio(current_step) ``` See `docs/OFFLINE_RL_SIM_TO_REAL_GUIDE.md` for complete documentation. *** ## Hyperparameter optimization (HPO) Install with `stateset-agents[hpo]`, then: ```python theme={null} from stateset_agents.training import TrainingConfig, TrainingProfile from stateset_agents.training.hpo import quick_hpo base_cfg = TrainingConfig.from_profile( TrainingProfile.BALANCED, num_episodes=100 ) summary = await quick_hpo( agent=agent, environment=env, reward_function=reward_fn, base_config=base_cfg, n_trials=30, ) print(summary.best_params) ``` See `docs/HPO_GUIDE.md` and `examples/hpo_training_example.py`. *** ## Custom rewards Use the decorator for quick experiments: ```python theme={null} from stateset_agents.core.reward import reward_function @reward_function(weight=0.5) async def politeness_reward(turns, context=None) -> float: return 1.0 if any("please" in t.content.lower() for t in turns) else 0.0 ``` Combine with built‑ins via `CompositeReward`. *** ## Custom environments Subclass `Environment` for task‑specific dynamics: ```python theme={null} from stateset_agents.core.environment import Environment, EnvironmentState from stateset_agents.core.trajectory import ConversationTurn class MyEnv(Environment): async def reset(self, scenario=None) -> EnvironmentState: ... async def step( self, state: EnvironmentState, action: ConversationTurn ): ... ``` *** ## Checkpoints * `train(..., save_path="...")` saves an agent checkpoint. * Load later: ```python theme={null} from stateset_agents.core.agent import load_agent_from_checkpoint agent = await load_agent_from_checkpoint("./outputs/refund_agent") ``` *** ## CLI The CLI is a thin wrapper around the Python API: ```bash theme={null} stateset-agents version stateset-agents doctor stateset-agents train --stub stateset-agents train --config ./config.yaml --dry-run false --save ./outputs/ckpt stateset-agents evaluate --checkpoint ./outputs/ckpt --message "Hello" stateset-agents serve --host 0.0.0.0 --port 8001 ``` For complex runs prefer the Python API and the examples folder. ### Starter presets `stateset-agents init` scaffolds a starter config. Model-specific presets are available via `--preset`: ```bash theme={null} stateset-agents init --preset kimi-k3 --task customer_service --starter-profile balanced ``` | Preset | Starter module | | -------------- | ------------------- | | `default` | Generic scaffold | | `qwen3-5-0-8b` | `qwen3_5_starter` | | `kimi-k2-6` | `kimi_k2_6_starter` | | `kimi-k3` | `kimi_k3_starter` | | `gemma-4-31b` | `gemma4_starter` | GLM starter paths (`glm5_1_starter`, `glm5_2_starter`) are exported from `stateset_agents.training` for direct use from the Python API. `--task` selects the task preset and `--starter-profile` the profile; both apply only to the model-specific presets, not to `default`. The Kimi-K3 starter ships against **provisional specs** — expect it to move as the model's final specification lands. ### Batch evaluation `evaluate` also runs in batch mode, for nightly or PR-blocking evaluation: ```bash theme={null} stateset-agents evaluate --checkpoint outputs/v1 \ --scenarios eval_set.jsonl \ --reward customer_support \ --output eval_report.md ``` ### Rewards `PartialCreditGSM8KReward` provides a denser training signal than binary correctness on math-style tasks. *** ## Examples and docs Good starting points: * `examples/hello_world.py` – stub mode walkthrough * `examples/quick_start.py` – basic agent + environment * `examples/complete_grpo_training.py` – end‑to‑end GRPO training * `examples/train_with_gspo.py` – GSPO + GSPO‑token training * `examples/train_with_trl_grpo.py` – Hugging Face TRL GRPO integration Key docs: * `docs/USAGE_GUIDE.md` * `docs/RL_FRAMEWORK_GUIDE.md` * `docs/GSPO_GUIDE.md` * `docs/OFFLINE_RL_SIM_TO_REAL_GUIDE.md` * `docs/HPO_GUIDE.md` * `docs/CLI_REFERENCE.md` * `docs/ARCHITECTURE.md` *** ## Related Projects * [stateset-nsr](https://github.com/stateset/stateset-nsr) - Neuro‑symbolic reasoning engine for explainable tools. * [stateset-api](https://github.com/stateset/stateset-api) - Commerce/operations API that agents can drive. * [stateset-sync-server](https://github.com/stateset/stateset-sync-server) - Multi‑tenant orchestration and integrations. * [core](https://github.com/stateset/core) - Cosmos SDK blockchain for on‑chain commerce. * Public API docs: [https://docs.stateset.com](https://docs.stateset.com) *** ## Contributing See `CONTRIBUTING.md`. Please run `pytest -q` and format with `black`/`isort` before opening a PR. *** ## License Business Source License 1.1. Non‑production use permitted until **2029‑09‑03**, then transitions to Apache 2.0. See `LICENSE`. # StateSet iCommerce Sandbox Engine Skill Source: https://docs.stateset.com/stateset-sandbox-skill Self-hosted Kubernetes sandbox for running AI agents with full code execution in isolated containers # StateSet Sandbox Self-hosted Kubernetes sandbox infrastructure for running AI agents and code execution workloads in isolated containers. REST and WebSocket APIs for creating sandboxes, streaming command output, and managing files. ## Skill Files | File | URL | | ------------------------ | ---------------------------------------------------- | | **SKILL.md** (this file) | `https://doc.stateset.com/stateset-sandbox-skill.md` | **Check for updates:** Re-fetch these files anytime to see new features! *** ## API Base URL | Service | Base URL | Purpose | | --------------- | ----------------------------------------- | ------------------------------------ | | **Sandbox API** | `https://api.sandbox.stateset.app/api/v1` | Sandbox management, execution, files | | **WebSocket** | `wss://api.sandbox.stateset.app/ws` | Real-time streaming | **Environment Variables:** ```bash theme={null} export STATESET_SANDBOX_API_KEY=sk_sandbox_xxx export STATESET_SANDBOX_URL=https://api.sandbox.stateset.app ``` *** This page is the skill's own reference — everything an agent is given for driving the sandbox. For a person setting the sandbox up, the [sandbox quickstart](/stateset-sandbox/stateset-sandbox-quickstart-guide) is the shorter path; come back here for a specific call. ## Architecture Overview ``` ┌─────────────────────────────────────────────────────────────┐ │ Your App / Agent │ └─────────────────────┬───────────────────────────────────────┘ │ HTTP / WebSocket ▼ ┌─────────────────────────────────────────────────────────────┐ │ Sandbox Controller │ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │ │ │ REST API │ │ WebSocket │ │ Warm Pod Pool │ │ │ └─────────────┘ └─────────────┘ └─────────────────────┘ │ └─────────────────────┬───────────────────────────────────────┘ │ Kubernetes API ▼ ┌─────────────────────────────────────────────────────────────┐ │ Sandbox Pods │ │ ┌─────────────────────────────────────────────────────┐ │ │ │ /workspace/ - Your code and files │ │ │ │ Node.js, Python, Go, Rust, Git, Docker CLI │ │ │ │ Isolated network, resource limits, timeouts │ │ │ └─────────────────────────────────────────────────────┘ │ └─────────────────────────────────────────────────────────────┘ ``` **Key Capabilities:** * Isolated execution per sandbox pod with resource limits * REST and WebSocket APIs for command execution and streaming * File read/write APIs for workspace workflows * Prebuilt runtime with Node.js, Python, Go, Rust, and CLI tools * Automatic cleanup with per-sandbox timeouts * Warm pool support for sub-100ms startup *** ## Agent Registration AI agents must register to receive an API key for authentication. ### Register a New Agent ```bash theme={null} curl -X POST https://api.sandbox.stateset.app/api/v1/register \ -H "Content-Type: application/json" \ -d '{ "first_name": "Commerce", "last_name": "Agent", "organization_name": "My AI Company", "email": "agent@example.com" }' ``` **Response:** ```json theme={null} { "success": true, "user": { "id": "user_xxx", "email": "agent@example.com" }, "organization": { "id": "org_xxx", "name": "My AI Company" }, "api_key": "sk_sandbox_xxx_xxxxxxxxxxxxxxxxxxxxxxxx", "message": "Registration successful. Store your API key securely." } ``` **Important:** Store the `api_key` securely. It is only returned once. ### Authentication All API requests require authentication: ```bash theme={null} curl https://api.sandbox.stateset.app/api/v1/sandboxes \ -H "Authorization: ApiKey YOUR_API_KEY" ``` Or with Bearer token (JWT): ```bash theme={null} curl https://api.sandbox.stateset.app/api/v1/sandboxes \ -H "Authorization: Bearer YOUR_JWT_TOKEN" ``` *** ## Quick Start ### 1. Create a Sandbox ```bash theme={null} curl -X POST https://api.sandbox.stateset.app/api/v1/sandbox/create \ -H "Authorization: ApiKey YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "cpus": "2", "memory": "2Gi", "timeout_seconds": 300 }' ``` **Response:** ```json theme={null} { "sandbox_id": "sandbox-a1b2c3d4", "org_id": "org_xxx", "status": "running", "pod_ip": "10.0.1.42", "created_at": "2026-01-30T10:00:00Z", "expires_at": "2026-01-30T10:05:00Z", "startup_metrics": { "total_ms": 87, "pod_creation_ms": 12, "pod_ready_ms": 75 } } ``` ### 2. Write Files ```bash theme={null} # Base64 encode your content CONTENT=$(echo 'console.log("Hello from sandbox!");' | base64) curl -X POST https://api.sandbox.stateset.app/api/v1/sandbox/SANDBOX_ID/files \ -H "Authorization: ApiKey YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d "{ \"files\": [{ \"path\": \"/workspace/hello.js\", \"content\": \"$CONTENT\" }] }" ``` ### 3. Execute Commands ```bash theme={null} curl -X POST https://api.sandbox.stateset.app/api/v1/sandbox/SANDBOX_ID/execute \ -H "Authorization: ApiKey YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{"command": "node /workspace/hello.js"}' ``` **Response:** ```json theme={null} { "exit_code": 0, "stdout": "Hello from sandbox!\n", "stderr": "" } ``` ### 4. Read Files ```bash theme={null} curl "https://api.sandbox.stateset.app/api/v1/sandbox/SANDBOX_ID/files?path=/workspace/output.txt" \ -H "Authorization: ApiKey YOUR_API_KEY" ``` **Response:** ```json theme={null} { "path": "/workspace/output.txt", "content": "SGVsbG8gV29ybGQh", "size": 12 } ``` ### 5. Stop Sandbox ```bash theme={null} curl -X POST https://api.sandbox.stateset.app/api/v1/sandbox/SANDBOX_ID/stop \ -H "Authorization: ApiKey YOUR_API_KEY" ``` *** ## TypeScript SDK ### Installation ```bash theme={null} npm install @stateset/sandbox-sdk ``` ### Basic Usage ```typescript theme={null} import { StateSetSandbox } from "@stateset/sandbox-sdk"; const sandbox = new StateSetSandbox({ baseUrl: "https://api.sandbox.stateset.app", authToken: "sk_sandbox_xxx", timeout: 120000, }); // Create sandbox const instance = await sandbox.create({ cpus: "2", memory: "4Gi", timeout_seconds: 600, }); // Write file await sandbox.writeFile( instance.sandbox_id, "/workspace/app.py", 'print("Hello from Python!")' ); // Execute command const result = await sandbox.execute(instance.sandbox_id, { command: "python3 /workspace/app.py", }); console.log(result.stdout); // "Hello from Python!" // Cleanup await sandbox.stop(instance.sandbox_id); ``` ### Streaming Execution ```typescript theme={null} await sandbox.executeStream( instance.sandbox_id, { command: "npm install", stream: true }, { onStdout: (data) => console.log("stdout:", data), onStderr: (data) => console.error("stderr:", data), onExit: (code) => console.log("exit code:", code), } ); ``` ### Extended SDK (Advanced Features) ```typescript theme={null} import { StateSetSandboxExtended } from "@stateset/sandbox-sdk"; const sdk = new StateSetSandboxExtended({ baseUrl: "https://api.sandbox.stateset.app", authToken: "sk_sandbox_xxx", }); // Create checkpoint const checkpoint = await sdk.createCheckpoint(sandboxId, { name: "baseline", include_paths: ["/workspace"], include_env: true, }); // Restore from checkpoint await sdk.restoreCheckpoint(sandboxId, checkpoint.id); // Upload artifact to cloud storage const artifact = await sdk.uploadArtifact(sandboxId, { path: "/workspace/report.pdf", remote_path: "reports/2026/report.pdf", }); // Start MCP server const mcpServer = await sdk.startMCPServer(sandboxId, { name: "postgres", command: "npx", args: ["@modelcontextprotocol/server-postgres"], env: { DATABASE_URL: "postgresql://..." }, }); ``` *** ## Python SDK ### Installation ```bash theme={null} pip install stateset-sandbox ``` ### Basic Usage ```python theme={null} from stateset_sandbox import StateSetSandbox client = StateSetSandbox( base_url="https://api.sandbox.stateset.app", auth_token="sk_sandbox_xxx", timeout=30000 ) # Context manager for automatic cleanup with client.create(cpus="2", memory="4Gi") as sandbox: # Write file client.write_file( sandbox.sandbox_id, "/workspace/script.py", "print('Hello from Python!')" ) # Execute result = client.execute(sandbox.sandbox_id, "python3 /workspace/script.py") print(result.stdout) # Sandbox automatically stopped on exit ``` ### Running Claude Code Agent ```python theme={null} sandbox = client.create(cpus="4", memory="8Gi", timeout_seconds=1800) result = client.execute(sandbox.sandbox_id, command=[ "claude", "-p", "Create a REST API with Express.js that has CRUD endpoints for users", "--allowedTools", "Write,Bash,Read,Edit" ]) print(result.stdout) client.stop(sandbox.sandbox_id) ``` *** ## WebSocket API Connect for real-time streaming: ```javascript theme={null} const ws = new WebSocket('wss://api.sandbox.stateset.app/ws?token=YOUR_JWT'); // Execute with streaming ws.send(JSON.stringify({ type: 'execute', sandboxId: 'sandbox-xxx', command: 'npm run build', stream: true })); ws.onmessage = (event) => { const msg = JSON.parse(event.data); switch (msg.type) { case 'stdout': console.log(msg.data); break; case 'stderr': console.error(msg.data); break; case 'exit': console.log('Exit code:', msg.code); break; case 'done': console.log('Complete'); break; } }; ``` *** ## API Reference ### Sandbox Lifecycle | Method | Endpoint | Description | | ------ | ---------------------------- | ------------------- | | POST | `/api/v1/sandbox/create` | Create new sandbox | | GET | `/api/v1/sandbox/:id` | Get sandbox details | | GET | `/api/v1/sandbox/:id/status` | Get sandbox status | | GET | `/api/v1/sandboxes` | List all sandboxes | | POST | `/api/v1/sandbox/:id/stop` | Stop sandbox | | DELETE | `/api/v1/sandbox/:id` | Delete sandbox | ### File Operations | Method | Endpoint | Description | | ------ | --------------------------------------------- | ---------------------- | | POST | `/api/v1/sandbox/:id/files` | Write files (base64) | | GET | `/api/v1/sandbox/:id/files?path=...` | Read file (base64) | | GET | `/api/v1/sandbox/:id/files/download?path=...` | Download file (binary) | ### Command Execution | Method | Endpoint | Description | | ------ | ----------------------------- | --------------- | | POST | `/api/v1/sandbox/:id/execute` | Execute command | **Execute Request:** ```json theme={null} { "command": "npm run build", "working_dir": "/workspace", "env": {"NODE_ENV": "production"}, "stream": false, "timeout": 30000 } ``` ### Checkpoints | Method | Endpoint | Description | | ------ | ----------------------------------------------- | -------------------- | | POST | `/api/v1/sandbox/:id/checkpoints` | Create checkpoint | | GET | `/api/v1/sandbox/:id/checkpoints` | List checkpoints | | POST | `/api/v1/sandbox/:id/checkpoints/:cpId/restore` | Restore checkpoint | | POST | `/api/v1/sandbox/:id/checkpoints/:cpId/clone` | Clone to new sandbox | | DELETE | `/api/v1/sandbox/:id/checkpoints/:cpId` | Delete checkpoint | ### Artifacts (Cloud Storage) | Method | Endpoint | Description | | ------ | ---------------------------------------- | ---------------------- | | POST | `/api/v1/sandbox/:id/artifacts/upload` | Upload to S3/GCS/Azure | | POST | `/api/v1/sandbox/:id/artifacts/download` | Download from storage | | GET | `/api/v1/artifacts` | List artifacts | | GET | `/api/v1/artifacts/:id/url` | Get pre-signed URL | | DELETE | `/api/v1/artifacts/:id` | Delete artifact | ### MCP Servers | Method | Endpoint | Description | | ------ | --------------------------------- | ---------------------- | | POST | `/api/v1/sandbox/:id/mcp/start` | Start MCP server | | POST | `/api/v1/sandbox/:id/mcp/stop` | Stop MCP server | | GET | `/api/v1/sandbox/:id/mcp/servers` | List running servers | | GET | `/api/v1/sandbox/:id/mcp/presets` | List available presets | ### Agent Sessions | Method | Endpoint | Description | | ------ | ----------------------------------- | -------------------- | | POST | `/api/v1/agent-sessions` | Create agent session | | GET | `/api/v1/agent-sessions/:id` | Get session details | | POST | `/api/v1/agent-sessions/:id/start` | Start session | | POST | `/api/v1/agent-sessions/:id/exec` | Execute in session | | POST | `/api/v1/agent-sessions/:id/pause` | Pause session | | POST | `/api/v1/agent-sessions/:id/resume` | Resume session | | POST | `/api/v1/agent-sessions/:id/stop` | Stop session | | GET | `/api/v1/agent-sessions/:id/events` | Get session events | ### API Keys | Method | Endpoint | Description | | ------ | ----------------------------- | ------------------ | | POST | `/api/v1/api-keys` | Create new API key | | GET | `/api/v1/api-keys` | List API keys | | DELETE | `/api/v1/api-keys/:id` | Revoke API key | | POST | `/api/v1/api-keys/:id/rotate` | Rotate API key | ### Webhooks | Method | Endpoint | Description | | ------ | ---------------------- | ---------------- | | POST | `/api/v1/webhooks` | Register webhook | | GET | `/api/v1/webhooks` | List webhooks | | PUT | `/api/v1/webhooks/:id` | Update webhook | | DELETE | `/api/v1/webhooks/:id` | Delete webhook | ### Tunnels (Port Forwarding) | Method | Endpoint | Description | | ------ | --------------------------------------- | ------------- | | POST | `/api/v1/sandbox/:id/tunnels` | Create tunnel | | GET | `/api/v1/sandbox/:id/tunnels` | List tunnels | | DELETE | `/api/v1/sandbox/:id/tunnels/:tunnelId` | Delete tunnel | *** ## Create Sandbox Options | Field | Type | Default | Description | | ----------------- | ------ | ------------- | ------------------------------------------- | | `cpus` | string | `"2"` | CPU limit ("1", "2", "500m") | | `memory` | string | `"2Gi"` | Memory limit ("1Gi", "2Gi", "512Mi") | | `timeout_seconds` | number | `600` | Sandbox lifetime (60-86400) | | `env` | object | `{}` | Environment variables | | `isolation` | string | `"container"` | Isolation level: container, gvisor, microvm | | `gpu.count` | number | - | GPU count (1-8) | | `gpu.type` | string | - | GPU type ("nvidia.com/gpu") | | `gpu.memory_gb` | number | - | GPU memory (1-256) | *** ## Agent Sessions Long-running agent sessions with automatic sandbox rotation: ```typescript theme={null} // Create session with budget controls const session = await sdk.createAgentSession({ name: "code-review-agent", budget: { cost_cap_cents: 1000, iteration_limit: 50, duration_limit_seconds: 3600, }, sandbox: { cpus: "4", memory: "8Gi", timeout_seconds: 600, }, rotation: { pre_rotate_buffer_seconds: 60, include_process_state: true, }, }); // Start session await sdk.startAgentSession(session.id); // Execute commands (auto-rotates sandbox as needed) const result = await sdk.executeInAgentSession(session.id, { command: "claude -p 'Review the codebase' --allowedTools Write,Bash,Read", }); // Get session events const events = await sdk.getAgentSessionEvents(session.id); // Stop when done await sdk.stopAgentSession(session.id); ``` **Session States:** * `pending` - Created, not started * `running` - Active execution * `rotating` - Sandbox rotation in progress * `paused` - Temporarily suspended * `completed` - Successfully finished * `failed` - Error occurred * `cancelled` - Manually stopped *** ## MCP Server Integration Start Model Context Protocol servers inside sandboxes: ```typescript theme={null} // Start filesystem MCP server await sdk.startMCPServer(sandboxId, { name: "filesystem", command: "npx", args: ["-y", "@modelcontextprotocol/server-filesystem", "/workspace"], port: 3000, }); // Start PostgreSQL MCP server await sdk.startMCPServer(sandboxId, { name: "postgres", command: "npx", args: ["-y", "@modelcontextprotocol/server-postgres"], env: { DATABASE_URL: "postgresql://user:pass@host/db", }, port: 3001, }); // List running servers const servers = await sdk.listMCPServers(sandboxId); // Stop server await sdk.stopMCPServer(sandboxId, "postgres"); ``` **Built-in MCP Presets:** * `filesystem` - File operations * `github` - GitHub repository access * `postgres` - PostgreSQL queries * `slack` - Slack messaging * `brave-search` - Web search * `puppeteer` - Browser automation *** ## Checkpoints Save and restore sandbox state: ```typescript theme={null} // Create checkpoint const checkpoint = await sdk.createCheckpoint(sandboxId, { name: "after-setup", description: "Initial project setup complete", include_paths: ["/workspace"], exclude_paths: ["/workspace/node_modules"], include_env: true, }); // List checkpoints const checkpoints = await sdk.listCheckpoints(sandboxId); // Restore to checkpoint await sdk.restoreCheckpoint(sandboxId, checkpointId); // Clone checkpoint to new sandbox const newSandbox = await sdk.cloneCheckpoint(sandboxId, checkpointId); // Compare two checkpoints const diff = await sdk.compareCheckpoints(sandboxId, checkpointId1, checkpointId2); ``` *** ## Webhooks Subscribe to sandbox events: ```typescript theme={null} const webhook = await sdk.registerWebhook({ url: "https://your-app.com/webhooks/sandbox", events: [ "sandbox.created", "sandbox.ready", "sandbox.stopped", "command.completed", "checkpoint.created", ], secret: "whsec_your_webhook_secret", }); ``` **Webhook Events:** * `sandbox.created`, `sandbox.ready`, `sandbox.stopped`, `sandbox.error`, `sandbox.timeout` * `command.started`, `command.completed`, `command.failed` * `file.written`, `artifact.uploaded`, `artifact.deleted` * `checkpoint.created`, `checkpoint.restored`, `checkpoint.deleted` * `mcp.started`, `mcp.stopped` * `resource.warning`, `resource.critical` * `security.alert` *** ## Configuration ### Environment Variables | Variable | Default | Description | | ----------------------- | --------- | ----------------------------- | | `SANDBOX_IMAGE` | - | Docker image for sandbox pods | | `DEFAULT_CPUS` | `"2"` | Default CPU limit | | `DEFAULT_MEMORY` | `"2Gi"` | Default memory limit | | `DEFAULT_TIMEOUT` | `600` | Default timeout (seconds) | | `MAX_SANDBOXES_PER_ORG` | `5` | Max concurrent sandboxes | | `WARM_POOL_ENABLED` | `false` | Enable warm pod pool | | `WARM_POOL_SIZE` | `5` | Warm pool size | | `SANDBOX_EXEC_BACKEND` | `kubectl` | Exec backend: kubectl or k8s | *** ## Self-Hosted Deployment ### Prerequisites * Kubernetes cluster (1.24+) * kubectl configured * PostgreSQL database * Redis (optional, for warm pools) ### Deploy ```bash theme={null} # Clone repository git clone https://github.com/stateset/stateset-sandbox cd stateset-sandbox # Apply Kubernetes manifests kubectl apply -f k8s/namespace.yaml kubectl apply -f k8s/rbac.yaml kubectl apply -f k8s/configmap.yaml kubectl apply -f k8s/secret.yaml kubectl apply -f k8s/service.yaml kubectl apply -f k8s/deployment.yaml # Verify deployment kubectl get pods -n stateset-sandbox ``` ### Configure Secrets ```bash theme={null} kubectl create secret generic sandbox-secrets -n stateset-sandbox \ --from-literal=JWT_SECRET=your-jwt-secret \ --from-literal=DATABASE_URL=postgresql://user:pass@host/db \ --from-literal=REDIS_URL=redis://host:6379 ``` *** ## Security ### Container Isolation * **Non-root user:** Runs as UID 1001 * **Dropped capabilities:** ALL capabilities dropped * **Seccomp profile:** Enabled by default * **Read-only filesystem:** Only /workspace is writable * **Resource limits:** CPU, memory, ephemeral storage enforced ### Network Isolation * **Egress allowed:** HTTPS (443), DNS (53) * **Private ranges blocked:** 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 * **Network policies:** Restrictive by default ### Authentication * **API Keys:** Revocable, organization-scoped * **JWTs:** Signed with secret, expiring tokens * **Rate limiting:** Per-organization limits *** ## Preinstalled Tools Each sandbox includes: | Category | Tools | | -------------------- | ------------------------------------------- | | **Languages** | Node.js 20, Python 3.11, Go 1.21, Rust 1.75 | | **Package Managers** | npm, yarn, pnpm, pip, cargo | | **Version Control** | git, gh (GitHub CLI) | | **Build Tools** | make, cmake, gcc, g++ | | **Utilities** | curl, wget, jq, ripgrep, fd | | **Containers** | Docker CLI (socket mount optional) | *** ## Quick Reference ### CLI Commands (curl) ```bash theme={null} # Register curl -X POST https://api.sandbox.stateset.app/api/v1/register \ -H "Content-Type: application/json" \ -d '{"first_name":"...", "last_name":"...", "organization_name":"...", "email":"..."}' # Create sandbox curl -X POST https://api.sandbox.stateset.app/api/v1/sandbox/create \ -H "Authorization: ApiKey KEY" \ -H "Content-Type: application/json" \ -d '{"timeout_seconds": 300}' # Execute command curl -X POST https://api.sandbox.stateset.app/api/v1/sandbox/ID/execute \ -H "Authorization: ApiKey KEY" \ -H "Content-Type: application/json" \ -d '{"command": "echo hello"}' # Write file curl -X POST https://api.sandbox.stateset.app/api/v1/sandbox/ID/files \ -H "Authorization: ApiKey KEY" \ -H "Content-Type: application/json" \ -d '{"files": [{"path": "/workspace/file.txt", "content": "BASE64_CONTENT"}]}' # Read file curl "https://api.sandbox.stateset.app/api/v1/sandbox/ID/files?path=/workspace/file.txt" \ -H "Authorization: ApiKey KEY" # Stop sandbox curl -X POST https://api.sandbox.stateset.app/api/v1/sandbox/ID/stop \ -H "Authorization: ApiKey KEY" # List sandboxes curl https://api.sandbox.stateset.app/api/v1/sandboxes \ -H "Authorization: ApiKey KEY" ``` ### SDK Quick Reference ```typescript theme={null} // TypeScript const sandbox = new StateSetSandbox({ baseUrl, authToken }); await sandbox.create({ cpus, memory, timeout_seconds }); await sandbox.writeFile(id, path, content); await sandbox.execute(id, { command }); await sandbox.readFile(id, path); await sandbox.stop(id); ``` ```python theme={null} # Python client = StateSetSandbox(base_url, auth_token) sandbox = client.create(cpus="2", memory="4Gi") client.write_file(sandbox.sandbox_id, path, content) result = client.execute(sandbox.sandbox_id, command) content = client.read_file(sandbox.sandbox_id, path) client.stop(sandbox.sandbox_id) ``` *** ## Ideas to Try * Register and create your first sandbox * Run a Claude Code agent to generate and test code * Set up checkpoints to save and restore work * Use MCP servers for database or GitHub integration * Create agent sessions for long-running tasks * Set up webhooks for real-time notifications * Deploy self-hosted with warm pools for fast startup * Use GPU sandboxes for ML workloads *** *StateSet Sandbox v0.4.0* *January 2026* ## Next steps The human path to a running sandbox. The controller surface behind these calls. What an agent can and cannot reach from inside. Driving the sandbox over MCP instead. # Agent sessions Source: https://docs.stateset.com/stateset-sandbox/stateset-sandbox-agent-sessions A long-running agent loop with a cost cap, an iteration limit and sandbox rotation — state survives the sandbox it started in. A sandbox is ephemeral; an agent's work often is not. An **agent session** is a durable wrapper around a series of executions: it carries a budget, keeps context across sandbox rotations, and can be reattached to after a client disconnects. Use a plain [sandbox](/stateset-sandbox/stateset-sandbox-api-flow) when the work is one command. Use a session when an agent will run many commands over a long enough period that the sandbox underneath it may be replaced. Sessions are served from the sandbox controller at `https://api.sandbox.stateset.app/api/v1`, and authenticate with the sandbox scheme — `Authorization: ApiKey `, not `Bearer`. The key needs [`sandbox:write`](/stateset-sandbox/stateset-sandbox-security-guide#scopes) for anything that mutates a session, and `sandbox:read` to inspect one. ## Create a session The budget is the point. All three limits are optional, and a session with none of them set has nothing stopping it. ```bash theme={null} curl --request POST "https://api.sandbox.stateset.app/api/v1/agent/sessions" \ --header "Authorization: ApiKey $STATESET_SANDBOX_API_KEY" \ --header "Content-Type: application/json" \ --data '{ "name": "backlog-triage", "budget": { "costCapCents": 500, "iterationLimit": 200, "durationLimitSeconds": 7200 }, "rotation": { "preRotateBufferSeconds": 30, "includeProcessState": false }, "sandbox": { "cpus": 2, "memory": "4Gi", "isolation": "gvisor", "timeoutSeconds": 3600 }, "clientId": "worker-7" }' ``` | Field | Purpose | | --------------------------------- | ------------------------------------------------------------- | | `budget.costCapCents` | Spend ceiling for the whole session | | `budget.iterationLimit` | Maximum executions | | `budget.durationLimitSeconds` | Wall-clock ceiling | | `rotation.preRotateBufferSeconds` | Grace period before a sandbox is replaced, for cleanup | | `rotation.includeProcessState` | Carry process state across a rotation, not just files and env | | `sandbox` | The sandbox spec each rotation is created from | | `clientId` | Your own handle for reattaching later | `costCapCents` is a cap on the session, not a per-execution limit, and it is accounted after each execution rather than predicted before one. A single expensive command can cross the cap; the session stops afterwards. Size the cap for what you can afford to overshoot by one execution. ## Run work in it ```bash theme={null} curl --request POST "https://api.sandbox.stateset.app/api/v1/agent/sessions/$SESSION_ID/exec" \ --header "Authorization: ApiKey $STATESET_SANDBOX_API_KEY" \ --header "Content-Type: application/json" \ --data '{ "command": ["python3", "triage.py", "--batch", "7"], "workingDir": "/workspace", "timeoutMs": 120000 }' ``` `timeoutMs` is bounded to 1s–10min. Each execution is charged against the budget, and the response carries what remains. ## Rotation, and what survives it A session outlives the sandbox it started on. When the underlying sandbox reaches its timeout, the session enters `rotating`, provisions a fresh one, and carries context across: * the working directory * environment variables * any custom state you have written to the session context ```bash theme={null} # Anything the next sandbox must know goes in the session context curl --request POST "https://api.sandbox.stateset.app/api/v1/agent/sessions/$SESSION_ID/context" \ --header "Authorization: ApiKey $STATESET_SANDBOX_API_KEY" \ --header "Content-Type: application/json" \ --data '{ "cursor": "batch-7", "processed": 412 }' ``` Files written outside the working directory do **not** survive a rotation, and neither do running processes unless `rotation.includeProcessState` is set. Anything the next sandbox needs belongs in the session context or in [artifact storage](/stateset-sandbox/stateset-sandbox-api-reference) — not in `/tmp`. ## Surviving a disconnect A session is addressable after your client dies. Reattach with the `clientId` you supplied at creation: ```bash theme={null} curl --request POST "https://api.sandbox.stateset.app/api/v1/agent/sessions/reattach" \ --header "Authorization: ApiKey $STATESET_SANDBOX_API_KEY" \ --header "Content-Type: application/json" \ --data '{ "clientId": "worker-7" }' ``` Send `POST .../heartbeat` while you hold a session so the controller can tell a live client from an abandoned one. ## Lifecycle ```text theme={null} pending ──▶ running ──▶ rotating ──▶ running ──▶ completed │ ▲ ▲ │ └── resume ── paused ──────────────┘ └──────────────────────▶ failed | cancelled ``` | Endpoint | Purpose | | --------------------------------------------------- | ------------------------------------------- | | `POST /agent/sessions` | Create | | `GET /agent/sessions` | List, filterable by `status` | | `POST /agent/sessions/reattach` | Reattach by `clientId` | | `GET /agent/sessions/{id}` | Session detail | | `DELETE /agent/sessions/{id}` | Delete | | `POST /agent/sessions/{id}/start` | Start a pending session | | `POST /agent/sessions/{id}/exec` | Execute, charged against the budget | | `POST /agent/sessions/{id}/pause` · `/resume` | Hold and continue | | `POST /agent/sessions/{id}/approve` | Approve a step that needs sign-off | | `POST /agent/sessions/{id}/cancel` · `/stop` | End the session | | `POST /agent/sessions/{id}/heartbeat` | Keepalive | | `GET /agent/sessions/{id}/events` | Event history | | `POST /agent/sessions/{id}/context` | Update carried context | | `GET` · `POST /agent/sessions/{id}/tools` | List and register agent tools | | `POST` · `GET /agent/sessions/{id}/files` | Write and read files on the current sandbox | | `GET /agent/sessions/{id}/files/list` · `/download` | Browse and fetch | ## Next steps The durable supervisor that drives sessions like these across hours. The one-shot path, for work that fits in a single sandbox. Container, gVisor or MicroVM for the `isolation` field above. What a session can reach, and how to narrow it. # Sandbox API Flow Source: https://docs.stateset.com/stateset-sandbox/stateset-sandbox-api-flow Create, execute, manage files, and tear down a sandbox. This guide shows a typical end-to-end flow using the Sandbox API. ## 1) Register and get an API key ```bash theme={null} curl -X POST https://api.sandbox.stateset.app/api/v1/register \ -H "Content-Type: application/json" \ -d '{"first_name": "Your", "last_name": "Name", "organization_name": "Company", "email": "you@example.com"}' ``` ## 2) Create a sandbox ```bash theme={null} curl -X POST https://api.sandbox.stateset.app/api/v1/sandbox/create \ -H "Authorization: ApiKey YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{"timeout_seconds": 300}' ``` ## 3) Execute a command ```bash theme={null} curl -X POST https://api.sandbox.stateset.app/api/v1/sandbox/SANDBOX_ID/execute \ -H "Authorization: ApiKey YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{"command": "python3 -c \"print(2 ** 10)\""}' ``` `execute` is synchronous and returns when the command exits, so a long build holds the connection open for its whole duration. Bound it with the sandbox's `timeout_seconds` rather than your client's HTTP timeout — a client that gives up first leaves the command running and the sandbox billing. ## 4) Write a file ```bash theme={null} curl -X POST https://api.sandbox.stateset.app/api/v1/sandbox/SANDBOX_ID/files/write \ -H "Authorization: ApiKey YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{"path": "/workspace/hello.txt", "content": "SGVsbG8h", "encoding": "base64"}' ``` File content is **base64** on the way in and on the way out — `SGVsbG8h` above is `Hello!`. Sending raw text writes the literal characters of your string, not the bytes you meant. ## 5) Read a file ```bash theme={null} curl -X POST https://api.sandbox.stateset.app/api/v1/sandbox/SANDBOX_ID/files/read \ -H "Authorization: ApiKey YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{"path": "/workspace/hello.txt"}' ``` This flow creates one sandbox and stops it. When an agent needs to keep working past a single sandbox's lifetime, use an [agent session](/stateset-sandbox/stateset-sandbox-agent-sessions) instead — it carries a budget and rotates the sandbox underneath while keeping context. ## 6) Stop the sandbox ```bash theme={null} curl -X POST https://api.sandbox.stateset.app/api/v1/sandbox/SANDBOX_ID/stop \ -H "Authorization: ApiKey YOUR_API_KEY" ``` Step 6 is not optional. A sandbox runs, and bills, until it is stopped or its `timeout_seconds` expires — whichever comes first. Stop it in a `finally` block rather than at the end of the happy path, or a failed run leaves one up. ## Next steps The same flow through an SDK rather than curl. Every endpoint, with parameters and responses. Container, gVisor or MicroVM, and what each costs in start-up time. Before this runs anything you did not write. # Sandbox API Reference Source: https://docs.stateset.com/stateset-sandbox/stateset-sandbox-api-reference Full HTTP API reference for the StateSet Sandbox node controller. # API Reference (Node Controller) Base URL: `https://api.sandbox.stateset.app/api/v1` The Node controller is the default deployed by `k8s/deployment.yaml`. This API is served from `api.sandbox.stateset.app`, which is a different host from the commerce and agent APIs on `api.stateset.com`, and it takes a sandbox key rather than a platform key. A key that works against one will return `401` against the other. ## Authentication This API supports three auth modes: 1. API key (recommended for servers/CLI): ```text theme={null} Authorization: ApiKey sk_your_api_key ``` 2. Session JWT (recommended for same-origin apps): ```text theme={null} Authorization: Bearer eyJhbGciOi... ``` 3. Browser session cookie (recommended for dashboards): * `stateset_session` is an `httpOnly` cookie set by registration/login/invite accept/SSO callback. * For state-changing requests authenticated via the session cookie (POST/PUT/PATCH/DELETE), include `X-CSRF-Token: `. * After you have a session cookie, the controller returns an `x-csrf-token` response header on safe requests (GET/HEAD/OPTIONS) and sets a `__stateset_csrf` cookie. Use that token value for subsequent writes. * Requests authenticated via an `Authorization` header (API key or Bearer JWT) do not require CSRF. ## Errors Errors are returned as: ```json theme={null} { "error": { "code": "AUTH_REQUIRED", "message": "Authentication required" } } ``` ## Health & Monitoring ### GET /health No auth. ### GET /ready No auth. Returns 503 if not ready. ### GET /health/detailed No auth. ### GET /metrics Internal auth in production (`X-Internal-Token` or `Authorization: Bearer `). ## Authentication Endpoints ### POST /register Creates an organization + user + API key and sets a session cookie. Request: ```json theme={null} { "first_name": "Ada", "last_name": "Lovelace", "organization_name": "Example Co", "email": "ada@example.com", "password": "SecurePassword123!", "use_case": "AI agent development" } ``` Response: ```json theme={null} { "organization": { "id": "org_...", "name": "Example Co", "slug": "example-co", "plan": "hobby" }, "user": { "id": "usr_...", "email": "ada@example.com", "role": "owner" }, "api_key": { "key": "sk_test_...", "key_prefix": "sk_test_...", "name": "Default API Key" }, "token": "eyJhbGciOi..." } ``` ### POST /auth/login Authenticates and returns a session payload (also sets `stateset_session` cookie). Request: ```json theme={null} { "email": "ada@example.com", "password": "SecurePassword123!" } ``` Response: ```json theme={null} { "token": "eyJhbGciOi...", "expiresAt": "2030-01-01T00:00:00.000Z", "user": { "id": "usr_...", "email": "ada@example.com", "firstName": "Ada", "lastName": "Lovelace", "role": "owner", "emailVerified": false }, "organization": { "id": "org_...", "name": "Example Co", "slug": "example-co", "plan": "hobby" } } ``` ### POST /auth/logout Clears the session cookie. ### GET /auth/providers Returns which optional auth providers are configured. Response shape: ```json theme={null} { "workos": { "enabled": true, "redirect_uri": "https://sandbox.stateset.com/auth/callback" } } ``` ### GET /auth/session Returns the current session payload (same shape as `/auth/login`). Auth: cookie session (`stateset_session`) or `Authorization: Bearer `. ### GET /auth/workos/authorize Starts WorkOS AuthKit SSO. Redirects to WorkOS and sets short-lived PKCE cookies. Requires `WORKOS_API_KEY`, `WORKOS_CLIENT_ID`, `WORKOS_REDIRECT_URI`. Query params: * `return_to`: redirect destination after successful login (same-origin relative paths only; protocol-relative values like `//example.com` are rejected) * `organization_id`, `connection_id`, `domain_hint`, `login_hint`, `screen_hint`: WorkOS hints ### GET /auth/workos/callback WorkOS callback. Exchanges `code` for user identity, then sets a `stateset_session` cookie and redirects back. ### GET /auth/workos/link Returns the current WorkOS linking status for the authenticated organization. Auth: **cookie session** (admin/owner). ### POST /auth/workos/link Links a WorkOS Organization ID to the authenticated organization. Auth: **cookie session** (owner). Request: ```json theme={null} { "workos_organization_id": "workos_org_..." } ``` ### DELETE /auth/workos/link Unlinks WorkOS from the authenticated organization. Auth: **cookie session** (owner). When `AUDIT_LOGS_ENABLED=true`, WorkOS link/unlink operations emit audit actions: * `auth.sso.link` * `auth.sso.unlink` ## Sandboxes ### POST /sandbox/create Creates a new sandbox. Request: ```json theme={null} { "cpus": "2", "memory": "2Gi", "timeout_seconds": 600, "isolation": "wasm", "env": { "EXAMPLE": "true" }, "wasm_network": { "enabled": true, "allow_tcp": true, "allow_udp": false, "allow_ip_lookup": true } } ``` Notes: * `isolation` supports `container`, `gvisor`, `microvm`, and `wasm`. * `wasm_network` is valid only when `isolation` is `wasm`. Response: ```json theme={null} { "sandbox_id": "f3533033-a515-4f29-a440-74ab29021be5", "org_id": "org_...", "session_id": "1c3a...", "status": "running", "pod_ip": "10.34.3.135", "created_at": "2026-01-16T03:24:05.549Z", "expires_at": "2026-01-16T03:34:05.549Z", "startup_metrics": { "total_ms": 2634 } } ``` ### GET /sandboxes?limit=20\&offset=0 Lists sandboxes for the authenticated organization. ### GET /sandbox/:id Returns sandbox details. ### GET /sandbox/:id/status Returns status + expiry (lightweight). ### POST /sandbox/:id/stop Stops/deletes the sandbox. Alias: * `DELETE /sandbox/:id` ## Command Execution ### POST /sandbox/:id/execute Executes a command in the sandbox. Request: ```json theme={null} { "command": ["python3", "-c", "print(\"hello\")"], "working_dir": "/workspace", "env": { "DEBUG": "true" }, "timeout": 60000, "stream": false } ``` Response (non-streaming): ```json theme={null} { "exit_code": 0, "stdout": "hello\n", "stderr": "" } ``` Streaming (SSE): * Set `"stream": true` * Response `Content-Type: text/event-stream` * Events are emitted as `data: {...}\n\n` with `type` values: `stdout`, `stderr`, `exit`, `done`, `error` ## File Operations File contents are base64 encoded. ### POST /sandbox/:id/files Writes one or more files. Request: ```json theme={null} { "files": [ { "path": "/workspace/main.py", "content": "cHJpbnQoImhlbGxvIikK" } ] } ``` ### GET /sandbox/:id/files?path=/workspace/main.py Reads a file (JSON response containing base64 content). ### GET /sandbox/:id/files/download?path=/workspace/main.py Downloads a file as `application/octet-stream`. ## REPL Sessions Interactive Python REPL with persistent session state. Variables and imports survive across execute calls. Requires `REPL_ENABLED=true`. ### POST /sandbox/:id/repl/sessions Creates a new REPL session. Request: ```json theme={null} { "language": "python" } ``` `language` is optional and defaults to `python`. Response: ```json theme={null} { "session": { "id": "repl_01H...", "sandboxId": "f3533033-a515-4f29-a440-74ab29021be5", "language": "python", "kernelId": "k_abc123", "status": "idle" } } ``` ### GET /sandbox/:id/repl/sessions Lists REPL sessions for a sandbox. Response: ```json theme={null} { "sessions": [ { "id": "repl_01H...", "sandboxId": "f3533033-a515-4f29-a440-74ab29021be5", "language": "python", "kernelId": "k_abc123", "status": "idle" } ] } ``` ### POST /sandbox/:id/repl/sessions/:sessionId/execute Executes code in a REPL session. Variables and imports persist across calls. Request: ```json theme={null} { "code": "x = 42\nprint(x)" } ``` Response: ```json theme={null} { "executionCount": 1, "outputs": [ { "type": "stdout", "text": "42\n" }, { "type": "result", "data": { "text/plain": "42" } } ], "status": "ok", "durationMs": 15 } ``` ### POST /sandbox/:id/repl/sessions/:sessionId/stream Executes code with streaming output via SSE. Request: ```json theme={null} { "code": "for i in range(5): print(i)" } ``` Response `Content-Type: text/event-stream`. Events are emitted as output is produced: ```text theme={null} event: output data: {"type":"stdout","text":"0\n"} event: output data: {"type":"stdout","text":"1\n"} event: done data: {"executionCount":2,"status":"ok","durationMs":25} ``` ### POST /sandbox/:id/repl/sessions/:sessionId/interrupt Interrupts a running execution in the session. Response: ```json theme={null} { "status": "interrupted" } ``` ### DELETE /sandbox/:id/repl/sessions/:sessionId Destroys a REPL session and its kernel. Response: ```json theme={null} { "status": "destroyed" } ``` ### Session Status Values | Status | Description | | ---------- | ------------------------------- | | `starting` | Kernel is being provisioned | | `idle` | Ready for code execution | | `busy` | Currently executing code | | `dead` | Kernel crashed or was destroyed | ### Output Types | Type | Description | | -------- | ------------------------------------------------------------ | | `stdout` | Standard output text | | `stderr` | Standard error text | | `result` | Execution result (rich MIME data in `data` field) | | `error` | Execution error with `name`, `value`, and `traceback` fields | *** ## Templates ### GET /templates Lists templates (summaries). ### GET /templates/categories Lists categories. ### GET /templates/:id Returns a template including file contents. ### POST /sandbox/create-from-template Creates a sandbox based on a template. ## Secrets (BYOK) Secrets are organization-scoped and injected into sandbox pods as environment variables. ### POST /secrets Creates a secret. ### GET /secrets Lists secrets (values are never returned). ### PUT /secrets/:name Updates a secret. ### DELETE /secrets/:name Deletes a secret. ## API Keys ### GET /api-keys Lists API keys for the organization. ### POST /api-keys Creates a new API key (the plaintext key is only returned once). ### DELETE /api-keys/:id Revokes an API key. ### POST /api-keys/:id/regenerate Regenerates a key (returns a new plaintext key). ## GitHub Integrations These routes are available when GitHub integration routes are enabled. ### GET /github/installations Lists linked/pending GitHub App installations for the authenticated organization. ### POST /github/installations/:id/connect Links an installation to the authenticated organization. ### DELETE /github/installations/:id Removes a linked installation from the authenticated organization. ### GET /github/settings Returns GitHub automation settings for the organization. Includes `auto_pr_comment_on_sandbox` to control whether PR-triggered sandboxes post an automatic GitHub comment with dashboard/sandbox details. Includes `auto_stop_sandbox_on_pr_close` to control whether PR-triggered sandboxes are automatically stopped when the pull request is closed. ### PUT /github/settings Updates GitHub automation settings. ### GET /github/events Returns recent GitHub webhook processing events for the organization. ### GET /github/health?window\_hours=24 Returns integration health summary (installations, webhook success/failure counts, status, latest event/failure). * `window_hours` is optional and must be an integer between `1` and `168`. ## Internal Snapshot API (`/internal`) These endpoints require internal auth (`X-Internal-Token` or `Authorization: Bearer `) in production. ### GET /internal/snapshots List all snapshot profiles. Response: ```json theme={null} { "snapshots": [ { "profileKey": "2vcpu-2048mb", "memoryPath": "/var/lib/firecracker/snapshots/2vcpu-2048mb/memory.bin", "statePath": "/var/lib/firecracker/snapshots/2vcpu-2048mb/state.bin", "createdAt": "2026-02-20T12:00:00.000Z", "vcpuCount": 2, "memSizeMib": 2048, "snapshotType": "full", "memoryBackend": "file", "version": 1 } ], "total": 1, "enabled": true } ``` ### GET /internal/snapshots/status Snapshot service health and configuration. ```json theme={null} { "enabled": true, "snapshotCount": 1, "snapshotDir": "/var/lib/firecracker/snapshots", "memoryBackend": "file", "snapshotType": "full" } ``` ### POST /internal/snapshots Create a snapshot profile. Request example: ```json theme={null} { "profile": "2vcpu-2048mb", "vcpu_count": 2, "mem_size_mib": 2048, "kernel_path": "/var/lib/firecracker/kernel/vmlinux", "rootfs_path": "/var/lib/firecracker/rootfs/rootfs.ext4" } ``` ### GET /internal/snapshots/:profileKey Get metadata for a single profile. ### DELETE /internal/snapshots/:profileKey Delete a snapshot profile. ### POST /internal/snapshots/:profileKey/restore Restore a snapshot profile into a mock Firecracker runtime. ### POST /internal/snapshots/warm Warm cache, with optional body `{ "count": 3 }`. ### POST /internal/snapshots/cleanup Remove old snapshots by age and/or keep policy. Request example: ```json theme={null} { "keep_most_recent": 10, "max_age_seconds": 86400, "dry_run": true } ``` Response example: ```json theme={null} { "message": "Snapshot cleanup completed", "dry_run": true, "profiles_removed": 2, "removed_profiles": [ "1vcpu-512mb", "2vcpu-1024mb" ], "profiles_removed_by_age": 1, "profiles_removed_by_keep_limit": 1, "total_found": 4, "total_remaining": 2 } ``` ## Idempotency POST endpoints support safe retries via the `Idempotency-Key` header. When enabled (`IDEMPOTENCY_ENABLED=true`, default), the controller caches the response for each unique `(orgId, key, route)` tuple for 24 hours. ### Usage ```text theme={null} POST /api/v1/sandbox/create Authorization: ApiKey sk_... Idempotency-Key: my-unique-request-id-123 Content-Type: application/json ``` ### Behavior * **First request**: Executes normally. Response is stored and returned with `Idempotency-Status: stored`. * **Replay**: If the same `(orgId, key, route)` is seen again within 24h, the cached response is returned with `Idempotency-Status: replayed`. * **Body mismatch**: If a replay has a different request body hash, the server returns `409 Conflict` with error code `IDEMPOTENCY_KEY_REUSE_MISMATCH`. * **Scope**: POST requests only. The header is ignored on GET/PUT/PATCH/DELETE. * **TTL**: 24 hours. After expiry the key can be reused. ### Response Headers | Header | Values | Description | | -------------------- | -------------------- | ---------------------------------------------------------------- | | `Idempotency-Status` | `stored`, `replayed` | Whether the response was freshly computed or replayed from cache | *** ## Real-time Events (SSE) `GET /events/stream` provides a Server-Sent Events stream of sandbox lifecycle events for the authenticated organization. ### Authentication Because the browser `EventSource` API does not support custom headers, this endpoint accepts auth via multiple mechanisms: 1. `Authorization: Bearer ` header 2. `Authorization: ApiKey ` header 3. `?token=` query parameter 4. `?apiKey=` query parameter (requires `&orgId=`) 5. Session cookie (`stateset_session`) ### Event Format ```text theme={null} event: sandbox.created id: evt_01H... data: {"sandboxId":"sbx-abc","event":"sandbox.created","timestamp":"2026-02-27T12:00:00Z",...} event: sandbox.stopped id: evt_01H... data: {"sandboxId":"sbx-abc","event":"sandbox.stopped",...} ``` ### Event Types `sandbox.created`, `sandbox.stopped`, `sandbox.deleted`, `artifact.uploaded`, `checkpoint.created`, `checkpoint.restored`, `mcp.tool_called`, `security.violation`, `resources.limit_warning` ### Resumable Streams On reconnect, include the last received event ID to replay missed events: * `Last-Event-ID` request header (standard SSE reconnect) * `?lastEventId=` query parameter The server buffers recent events in memory and replays any events published after the given ID. ### Heartbeat A `:heartbeat` comment is sent every 20 seconds to keep the connection alive. *** ## Concurrency Budgets & Queue Mode Each organization has per-plan concurrency budgets for sandboxes and command executions. ### Per-Plan Budgets | Plan | Max Concurrent Sandboxes | Max Concurrent Executions | | ---------- | ------------------------ | ------------------------- | | Hobby | 3 | 5 | | Pro | 10 | 20 | | Team | 25 | 50 | | Enterprise | 100 | 200 | Limits are configurable via `ORG_BUDGET__MAX_CONCURRENT_SANDBOXES` and `ORG_BUDGET__MAX_CONCURRENT_EXECUTES` environment variables. ### Exceeding Limits When a request would exceed the budget: * **Without queue mode**: `429 Too Many Requests` with error code `CONCURRENT_LIMIT_EXCEEDED`. * **With queue mode** (`QUEUE_MODE_ENABLED=true`): `202 Accepted` with a queued request ID. ### Queued Response Format ```json theme={null} { "status": "queued", "requestId": "req_abc123", "position": 3, "queue_depth": 5, "estimated_wait_seconds": 15 } ``` ### Queued Response Headers | Header | Description | | --------------------------- | ------------------------------ | | `X-Stateset-Queue-Status` | `queued` | | `X-Stateset-Queue-Position` | Position in queue | | `Retry-After` | Seconds to wait before polling | ### Polling `GET /requests/:requestId` returns the current status of a queued request, including `position`, `queue_depth`, and `estimated_wait_seconds`. Once completed, `response_body` and `response_status` are included. ### Queue Health `GET /queue/health` returns per-org queue telemetry: `queue_depth`, `processing_count`, `oldest_queued_age_seconds`, `estimated_wait_seconds`, and rolling-window statistics (`completed_count`, `failed_count`, `expired_count`, `average_wait_ms`, `p95_wait_ms`). ### Error Codes | Code | Status | Description | | --------------------------- | ------ | ----------------------------------------- | | `CONCURRENT_LIMIT_EXCEEDED` | 429 | Budget exceeded and queue mode is off | | `QUEUE_DEPTH_EXCEEDED` | 429 | Queue is full (`QUEUE_MAX_DEPTH` reached) | *** ## Egress Policies Per-organization outbound network policies. Requires `EGRESS_POLICIES_ENABLED=true`. ### Policy Model ```json theme={null} { "id": "pol_abc123", "organizationId": "org_...", "policyName": "default", "allowAll": false, "allowedDomains": ["api.github.com", "*.npmjs.org"], "deniedDomains": ["evil.example.com"], "allowedPorts": [80, 443], "logAllRequests": true } ``` ### Domain Matching * **Exact match**: `api.github.com` matches only `api.github.com` * **Wildcard**: `*.npmjs.org` matches `registry.npmjs.org`, `www.npmjs.org`, etc. ### Enforcement Modes (`EGRESS_PROXY_MODE`) | Mode | Behavior | | --------- | -------------------------------------------------- | | `off` | No enforcement (default) | | `monitor` | Log violations but allow all traffic | | `enforce` | Block denied traffic, return `EGRESS_DENIED` (403) | ### Error Code | Code | Status | Description | | --------------- | ------ | -------------------------------- | | `EGRESS_DENIED` | 403 | Request blocked by egress policy | *** ## Pagination List endpoints support standard offset-based pagination. ### Query Parameters | Parameter | Type | Default | Max | Description | | --------- | ------- | ------- | ---- | ------------------------- | | `limit` | integer | 100 | 1000 | Number of items to return | | `offset` | integer | 0 | — | Number of items to skip | ### Response Format ```json theme={null} { "data": [...], "total": 42, "limit": 100, "offset": 0, "has_more": false } ``` Endpoints that support pagination: `/sandboxes`, `/artifacts`, `/checkpoints`, `/webhooks`, `/audit`, `/egress/audit`, `/requests`. *** ## Optional Features Some routes are only registered when corresponding services are enabled and the database is configured. * REPL sessions: `/sandbox/:id/repl/sessions`, execute, stream, interrupt, destroy (`REPL_ENABLED=true`) * Checkpoints: `/sandbox/:id/checkpoints`, `/checkpoints`, ... * Artifacts: `/sandbox/:id/artifacts/upload`, `/artifacts`, multipart uploads at `/artifacts/uploads/initiate`, `/artifacts/uploads/:upload_id/parts/:part_number`, `/artifacts/uploads/:upload_id/complete`, `/artifacts/uploads/:upload_id/abort` * Egress audit: `/egress/audit` (org query), `/internal/egress/audit` (proxy/internal ingest) * Egress authz adapter: `/internal/egress/authorize` (proxy authorization check) * Egress policy snapshot: `/internal/egress/policies` (proxy polling/sync) * Webhooks: `/webhooks`, ... * Tunnels: `/sandbox/:id/tunnels`, ... * Audit: `/audit`, `/sandbox/:id/audit/summary` Queue/backpressure notes: * When a create/execute request is queued (`202`), responses include: * `X-Stateset-Queue-Status: queued` * `X-Stateset-Queue-Position` * `Retry-After` * Queue poll (`GET /requests/:requestId`) includes `position`, `queue_depth`, and `estimated_wait_seconds`. * Queue health (`GET /queue/health`) returns per-org queue telemetry for dashboards: * `queue_depth`, `processing_count`, `oldest_queued_age_seconds` * `estimated_wait_seconds`, `seconds_per_queue_position` * rolling-window counts and latency stats (`completed_count`, `failed_count`, `expired_count`, `average_wait_ms`, `p95_wait_ms`) ## Next steps Create and run your first sandbox against these endpoints. The typed client over this API. The same, for Python. Health, metrics and runbooks for the controller serving this API. # Sandbox Architecture Source: https://docs.stateset.com/stateset-sandbox/stateset-sandbox-architecture The full architecture of the StateSet Sandbox platform. # StateSet Sandbox Architecture > **Scope of this document:** a component-level overview suitable for new > contributors and integrators. For the deep operational architecture — > warm-pool internals, GKE-specific infrastructure, data layer, and event > flows — see `docs/ARCHITECTURE.md` in the sandbox repository, which is the > canonical reference. For the Rust-controller-specific architecture see > [`stateset-sandbox-controller/README.md`](stateset-sandbox-controller/README.md). ## Overview StateSet Sandbox provides an isolated execution environment for running Claude Code CLI and AI agents. The system uses a controller-based architecture where a central API manages ephemeral sandbox pods on Kubernetes. ## System Components ### 1. Sandbox Controller The controller is a Node.js/Express API that manages the lifecycle of sandbox pods. * **Image**: `YOUR_REGISTRY/stateset/sandbox-controller:latest` * **Replicas**: 3 (high availability) * **Namespace**: `stateset-sandbox` * **Exposed at**: `sandbox.stateset.com` **Responsibilities:** * Authenticate API requests (JWT or API key) * Create/delete sandbox pods via Kubernetes API * Execute commands in running sandboxes * Stream output via SSE or WebSocket * Manage file uploads/downloads * Enforce resource limits and quotas ### 2. Sandbox Image The sandbox is a multi-language runtime environment with Claude Code pre-installed. * **Image**: `YOUR_REGISTRY/stateset/sandbox:latest` * **Not deployed directly** - pulled on-demand by the controller **Pre-installed Tools:** | Category | Tools | | ----------- | --------------------------------------------- | | AI/ML | Claude Code CLI, Anthropic SDK | | Node.js | Node 22, TypeScript, tsx, ts-node | | Python | Python 3.12, pip, anthropic, pydantic | | Go | Go 1.22 | | Rust | Latest stable | | MCP Servers | filesystem, github, puppeteer | | Dev Tools | Git, GitHub CLI, ESLint, Prettier, Playwright | *** ## CI/CD Pipeline Flow ``` ┌─────────────────────────────────────────────────────────────────────────────┐ │ GitLab Push │ │ (master branch) │ └─────────────────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────────────────┐ │ CI/CD Pipeline │ │ │ │ Step 1: Build Controller │ │ ──────────────────────── │ │ docker build --target controller \ │ │ -t YOUR_REGISTRY/sandbox-controller:${CI_COMMIT_SHA} │ │ │ │ Step 2: Build Sandbox │ │ ───────────────────── │ │ docker build --target sandbox \ │ │ -t YOUR_REGISTRY/sandbox:${CI_COMMIT_SHA} │ │ │ │ Step 3: Push Both Images to Artifact Registry │ │ │ │ Step 4: Deploy Controller Only │ │ ───────────────────────────── │ │ kubectl apply -k k8s/ (deploys sandbox-controller to stateset-sandbox) │ │ │ └─────────────────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────────────────┐ │ GKE Cluster (stateset-sandbox namespace) │ │ │ │ ┌─────────────────────────────────────────────────────────────────────┐ │ │ │ sandbox-controller (3 replicas) │ │ │ │ Exposed at: sandbox.stateset.com │ │ │ │ │ │ │ │ ConfigMap: │ │ │ │ ┌─────────────────────────────────────────────────────────────┐ │ │ │ │ │ SANDBOX_IMAGE: YOUR_REGISTRY/sandbox:latest│ │ │ │ │ │ DEFAULT_CPUS: "2" │ │ │ │ │ │ DEFAULT_MEMORY: "2Gi" │ │ │ │ │ │ NAMESPACE: "stateset-sandbox" │ │ │ │ │ └─────────────────────────────────────────────────────────────┘ │ │ │ └─────────────────────────────────────────────────────────────────────┘ │ │ │ │ Sandbox pods: (none yet - created on demand) │ │ │ └─────────────────────────────────────────────────────────────────────────────┘ ``` *** ## Runtime Flow ### Creating and Using a Sandbox ``` ┌─────────────────────────────────────────────────────────────────────────────┐ │ Your Application │ │ │ │ const client = new StateSetSandbox({ │ │ baseUrl: 'https://sandbox.stateset.com', │ │ authToken: 'sk-...' │ │ }); │ │ await client.create(); │ └─────────────────────────────────────────────────────────────────────────────┘ │ │ POST /api/sandbox/create ▼ ┌─────────────────────────────────────────────────────────────────────────────┐ │ sandbox-controller │ │ │ │ 1. Receives request │ │ 2. Reads SANDBOX_IMAGE from env (ConfigMap) │ │ 3. Calls K8s API to create pod: │ │ ┌──────────────────────────────────────────────────────────────────┐ │ │ │ apiVersion: v1 │ │ │ │ kind: Pod │ │ │ │ metadata: │ │ │ │ name: sandbox-org_xxx-abc123 │ │ │ │ namespace: stateset-sandbox │ │ │ │ spec: │ │ │ │ containers: │ │ │ │ - image: YOUR_REGISTRY/sandbox:latest │ │ │ │ env: │ │ │ │ - name: ANTHROPIC_API_KEY │ │ │ │ valueFrom: secretKeyRef... │ │ │ └──────────────────────────────────────────────────────────────────┘ │ │ 4. Waits for pod to be ready │ │ 5. Returns sandbox_id to caller │ └─────────────────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────────────────┐ │ Sandbox Pod (ephemeral) │ │ ──────────────────────── │ │ Image: YOUR_REGISTRY/sandbox:latest │ │ │ │ Pre-installed: │ │ • Claude Code CLI (@anthropic-ai/claude-code) │ │ • Node.js 22, Python 3.12, Go 1.22, Rust │ │ • MCP servers (filesystem, github, puppeteer) │ │ • Git, GitHub CLI │ │ │ │ Ready to execute commands via kubectl exec │ └─────────────────────────────────────────────────────────────────────────────┘ ``` ### Command Execution Flow ``` ┌──────────────┐ POST /api/sandbox/{id}/execute ┌────────────────────┐ │ │ ──────────────────────────────────────► │ │ │ Your App │ { "command": "claude -p '...'" } │ sandbox-controller│ │ │ ◄────────────────────────────────────── │ │ └──────────────┘ SSE stream: stdout/stderr/exit └────────────────────┘ │ │ kubectl exec ▼ ┌────────────────────┐ │ Sandbox Pod │ │ │ │ $ claude -p '...' │ │ (calls Anthropic) │ │ │ └────────────────────┘ ``` *** ## API Endpoints ### Sandbox Management | Method | Endpoint | Description | | ------ | ------------------------- | -------------------------------- | | POST | `/api/sandbox/create` | Create a new sandbox | | GET | `/api/sandbox/:id` | Get sandbox details | | GET | `/api/sandbox/:id/status` | Get sandbox status (lightweight) | | GET | `/api/sandboxes` | List all sandboxes | | POST | `/api/sandbox/:id/stop` | Stop and delete sandbox | | DELETE | `/api/sandbox/:id` | Delete sandbox | ### File Operations | Method | Endpoint | Description | | ------ | ------------------------------------------ | ---------------------- | | POST | `/api/sandbox/:id/files` | Write files to sandbox | | GET | `/api/sandbox/:id/files?path=...` | Read file (base64) | | GET | `/api/sandbox/:id/files/download?path=...` | Download file (binary) | ### Command Execution | Method | Endpoint | Description | | ------ | -------------------------- | ------------------------------------ | | POST | `/api/sandbox/:id/execute` | Execute command (streaming optional) | | WS | `/ws` | WebSocket for real-time streaming | *** ## Kubernetes Resources ### Namespace: `stateset-sandbox` | Resource | Name | Purpose | | ---------------- | --------------------------- | --------------------------------- | | Deployment | `sandbox-controller` | Controller API (3 replicas) | | Service | `sandbox-controller` | ClusterIP for internal access | | Ingress | `sandbox-controller` | External access via nginx | | ConfigMap | `sandbox-controller-config` | Controller configuration | | Secret | `sandbox-controller-auth` | JWT secret, API keys | | Secret | `anthropic-credentials` | Anthropic API key for sandboxes | | ServiceAccount | `sandbox-controller` | K8s API access for pod management | | Role/RoleBinding | `sandbox-controller` | RBAC for pod CRUD operations | | NetworkPolicy | `sandbox-controller` | Network isolation rules | | ResourceQuota | `sandbox-quota` | Limit total sandbox resources | *** ## Docker Build Targets The unified `Dockerfile` supports multiple build targets: ```bash theme={null} # Build controller image docker build --target controller \ -t YOUR_REGISTRY/stateset/sandbox-controller:latest . # Build sandbox image docker build --target sandbox \ -t YOUR_REGISTRY/stateset/sandbox:latest . ``` | Target | Base Image | Size | Purpose | | ------------ | ------------ | ------- | -------------------- | | `controller` | node:22-slim | \~200MB | API server | | `sandbox` | node:22-slim | \~2.5GB | Full dev environment | *** ## SDK Usage ### TypeScript SDK ```typescript theme={null} import { StateSetSandbox, AgentRunner } from '@stateset/sandbox-sdk'; // Initialize client const client = new StateSetSandbox({ baseUrl: 'https://sandbox.stateset.com', authToken: process.env.STATESET_API_KEY!, orgId: 'org_xxx' }); // Low-level: Create sandbox and execute commands const sandbox = await client.create(); const result = await client.execute(sandbox.sandbox_id, { command: 'claude -p "Write a hello world in Python"' }); console.log(result.stdout); await client.stop(sandbox.sandbox_id); // High-level: Use AgentRunner const runner = new AgentRunner({ sandbox: client, anthropicApiKey: process.env.ANTHROPIC_API_KEY! }); const { events } = await runner.runAndCleanup({ prompt: 'Create a REST API using FastAPI', systemPrompt: 'You are a senior backend developer', maxTurns: 10 }); ``` *** ## Security ### Sandbox Isolation * **Ephemeral pods**: Each sandbox is a separate pod, destroyed after use * **Non-root user**: Sandboxes run as UID 1000 (`sandbox` user) * **Resource limits**: CPU/memory limits enforced per sandbox * **Network policies**: Sandboxes have limited network access * **Read-only filesystem**: Optional, configurable per deployment * **gVisor runtime**: Optional sandboxing via `RUNTIME_CLASS` config ### Authentication * **API Keys**: `Authorization: ApiKey sk-...` * **JWT Tokens**: `Authorization: Bearer eyJ...` * **Org isolation**: Sandboxes are scoped to organizations *** ## Configuration ### Environment Variables (Controller) | Variable | Default | Description | | ----------------------- | ------------------------- | -------------------------------------- | | `SANDBOX_IMAGE` | `stateset/sandbox:latest` | Image for sandbox pods | | `DEFAULT_CPUS` | `2` | Default CPU limit | | `DEFAULT_MEMORY` | `2Gi` | Default memory limit | | `DEFAULT_TIMEOUT` | `600` | Sandbox timeout (seconds) | | `MAX_SANDBOXES_PER_ORG` | `5` | Max concurrent sandboxes per org | | `NAMESPACE` | `stateset-sandbox` | K8s namespace for pods | | `LOG_LEVEL` | `info` | Logging verbosity | | `RUNTIME_CLASS` | (unset) | Optional: `gvisor` for extra isolation | *** ## Directory Structure ``` stateset-sandbox/ ├── Dockerfile # Multi-target: controller & sandbox ├── ARCHITECTURE.md # This file ├── controller/ │ ├── src/ │ │ ├── index.ts # Express server entrypoint │ │ ├── routes.ts # API route handlers │ │ ├── sandbox-manager.ts # K8s pod management │ │ └── middleware/ │ │ └── auth.ts # JWT/API key authentication │ ├── package.json │ └── tsconfig.json ├── sdk/ │ └── src/ │ ├── client.ts # StateSetSandbox client │ ├── agent-runner.ts # High-level agent runner │ └── types.ts # TypeScript types ├── docker/ │ ├── entrypoint.sh # Sandbox container entrypoint │ └── health-check.sh # Sandbox health check └── k8s/ ├── deployment.yaml # Controller deployment ├── service.yaml # Controller service ├── ingress.yaml # External access ├── configmap.yaml # Controller config ├── secret.yaml # Auth credentials ├── rbac.yaml # ServiceAccount & roles ├── network-policy.yaml # Network isolation ├── resource-quota.yaml # Resource limits └── kustomization.yaml # Kustomize config ``` ## Next steps Which isolation runtime each component actually runs under. The controller surface these components expose. How the boundaries drawn here are enforced. Mapping these components onto a managed Kubernetes cluster. # Sandbox Deployment: AWS Source: https://docs.stateset.com/stateset-sandbox/stateset-sandbox-deployment-aws Deploying the StateSet Sandbox on AWS. # AWS Deployment Guide (EKS) Complete guide for deploying StateSet Sandbox on Amazon Elastic Kubernetes Service (EKS). ## Prerequisites * AWS CLI configured with appropriate permissions * kubectl installed * eksctl installed (recommended) * Helm 3.x installed * Domain name for API endpoint ## Quick Start (Kustomize Overlay) Use the AWS-specific overlay to apply the correct ingress/controller defaults: ```bash theme={null} kubectl apply -k k8s/overlays/aws # or ./scripts/deploy-k8s.sh aws # or (CLI) stateset-sandbox deploy aws stateset-sandbox deploy aws --preflight --wait --smoke ``` `stateset-sandbox` is the `@stateset/sandbox-cli` binary. It also installs a `stateset` alias, but that name collides with `@stateset/cli`, whose `stateset` has no `deploy` command, so use the long name. For smoke checks, export credentials before running deploy: ```bash theme={null} export STATESET_API_KEY="your_api_key" export STATESET_API_URL="https://api.sandbox.stateset.app/api/v1" ``` Before applying the overlay, provision the required controller auth secret: ```bash theme={null} read -rsp "Admin password: " ADMIN_PASSWORD; echo bash scripts/create-controller-secret.sh \ --admin-username admin \ --admin-password "$ADMIN_PASSWORD" unset ADMIN_PASSWORD ``` ## Architecture Overview ``` ┌─────────────────────────────────────────────────────────────────────────────────┐ │ AWS Cloud │ │ ┌───────────────────────────────────────────────────────────────────────────┐ │ │ │ VPC (10.0.0.0/16) │ │ │ │ ┌────────────────────┐ ┌────────────────────┐ ┌────────────────────┐ │ │ │ │ │ Public Subnet 1 │ │ Public Subnet 2 │ │ Public Subnet 3 │ │ │ │ │ │ 10.0.1.0/24 │ │ 10.0.2.0/24 │ │ 10.0.3.0/24 │ │ │ │ │ │ ┌──────────────┐ │ │ ┌──────────────┐ │ │ ┌──────────────┐ │ │ │ │ │ │ │ NAT Gateway │ │ │ │ NAT Gateway │ │ │ │ NAT Gateway │ │ │ │ │ │ │ └──────────────┘ │ │ └──────────────┘ │ │ └──────────────┘ │ │ │ │ │ └────────────────────┘ └────────────────────┘ └────────────────────┘ │ │ │ │ │ │ │ │ ┌────────────────────┐ ┌────────────────────┐ ┌────────────────────┐ │ │ │ │ │ Private Subnet 1 │ │ Private Subnet 2 │ │ Private Subnet 3 │ │ │ │ │ │ 10.0.11.0/24 │ │ 10.0.12.0/24 │ │ 10.0.13.0/24 │ │ │ │ │ │ ┌────────────────┐ │ │ ┌────────────────┐ │ │ ┌────────────────┐ │ │ │ │ │ │ │ EKS Node Group │ │ │ │ EKS Node Group │ │ │ │ EKS Node Group │ │ │ │ │ │ │ │ Controller │ │ │ │ Sandboxes │ │ │ │ Sandboxes │ │ │ │ │ │ │ └────────────────┘ │ │ └────────────────┘ │ │ └────────────────┘ │ │ │ │ │ └────────────────────┘ └────────────────────┘ └────────────────────┘ │ │ │ └───────────────────────────────────────────────────────────────────────────┘ │ │ │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │ │ │ Amazon RDS │ │ ElastiCache │ │ S3 │ │ AWS Secrets Manager │ │ │ │ PostgreSQL │ │ Redis │ │ Artifacts │ │ API Keys / Secrets │ │ │ └──────────────┘ └──────────────┘ └──────────────┘ └──────────────────────┘ │ │ │ │ ┌──────────────────────────────────────────────────────────────────────────┐ │ │ │ AWS Load Balancer (ALB) │ │ │ │ api.sandbox.yourdomain.com │ │ │ └──────────────────────────────────────────────────────────────────────────┘ │ └─────────────────────────────────────────────────────────────────────────────────┘ ``` ## Step 1: Create EKS Cluster ### Option A: Using eksctl (Recommended) Create a cluster configuration file: ```yaml theme={null} # cluster.yaml apiVersion: eksctl.io/v1alpha5 kind: ClusterConfig metadata: name: stateset-sandbox region: us-east-1 version: "1.29" vpc: cidr: 10.0.0.0/16 nat: gateway: HighlyAvailable iam: withOIDC: true managedNodeGroups: # Controller nodes - name: controller instanceType: t3.large desiredCapacity: 3 minSize: 2 maxSize: 5 volumeSize: 50 labels: node-type: controller tags: environment: production iam: withAddonPolicies: imageBuilder: true autoScaler: true ebs: true albIngress: true cloudWatch: true # Sandbox nodes (gVisor-enabled) - name: sandbox-gvisor instanceType: c5.2xlarge desiredCapacity: 3 minSize: 1 maxSize: 20 volumeSize: 100 labels: node-type: sandbox isolation: gvisor taints: - key: stateset.com/sandbox value: "true" effect: NoSchedule preBootstrapCommands: # Install gVisor - | set -o errexit set -o nounset set -o pipefail ARCH=$(uname -m) URL="https://storage.googleapis.com/gvisor/releases/release/latest/${ARCH}" wget "${URL}/runsc" "${URL}/containerd-shim-runsc-v1" -P /tmp chmod a+rx /tmp/runsc /tmp/containerd-shim-runsc-v1 mv /tmp/runsc /tmp/containerd-shim-runsc-v1 /usr/local/bin/ cat < external-dns-policy.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": ["route53:ChangeResourceRecordSets"], "Resource": ["arn:aws:route53:::hostedzone/*"] }, { "Effect": "Allow", "Action": ["route53:ListHostedZones", "route53:ListResourceRecordSets"], "Resource": ["*"] } ] } EOF aws iam create-policy \ --policy-name ExternalDNSPolicy \ --policy-document file://external-dns-policy.json # Install external-dns helm repo add external-dns https://kubernetes-sigs.github.io/external-dns/ helm install external-dns external-dns/external-dns \ --namespace kube-system \ --set provider=aws \ --set aws.zoneType=public \ --set txtOwnerId=stateset-sandbox \ --set policy=sync ``` ### cert-manager ```bash theme={null} helm repo add jetstack https://charts.jetstack.io helm repo update helm install cert-manager jetstack/cert-manager \ --namespace cert-manager \ --create-namespace \ --set installCRDs=true # Create ClusterIssuer for Let's Encrypt cat < secrets-policy.json { "Version": "2012-10-17", "Statement": [{ "Effect": "Allow", "Action": ["secretsmanager:GetSecretValue"], "Resource": "arn:aws:secretsmanager:us-east-1:*:secret:stateset-sandbox/*" }] } EOF aws iam create-policy \ --policy-name StateSetSandboxSecretsPolicy \ --policy-document file://secrets-policy.json ``` ## Step 4: Install External Secrets Operator ```bash theme={null} helm repo add external-secrets https://charts.external-secrets.io helm install external-secrets external-secrets/external-secrets \ --namespace external-secrets \ --create-namespace # Create SecretStore cat < cluster-autoscaler-policy.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "autoscaling:DescribeAutoScalingGroups", "autoscaling:DescribeAutoScalingInstances", "autoscaling:DescribeLaunchConfigurations", "autoscaling:DescribeTags", "autoscaling:SetDesiredCapacity", "autoscaling:TerminateInstanceInAutoScalingGroup", "ec2:DescribeLaunchTemplateVersions" ], "Resource": "*" } ] } EOF aws iam create-policy \ --policy-name ClusterAutoscalerPolicy \ --policy-document file://cluster-autoscaler-policy.json # Install autoscaler helm repo add autoscaler https://kubernetes.github.io/autoscaler helm install cluster-autoscaler autoscaler/cluster-autoscaler \ --namespace kube-system \ --set autoDiscovery.clusterName=stateset-sandbox \ --set awsRegion=us-east-1 ``` ### Horizontal Pod Autoscaler ```yaml theme={null} # hpa.yaml apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: sandbox-controller namespace: stateset-sandbox spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: sandbox-controller minReplicas: 3 maxReplicas: 10 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 70 - type: Resource resource: name: memory target: type: Utilization averageUtilization: 80 ``` ## Security Hardening ### VPC Security Groups ```bash theme={null} # Create security group for controller aws ec2 create-security-group \ --group-name stateset-sandbox-controller \ --description "Stateset Sandbox Controller" \ --vpc-id vpc-xxx # Allow inbound from ALB aws ec2 authorize-security-group-ingress \ --group-id sg-xxx \ --protocol tcp \ --port 8080 \ --source-group sg-alb-xxx # Create security group for sandboxes aws ec2 create-security-group \ --group-name stateset-sandbox-pods \ --description "Stateset Sandbox Pods" \ --vpc-id vpc-xxx # Allow inbound from controller only aws ec2 authorize-security-group-ingress \ --group-id sg-yyy \ --protocol tcp \ --port 0-65535 \ --source-group sg-controller-xxx ``` ### KMS Encryption ```bash theme={null} # Create KMS key for secrets aws kms create-key \ --description "Stateset Sandbox secrets encryption" \ --tags TagKey=Application,TagValue=stateset-sandbox # Enable EKS secrets encryption aws eks associate-encryption-config \ --cluster-name stateset-sandbox \ --encryption-config '[{ "resources": ["secrets"], "provider": {"keyArn": "arn:aws:kms:us-east-1:xxx:key/xxx"} }]' ``` ## Cost Optimization ### Spot Instances for Sandbox Nodes ```yaml theme={null} # Add to cluster.yaml managedNodeGroups: - name: sandbox-spot instanceTypes: - c5.2xlarge - c5a.2xlarge - c5n.2xlarge spot: true desiredCapacity: 3 labels: node-type: sandbox lifecycle: spot taints: - key: stateset.com/sandbox value: "true" effect: NoSchedule ``` ### Savings Plans Consider purchasing Compute Savings Plans for the controller nodes (always running) and using Spot for sandbox nodes (variable load). ## Estimated Monthly Costs | Resource | Configuration | Est. Cost | | ----------------- | -------------------------- | ------------------------- | | EKS Cluster | 1 cluster | \$73 | | Controller Nodes | 3x t3.large (on-demand) | \~\$175 | | Sandbox Nodes | 3-10x c5.2xlarge (spot) | \~$200-$800 | | RDS PostgreSQL | db.r6g.large Multi-AZ | \~\$350 | | ElastiCache Redis | cache.r6g.large 2 replicas | \~\$400 | | ALB | 1 ALB + traffic | \~\$50 | | S3 | Storage + requests | \~\$25 | | Data Transfer | \~500GB/month | \~\$45 | | **Total** | | **\~$1,300-$1,900/month** | ## Troubleshooting ### Common Issues **Pod stuck in Pending** ```bash theme={null} kubectl describe pod -n stateset-sandbox # Check for resource constraints or node affinity issues ``` **ALB not routing traffic** ```bash theme={null} kubectl describe ingress sandbox-controller -n stateset-sandbox # Check ALB controller logs kubectl logs -n kube-system -l app.kubernetes.io/name=aws-load-balancer-controller ``` **gVisor pods failing** ```bash theme={null} # Check node has gVisor installed kubectl get nodes -l isolation=gvisor kubectl debug node/ -it --image=busybox -- ls /usr/local/bin/runsc ``` ### Support Resources * [AWS EKS Documentation](https://docs.aws.amazon.com/eks/) * [StateSet Sandbox Issues](https://github.com/stateset/sandbox/issues) * [AWS Support](https://aws.amazon.com/support/) The AWS cluster above runs sandboxed, untrusted code. Do not reuse an existing EKS cluster that also runs your own services: the isolation story in the [security guide](/stateset-sandbox/stateset-sandbox-security-guide) assumes a dedicated node pool, and a shared one gives a sandboxed process neighbours worth attacking. ## Next steps Read before this cluster takes untrusted code — network policy is what contains it. Health probes, metrics and runbooks for the cluster you just built. Capacity, warm pools and the settings that decide cost. The same deployment on the other two clouds. # Sandbox Deployment: Azure Source: https://docs.stateset.com/stateset-sandbox/stateset-sandbox-deployment-azure Deploying the StateSet Sandbox on Azure. # Azure Deployment Guide (AKS) Complete guide for deploying StateSet Sandbox on Azure Kubernetes Service (AKS). ## Prerequisites * Azure CLI configured with appropriate permissions * kubectl installed * Helm 3.x installed * Domain name for API endpoint * Azure subscription with required resource providers enabled ## Quick Start (Kustomize Overlay) Use the Azure-specific overlay to apply the correct ingress/controller defaults: ```bash theme={null} kubectl apply -k k8s/overlays/azure # or ./scripts/deploy-k8s.sh azure # or (CLI) stateset-sandbox deploy azure stateset-sandbox deploy azure --preflight --wait --smoke ``` `stateset-sandbox` is the `@stateset/sandbox-cli` binary. It also installs a `stateset` alias, but that name collides with `@stateset/cli`, whose `stateset` has no `deploy` command, so use the long name. For smoke checks, export credentials before running deploy: ```bash theme={null} export STATESET_API_KEY="your_api_key" export STATESET_API_URL="https://api.sandbox.stateset.app/api/v1" ``` Before applying the overlay, provision the required controller auth secret: ```bash theme={null} read -rsp "Admin password: " ADMIN_PASSWORD; echo bash scripts/create-controller-secret.sh \ --admin-username admin \ --admin-password "$ADMIN_PASSWORD" unset ADMIN_PASSWORD ``` Before applying, update the Azure SAS token secret: ```bash theme={null} # Edit the placeholder SAS token vi k8s/overlays/azure/secret.yaml ``` ## Architecture Overview ``` ┌─────────────────────────────────────────────────────────────────────────────────┐ │ Microsoft Azure │ │ ┌───────────────────────────────────────────────────────────────────────────┐ │ │ │ VNet (10.0.0.0/16) │ │ │ │ ┌────────────────────┐ ┌────────────────────┐ ┌────────────────────┐ │ │ │ │ │ Subnet: AKS │ │ Subnet: Database │ │ Subnet: Redis │ │ │ │ │ │ 10.0.0.0/20 │ │ 10.0.16.0/24 │ │ 10.0.17.0/24 │ │ │ │ │ │ ┌──────────────┐ │ │ ┌──────────────┐ │ │ ┌──────────────┐ │ │ │ │ │ │ │ AKS Cluster │ │ │ │ Azure DB for │ │ │ │ Azure Cache │ │ │ │ │ │ │ │ - System Pool│ │ │ │ PostgreSQL │ │ │ │ for Redis │ │ │ │ │ │ │ │ - Sandbox │ │ │ │ Flex Server │ │ │ │ │ │ │ │ │ │ │ │ Pool │ │ │ └──────────────┘ │ │ └──────────────┘ │ │ │ │ │ │ └──────────────┘ │ │ │ │ │ │ │ │ │ └────────────────────┘ └────────────────────┘ └────────────────────┘ │ │ │ └───────────────────────────────────────────────────────────────────────────┘ │ │ │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │ │ │ Azure Blob │ │ Key Vault │ │ Application │ │ Container Registry │ │ │ │ Storage │ │ Secrets │ │ Gateway │ │ (ACR) │ │ │ └──────────────┘ └──────────────┘ └──────────────┘ └──────────────────────┘ │ └─────────────────────────────────────────────────────────────────────────────────┘ ``` ## Step 1: Set Up Azure Environment ```bash theme={null} # Login to Azure az login # Set subscription export SUBSCRIPTION_ID="your-subscription-id" az account set --subscription $SUBSCRIPTION_ID # Set variables export RESOURCE_GROUP="stateset-sandbox-rg" export LOCATION="eastus" export AKS_CLUSTER="stateset-sandbox-aks" # Register required providers az provider register --namespace Microsoft.ContainerService az provider register --namespace Microsoft.DBforPostgreSQL az provider register --namespace Microsoft.Cache az provider register --namespace Microsoft.KeyVault az provider register --namespace Microsoft.Storage az provider register --namespace Microsoft.Network # Create resource group az group create --name $RESOURCE_GROUP --location $LOCATION ``` ## Step 2: Create Virtual Network ```bash theme={null} # Create VNet az network vnet create \ --resource-group $RESOURCE_GROUP \ --name stateset-sandbox-vnet \ --address-prefix 10.0.0.0/16 # Create subnet for AKS az network vnet subnet create \ --resource-group $RESOURCE_GROUP \ --vnet-name stateset-sandbox-vnet \ --name aks-subnet \ --address-prefix 10.0.0.0/20 # Create subnet for PostgreSQL az network vnet subnet create \ --resource-group $RESOURCE_GROUP \ --vnet-name stateset-sandbox-vnet \ --name db-subnet \ --address-prefix 10.0.16.0/24 \ --delegations Microsoft.DBforPostgreSQL/flexibleServers # Create subnet for Redis az network vnet subnet create \ --resource-group $RESOURCE_GROUP \ --vnet-name stateset-sandbox-vnet \ --name redis-subnet \ --address-prefix 10.0.17.0/24 # Get subnet IDs AKS_SUBNET_ID=$(az network vnet subnet show \ --resource-group $RESOURCE_GROUP \ --vnet-name stateset-sandbox-vnet \ --name aks-subnet \ --query id -o tsv) DB_SUBNET_ID=$(az network vnet subnet show \ --resource-group $RESOURCE_GROUP \ --vnet-name stateset-sandbox-vnet \ --name db-subnet \ --query id -o tsv) ``` ## Step 3: Create Azure Container Registry ```bash theme={null} # Create ACR az acr create \ --resource-group $RESOURCE_GROUP \ --name statesetsandboxacr \ --sku Premium \ --admin-enabled false # Get ACR ID ACR_ID=$(az acr show --name statesetsandboxacr --query id -o tsv) ``` ## Step 4: Create AKS Cluster ```bash theme={null} # Create managed identity for AKS az identity create \ --resource-group $RESOURCE_GROUP \ --name stateset-sandbox-identity IDENTITY_ID=$(az identity show \ --resource-group $RESOURCE_GROUP \ --name stateset-sandbox-identity \ --query id -o tsv) IDENTITY_CLIENT_ID=$(az identity show \ --resource-group $RESOURCE_GROUP \ --name stateset-sandbox-identity \ --query clientId -o tsv) # Create AKS cluster az aks create \ --resource-group $RESOURCE_GROUP \ --name $AKS_CLUSTER \ --location $LOCATION \ --kubernetes-version 1.29 \ --node-count 3 \ --node-vm-size Standard_D4s_v3 \ --network-plugin azure \ --network-policy azure \ --vnet-subnet-id $AKS_SUBNET_ID \ --service-cidr 10.1.0.0/16 \ --dns-service-ip 10.1.0.10 \ --enable-managed-identity \ --assign-identity $IDENTITY_ID \ --enable-workload-identity \ --enable-oidc-issuer \ --enable-addons monitoring \ --attach-acr statesetsandboxacr \ --nodepool-name systempool \ --nodepool-labels node-type=controller \ --zones 1 2 3 \ --tier standard # Add sandbox node pool with Kata Containers az aks nodepool add \ --resource-group $RESOURCE_GROUP \ --cluster-name $AKS_CLUSTER \ --name sandboxpool \ --node-count 2 \ --node-vm-size Standard_D8s_v3 \ --node-osdisk-size 100 \ --workload-runtime KataMshvVmIsolation \ --os-sku AzureLinux \ --labels node-type=sandbox isolation=kata \ --node-taints stateset.com/sandbox=true:NoSchedule \ --min-count 0 \ --max-count 20 \ --enable-cluster-autoscaler \ --zones 1 2 3 # Get credentials az aks get-credentials --resource-group $RESOURCE_GROUP --name $AKS_CLUSTER ``` ### Alternative: gVisor Node Pool If you prefer gVisor over Kata: ```bash theme={null} # Add gVisor node pool (requires custom VM image) az aks nodepool add \ --resource-group $RESOURCE_GROUP \ --cluster-name $AKS_CLUSTER \ --name gvisorpool \ --node-count 2 \ --node-vm-size Standard_D8s_v3 \ --labels node-type=sandbox isolation=gvisor \ --node-taints stateset.com/sandbox=true:NoSchedule \ --min-count 0 \ --max-count 20 \ --enable-cluster-autoscaler ``` ## Step 5: Create Azure Database for PostgreSQL ```bash theme={null} # Create private DNS zone az network private-dns zone create \ --resource-group $RESOURCE_GROUP \ --name privatelink.postgres.database.azure.com # Link DNS zone to VNet az network private-dns link vnet create \ --resource-group $RESOURCE_GROUP \ --zone-name privatelink.postgres.database.azure.com \ --name stateset-sandbox-link \ --virtual-network stateset-sandbox-vnet \ --registration-enabled false # Create PostgreSQL Flexible Server az postgres flexible-server create \ --resource-group $RESOURCE_GROUP \ --name stateset-sandbox-pg \ --location $LOCATION \ --admin-user pgadmin \ --admin-password "YOUR_STRONG_PASSWORD" \ --sku-name Standard_D4s_v3 \ --tier GeneralPurpose \ --version 15 \ --storage-size 128 \ --subnet $DB_SUBNET_ID \ --private-dns-zone privatelink.postgres.database.azure.com \ --high-availability ZoneRedundant \ --backup-retention 7 # Create database az postgres flexible-server db create \ --resource-group $RESOURCE_GROUP \ --server-name stateset-sandbox-pg \ --database-name sandbox # Get connection string PG_HOST=$(az postgres flexible-server show \ --resource-group $RESOURCE_GROUP \ --name stateset-sandbox-pg \ --query fullyQualifiedDomainName -o tsv) ``` ## Step 6: Create Azure Cache for Redis ```bash theme={null} # Create Redis cache az redis create \ --resource-group $RESOURCE_GROUP \ --name stateset-sandbox-redis \ --location $LOCATION \ --sku Premium \ --vm-size P1 \ --subnet-id /subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.Network/virtualNetworks/stateset-sandbox-vnet/subnets/redis-subnet \ --enable-non-ssl-port false \ --minimum-tls-version 1.2 \ --zones 1 2 3 # Get Redis connection info REDIS_HOST=$(az redis show \ --resource-group $RESOURCE_GROUP \ --name stateset-sandbox-redis \ --query hostName -o tsv) REDIS_KEY=$(az redis list-keys \ --resource-group $RESOURCE_GROUP \ --name stateset-sandbox-redis \ --query primaryKey -o tsv) ``` ## Step 7: Create Azure Blob Storage ```bash theme={null} # Create storage account az storage account create \ --resource-group $RESOURCE_GROUP \ --name statesetsandboxstorage \ --location $LOCATION \ --sku Standard_ZRS \ --kind StorageV2 \ --min-tls-version TLS1_2 \ --allow-blob-public-access false \ --https-only true # Create container for artifacts az storage container create \ --account-name statesetsandboxstorage \ --name artifacts \ --auth-mode login # Enable versioning az storage account blob-service-properties update \ --resource-group $RESOURCE_GROUP \ --account-name statesetsandboxstorage \ --enable-versioning true # Set lifecycle management az storage account management-policy create \ --resource-group $RESOURCE_GROUP \ --account-name statesetsandboxstorage \ --policy '{ "rules": [{ "name": "deleteOldArtifacts", "enabled": true, "type": "Lifecycle", "definition": { "filters": {"blobTypes": ["blockBlob"], "prefixMatch": ["artifacts/"]}, "actions": { "baseBlob": {"delete": {"daysAfterModificationGreaterThan": 90}}, "version": {"delete": {"daysAfterCreationGreaterThan": 30}} } } }] }' ``` ## Step 8: Create Azure Key Vault ```bash theme={null} # Create Key Vault az keyvault create \ --resource-group $RESOURCE_GROUP \ --name stateset-sandbox-kv \ --location $LOCATION \ --sku premium \ --enable-rbac-authorization # Get Key Vault ID KEYVAULT_ID=$(az keyvault show --name stateset-sandbox-kv --query id -o tsv) # Add secrets az keyvault secret set \ --vault-name stateset-sandbox-kv \ --name jwt-secret \ --value "$(openssl rand -base64 32)" az keyvault secret set \ --vault-name stateset-sandbox-kv \ --name database-encryption-key \ --value "$(openssl rand -hex 32)" az keyvault secret set \ --vault-name stateset-sandbox-kv \ --name stripe-secret-key \ --value "your-stripe-secret-key" az keyvault secret set \ --vault-name stateset-sandbox-kv \ --name redis-password \ --value "$REDIS_KEY" # Grant AKS identity access to Key Vault az role assignment create \ --role "Key Vault Secrets User" \ --assignee $IDENTITY_CLIENT_ID \ --scope $KEYVAULT_ID ``` ## Step 9: Configure Workload Identity ```bash theme={null} # Get OIDC issuer URL OIDC_ISSUER=$(az aks show \ --resource-group $RESOURCE_GROUP \ --name $AKS_CLUSTER \ --query oidcIssuerProfile.issuerUrl -o tsv) # Create user-assigned identity for the controller az identity create \ --resource-group $RESOURCE_GROUP \ --name sandbox-controller-identity CONTROLLER_IDENTITY_CLIENT_ID=$(az identity show \ --resource-group $RESOURCE_GROUP \ --name sandbox-controller-identity \ --query clientId -o tsv) # Create federated credential az identity federated-credential create \ --name sandbox-controller-fed \ --identity-name sandbox-controller-identity \ --resource-group $RESOURCE_GROUP \ --issuer $OIDC_ISSUER \ --subject system:serviceaccount:stateset-sandbox:sandbox-controller # Grant permissions az role assignment create \ --role "Key Vault Secrets User" \ --assignee $CONTROLLER_IDENTITY_CLIENT_ID \ --scope $KEYVAULT_ID az role assignment create \ --role "Storage Blob Data Contributor" \ --assignee $CONTROLLER_IDENTITY_CLIENT_ID \ --scope /subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.Storage/storageAccounts/statesetsandboxstorage ``` ## Step 10: Deploy StateSet Sandbox ### Create Namespace ```bash theme={null} kubectl apply -f k8s/namespace.yaml ``` ### Configure Service Account with Workload Identity ```yaml theme={null} # azure-service-account.yaml apiVersion: v1 kind: ServiceAccount metadata: name: sandbox-controller namespace: stateset-sandbox annotations: azure.workload.identity/client-id: "YOUR_CONTROLLER_IDENTITY_CLIENT_ID" labels: azure.workload.identity/use: "true" ``` ### Install CSI Secrets Store Driver ```bash theme={null} # Install Secrets Store CSI driver helm repo add csi-secrets-store-provider-azure https://azure.github.io/secrets-store-csi-driver-provider-azure/charts helm install csi-secrets-store-provider-azure csi-secrets-store-provider-azure/csi-secrets-store-provider-azure \ --namespace kube-system ``` ### Create SecretProviderClass ```yaml theme={null} # azure-secret-provider.yaml apiVersion: secrets-store.csi.x-k8s.io/v1 kind: SecretProviderClass metadata: name: azure-keyvault-secrets namespace: stateset-sandbox spec: provider: azure parameters: usePodIdentity: "false" useVMManagedIdentity: "false" clientID: "YOUR_CONTROLLER_IDENTITY_CLIENT_ID" keyvaultName: "stateset-sandbox-kv" tenantId: "YOUR_TENANT_ID" objects: | array: - | objectName: jwt-secret objectType: secret - | objectName: database-encryption-key objectType: secret - | objectName: stripe-secret-key objectType: secret - | objectName: redis-password objectType: secret secretObjects: - secretName: sandbox-controller-auth type: Opaque data: - objectName: jwt-secret key: jwt-secret - objectName: database-encryption-key key: database-encryption-key - objectName: stripe-secret-key key: stripe-secret-key - objectName: redis-password key: redis-password ``` ### Create ConfigMap ```yaml theme={null} # azure-configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: sandbox-controller-config namespace: stateset-sandbox data: # Azure-specific configuration SANDBOX_IMAGE: "statesetsandboxacr.azurecr.io/stateset-sandbox:latest" # Database (Azure PostgreSQL) DATABASE_URL: "postgres://pgadmin:PASSWORD@stateset-sandbox-pg.postgres.database.azure.com:5432/sandbox?sslmode=require" # Redis (Azure Cache) REDIS_HOST: "stateset-sandbox-redis.redis.cache.windows.net" REDIS_PORT: "6380" REDIS_TLS: "true" # Storage (Azure Blob) STORAGE_PROVIDER: "azure" STORAGE_BUCKET: "artifacts" AZURE_STORAGE_ACCOUNT: "statesetsandboxstorage" # AZURE_STORAGE_SAS_TOKEN is injected via secret in the Azure kustomize overlay. # AZURE_STORAGE_ENDPOINT: "https://statesetsandboxstorage.blob.core.windows.net" # Isolation (Kata Containers on AKS) DEFAULT_ISOLATION: "kata" SANDBOX_RUNTIME_CLASS: "kata-mshv-vm-isolation" # Warm pool WARM_POOL_ENABLED: "true" WARM_POOL_SIZE: "5" # CORS CORS_ORIGIN: "https://sandbox.yourdomain.com" APP_URL: "https://api.sandbox.yourdomain.com" # Logging LOG_LEVEL: "info" NODE_ENV: "production" ``` ### Update Deployment for Azure ```yaml theme={null} # azure-deployment-patch.yaml spec: template: metadata: labels: azure.workload.identity/use: "true" spec: serviceAccountName: sandbox-controller containers: - name: sandbox-controller volumeMounts: - name: secrets-store mountPath: "/mnt/secrets-store" readOnly: true volumes: - name: secrets-store csi: driver: secrets-store.csi.k8s.io readOnly: true volumeAttributes: secretProviderClass: azure-keyvault-secrets ``` ### Deploy Components ```bash theme={null} kubectl apply -f azure-service-account.yaml kubectl apply -f azure-secret-provider.yaml kubectl apply -f azure-configmap.yaml kubectl apply -f k8s/rbac.yaml kubectl apply -f k8s/deployment.yaml kubectl apply -f k8s/service.yaml kubectl apply -f k8s/network-policy.yaml kubectl apply -f k8s/resource-quota.yaml ``` ### Create Kata RuntimeClass ```yaml theme={null} # kata-runtimeclass.yaml apiVersion: node.k8s.io/v1 kind: RuntimeClass metadata: name: kata-mshv-vm-isolation handler: kata-mshv-vm-isolation scheduling: nodeSelector: kubernetes.azure.com/kata-mshv-vm-isolation: "true" tolerations: - key: stateset.com/sandbox operator: Exists effect: NoSchedule ``` ## Step 11: Configure Application Gateway Ingress ### Option A: Application Gateway Ingress Controller (AGIC) ```bash theme={null} # Enable AGIC addon az aks addon update \ --resource-group $RESOURCE_GROUP \ --name $AKS_CLUSTER \ --addon ingress-appgw \ --appgw-name stateset-sandbox-agw \ --appgw-subnet-cidr 10.0.20.0/24 ``` ```yaml theme={null} # azure-ingress-agic.yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: sandbox-controller namespace: stateset-sandbox annotations: kubernetes.io/ingress.class: azure/application-gateway appgw.ingress.kubernetes.io/ssl-redirect: "true" appgw.ingress.kubernetes.io/backend-protocol: "http" appgw.ingress.kubernetes.io/health-probe-path: "/health" appgw.ingress.kubernetes.io/request-timeout: "300" # WAF policy (optional) # appgw.ingress.kubernetes.io/waf-policy-for-path: "/subscriptions/.../applicationGatewayWebApplicationFirewallPolicies/sandbox-waf" spec: tls: - hosts: - api.sandbox.yourdomain.com secretName: sandbox-tls-secret rules: - host: api.sandbox.yourdomain.com http: paths: - path: / pathType: Prefix backend: service: name: sandbox-controller port: number: 80 ``` ### Option B: NGINX Ingress Controller ```bash theme={null} # Install NGINX ingress controller helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx helm install ingress-nginx ingress-nginx/ingress-nginx \ --namespace ingress-nginx \ --create-namespace \ --set controller.service.annotations."service\.beta\.kubernetes\.io/azure-load-balancer-health-probe-request-path"=/healthz ``` ### Configure TLS Certificate ```bash theme={null} # Install cert-manager helm repo add jetstack https://charts.jetstack.io helm install cert-manager jetstack/cert-manager \ --namespace cert-manager \ --create-namespace \ --set installCRDs=true # Create ClusterIssuer cat <=5" \ --window-size 5m \ --evaluation-frequency 1m \ --action sandbox-alerts \ --description "High error rate in Stateset Sandbox" ``` ### Prometheus & Grafana (Optional) ```bash theme={null} # Install Prometheus stack helm repo add prometheus-community https://prometheus-community.github.io/helm-charts helm install prometheus prometheus-community/kube-prometheus-stack \ --namespace monitoring \ --create-namespace \ --set prometheus.prometheusSpec.serviceMonitorSelectorNilUsesHelmValues=false # Apply ServiceMonitor kubectl apply -f k8s/prometheus/service-monitor.yaml kubectl apply -f k8s/prometheus/alerts.yaml ``` ## Step 13: Configure Autoscaling ### Cluster Autoscaler Already enabled on sandbox node pool. Configure behavior: ```bash theme={null} # Update autoscaler profile az aks update \ --resource-group $RESOURCE_GROUP \ --name $AKS_CLUSTER \ --cluster-autoscaler-profile \ scale-down-delay-after-add=10m \ scale-down-unneeded-time=10m \ max-graceful-termination-sec=600 ``` ### Horizontal Pod Autoscaler ```yaml theme={null} # hpa.yaml apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: sandbox-controller namespace: stateset-sandbox spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: sandbox-controller minReplicas: 3 maxReplicas: 10 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 70 - type: Resource resource: name: memory target: type: Utilization averageUtilization: 80 ``` ### KEDA (Event-Driven Autoscaling) ```bash theme={null} # Install KEDA helm repo add kedacore https://kedacore.github.io/charts helm install keda kedacore/keda --namespace keda --create-namespace ``` ```yaml theme={null} # keda-scaledobject.yaml apiVersion: keda.sh/v1alpha1 kind: ScaledObject metadata: name: sandbox-controller-scaler namespace: stateset-sandbox spec: scaleTargetRef: name: sandbox-controller minReplicaCount: 3 maxReplicaCount: 20 triggers: - type: prometheus metadata: serverAddress: http://prometheus-server.monitoring.svc.cluster.local metricName: stateset_sandbox_active_sandboxes query: stateset_sandbox_active_sandboxes threshold: "50" ``` ## Security Hardening ### Azure Policy for AKS ```bash theme={null} # Enable Azure Policy addon az aks enable-addons \ --resource-group $RESOURCE_GROUP \ --name $AKS_CLUSTER \ --addons azure-policy # Assign built-in policies az policy assignment create \ --name "deny-privileged-containers" \ --policy "95edb821-ddaf-4404-9732-666045e056b4" \ --scope "/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.ContainerService/managedClusters/$AKS_CLUSTER" ``` ### Microsoft Defender for Containers ```bash theme={null} # Enable Defender for Containers az security pricing create \ --name Containers \ --tier Standard # Enable Defender for AKS cluster az aks update \ --resource-group $RESOURCE_GROUP \ --name $AKS_CLUSTER \ --enable-defender ``` ### Private Endpoints ```bash theme={null} # Create private endpoint for PostgreSQL az network private-endpoint create \ --resource-group $RESOURCE_GROUP \ --name pg-private-endpoint \ --vnet-name stateset-sandbox-vnet \ --subnet aks-subnet \ --private-connection-resource-id $(az postgres flexible-server show --resource-group $RESOURCE_GROUP --name stateset-sandbox-pg --query id -o tsv) \ --group-id postgresqlServer \ --connection-name pg-connection # Create private endpoint for Storage az network private-endpoint create \ --resource-group $RESOURCE_GROUP \ --name storage-private-endpoint \ --vnet-name stateset-sandbox-vnet \ --subnet aks-subnet \ --private-connection-resource-id $(az storage account show --resource-group $RESOURCE_GROUP --name statesetsandboxstorage --query id -o tsv) \ --group-id blob \ --connection-name storage-connection ``` ## Confidential Computing (Optional) For maximum isolation with Confidential VMs: ```bash theme={null} # Add confidential computing node pool az aks nodepool add \ --resource-group $RESOURCE_GROUP \ --cluster-name $AKS_CLUSTER \ --name confpool \ --node-count 2 \ --node-vm-size Standard_DC8s_v3 \ --os-sku AzureLinux \ --labels node-type=sandbox isolation=confidential \ --node-taints stateset.com/sandbox=true:NoSchedule \ --min-count 0 \ --max-count 10 \ --enable-cluster-autoscaler \ --enable-sgxquotehelper ``` ## Cost Optimization ### Azure Reserved Instances Purchase reserved capacity for predictable workloads: ```bash theme={null} az reservations reservation-order purchase \ --reservation-order-id "00000000-0000-0000-0000-000000000000" \ --sku Standard_D4s_v3 \ --location eastus \ --billing-scope-id "/subscriptions/$SUBSCRIPTION_ID" \ --term P1Y \ --billing-plan Monthly \ --quantity 3 ``` ### Spot VMs for Sandbox Nodes ```bash theme={null} # Add spot node pool az aks nodepool add \ --resource-group $RESOURCE_GROUP \ --cluster-name $AKS_CLUSTER \ --name spotpool \ --priority Spot \ --eviction-policy Delete \ --spot-max-price -1 \ --node-vm-size Standard_D8s_v3 \ --labels node-type=sandbox lifecycle=spot \ --node-taints stateset.com/sandbox=true:NoSchedule,kubernetes.azure.com/scalesetpriority=spot:NoSchedule \ --min-count 0 \ --max-count 20 \ --enable-cluster-autoscaler ``` ## Estimated Monthly Costs | Resource | Configuration | Est. Cost | | ------------------- | ------------------------------ | ------------------------- | | AKS Management | Free tier | \$0 | | System Node Pool | 3x Standard\_D4s\_v3 | \~\$350 | | Sandbox Node Pool | 2-10x Standard\_D8s\_v3 (Spot) | \~$150-$750 | | Azure PostgreSQL | Standard\_D4s\_v3 HA | \~\$450 | | Azure Cache Redis | Premium P1 | \~\$300 | | Application Gateway | WAF v2 | \~\$280 | | Blob Storage | 100GB + transactions | \~\$25 | | Egress | \~500GB/month | \~\$45 | | Key Vault | Operations | \~\$5 | | **Total** | | **\~$1,600-$2,200/month** | ## Troubleshooting ### Common Issues **Pod identity not working** ```bash theme={null} # Check workload identity configuration kubectl describe serviceaccount sandbox-controller -n stateset-sandbox # Verify federated credential az identity federated-credential show \ --name sandbox-controller-fed \ --identity-name sandbox-controller-identity \ --resource-group $RESOURCE_GROUP ``` **Database connection issues** ```bash theme={null} # Test connection from a pod kubectl run -it --rm debug --image=postgres:15 --restart=Never -- \ psql "postgres://pgadmin:pass@stateset-sandbox-pg.postgres.database.azure.com:5432/sandbox?sslmode=require" ``` **Kata containers not working** ```bash theme={null} # Check node pool configuration az aks nodepool show \ --resource-group $RESOURCE_GROUP \ --cluster-name $AKS_CLUSTER \ --name sandboxpool # Check RuntimeClass kubectl get runtimeclass kata-mshv-vm-isolation -o yaml ``` ### Useful Commands ```bash theme={null} # View AKS diagnostics az aks show \ --resource-group $RESOURCE_GROUP \ --name $AKS_CLUSTER \ --query "agentPoolProfiles[].{name:name, count:count, vmSize:vmSize}" # Stream logs kubectl logs -f -l app=sandbox-controller -n stateset-sandbox # Check node status kubectl get nodes -o wide # View resource usage kubectl top nodes kubectl top pods -n stateset-sandbox ``` ### Support Resources * [AKS Documentation](https://docs.microsoft.com/azure/aks/) * [Azure PostgreSQL Documentation](https://docs.microsoft.com/azure/postgresql/) * [StateSet Sandbox Issues](https://github.com/stateset/sandbox/issues) * [Azure Support](https://azure.microsoft.com/support/) The Azure cluster above runs sandboxed, untrusted code. Do not reuse an existing AKS cluster that also runs your own services: the isolation story in the [security guide](/stateset-sandbox/stateset-sandbox-security-guide) assumes a dedicated node pool, and a shared one gives a sandboxed process neighbours worth attacking. ## Next steps Read before this cluster takes untrusted code — network policy is what contains it. Health probes, metrics and runbooks for the cluster you just built. Capacity, warm pools and the settings that decide cost. The same deployment on the other two clouds. # Sandbox Deployment: GCP Source: https://docs.stateset.com/stateset-sandbox/stateset-sandbox-deployment-gcp Deploying the StateSet Sandbox on Google Cloud. # GCP Deployment Guide (GKE) Complete guide for deploying StateSet Sandbox on Google Kubernetes Engine (GKE). ## Prerequisites * gcloud CLI configured with appropriate permissions * kubectl installed * Helm 3.x installed * Domain name for API endpoint * GCP Project with billing enabled ## Quick Start (Kustomize Overlay) Use the GCP-specific overlay to apply the correct ingress/controller defaults: ```bash theme={null} kubectl apply -k k8s/overlays/gcp # or ./scripts/deploy-k8s.sh gcp # or (CLI) stateset-sandbox deploy gcp stateset-sandbox deploy gcp --preflight --wait --smoke ``` `stateset-sandbox` is the `@stateset/sandbox-cli` binary. It also installs a `stateset` alias, but that name collides with `@stateset/cli`, whose `stateset` has no `deploy` command, so use the long name. For smoke checks, export credentials before running deploy: ```bash theme={null} export STATESET_API_KEY="your_api_key" export STATESET_API_URL="https://api.sandbox.stateset.app/api/v1" ``` Before applying the overlay, provision the required controller auth secret: ```bash theme={null} read -rsp "Admin password: " ADMIN_PASSWORD; echo bash scripts/create-controller-secret.sh \ --admin-username admin \ --admin-password "$ADMIN_PASSWORD" unset ADMIN_PASSWORD ``` ## Architecture Overview ``` ┌─────────────────────────────────────────────────────────────────────────────────┐ │ Google Cloud │ │ ┌───────────────────────────────────────────────────────────────────────────┐ │ │ │ VPC (10.0.0.0/16) │ │ │ │ ┌─────────────────────────────────────────────────────────────────────┐ │ │ │ │ │ GKE Autopilot / Standard │ │ │ │ │ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │ │ │ │ │ │ Node Pool: │ │ Node Pool: │ │ Node Pool: │ │ │ │ │ │ │ │ Controller │ │ gVisor │ │ Confidential │ │ │ │ │ │ │ │ n2-standard-4 │ │ c2-standard-8 │ │ c2d-standard-8 │ │ │ │ │ │ │ │ (3 nodes) │ │ (auto-scale) │ │ (optional) │ │ │ │ │ │ │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ │ │ │ │ └─────────────────────────────────────────────────────────────────────┘ │ │ │ └───────────────────────────────────────────────────────────────────────────┘ │ │ │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │ │ │ Cloud SQL │ │ Memorystore │ │ Cloud Storage│ │ Secret Manager │ │ │ │ PostgreSQL │ │ Redis │ │ Artifacts │ │ API Keys / Secrets │ │ │ └──────────────┘ └──────────────┘ └──────────────┘ └──────────────────────┘ │ │ │ │ ┌──────────────────────────────────────────────────────────────────────────┐ │ │ │ Cloud Load Balancer (HTTPS) │ │ │ │ api.sandbox.yourdomain.com │ │ │ └──────────────────────────────────────────────────────────────────────────┘ │ └─────────────────────────────────────────────────────────────────────────────────┘ ``` ## Step 1: Set Up GCP Project ```bash theme={null} # Set project export PROJECT_ID="your-project-id" export REGION="us-central1" export ZONE="us-central1-a" gcloud config set project $PROJECT_ID gcloud config set compute/region $REGION gcloud config set compute/zone $ZONE # Enable required APIs gcloud services enable \ container.googleapis.com \ sqladmin.googleapis.com \ redis.googleapis.com \ secretmanager.googleapis.com \ certificatemanager.googleapis.com \ compute.googleapis.com \ artifactregistry.googleapis.com \ cloudkms.googleapis.com ``` ## Step 2: Create VPC Network ```bash theme={null} # Create VPC gcloud compute networks create stateset-sandbox-vpc \ --subnet-mode=custom # Create subnet for GKE nodes gcloud compute networks subnets create stateset-sandbox-subnet \ --network=stateset-sandbox-vpc \ --region=$REGION \ --range=10.0.0.0/20 \ --secondary-range=pods=10.4.0.0/14,services=10.8.0.0/20 # Create Cloud NAT for egress gcloud compute routers create stateset-sandbox-router \ --network=stateset-sandbox-vpc \ --region=$REGION gcloud compute routers nats create stateset-sandbox-nat \ --router=stateset-sandbox-router \ --region=$REGION \ --nat-all-subnet-ip-ranges \ --auto-allocate-nat-external-ips ``` ## Step 3: Create GKE Cluster ### Option A: GKE Standard with gVisor ```bash theme={null} # Create cluster gcloud container clusters create stateset-sandbox \ --region=$REGION \ --num-nodes=1 \ --machine-type=n2-standard-4 \ --disk-size=50GB \ --network=stateset-sandbox-vpc \ --subnetwork=stateset-sandbox-subnet \ --cluster-secondary-range-name=pods \ --services-secondary-range-name=services \ --enable-ip-alias \ --enable-network-policy \ --workload-pool=${PROJECT_ID}.svc.id.goog \ --enable-shielded-nodes \ --shielded-secure-boot \ --shielded-integrity-monitoring \ --release-channel=regular \ --node-labels=node-type=controller \ --addons=HorizontalPodAutoscaling,HttpLoadBalancing,GcePersistentDiskCsiDriver # Create gVisor-enabled node pool for sandboxes gcloud container node-pools create sandbox-gvisor \ --cluster=stateset-sandbox \ --region=$REGION \ --machine-type=c2-standard-8 \ --disk-size=100GB \ --num-nodes=1 \ --min-nodes=0 \ --max-nodes=20 \ --enable-autoscaling \ --sandbox=type=gvisor \ --node-labels=node-type=sandbox,isolation=gvisor \ --node-taints=stateset.com/sandbox=true:NoSchedule \ --shielded-secure-boot \ --shielded-integrity-monitoring # Get credentials gcloud container clusters get-credentials stateset-sandbox --region=$REGION ``` ### Option B: GKE Autopilot ```bash theme={null} # Create Autopilot cluster (automatically manages nodes) gcloud container clusters create-auto stateset-sandbox \ --region=$REGION \ --network=stateset-sandbox-vpc \ --subnetwork=stateset-sandbox-subnet \ --cluster-secondary-range-name=pods \ --services-secondary-range-name=services \ --workload-pool=${PROJECT_ID}.svc.id.goog \ --release-channel=regular # Get credentials gcloud container clusters get-credentials stateset-sandbox --region=$REGION ``` Note: GKE Autopilot supports gVisor via RuntimeClass with sandbox isolation. ## Step 4: Create Cloud SQL (PostgreSQL) ```bash theme={null} # Create Cloud SQL instance gcloud sql instances create stateset-sandbox \ --database-version=POSTGRES_15 \ --tier=db-custom-4-15360 \ --region=$REGION \ --network=stateset-sandbox-vpc \ --no-assign-ip \ --enable-google-private-path \ --storage-size=100GB \ --storage-type=SSD \ --storage-auto-increase \ --backup-start-time=03:00 \ --availability-type=REGIONAL \ --deletion-protection # Create database gcloud sql databases create sandbox --instance=stateset-sandbox # Create user gcloud sql users create sandbox-user \ --instance=stateset-sandbox \ --password="YOUR_STRONG_PASSWORD" # Get connection name for Kubernetes gcloud sql instances describe stateset-sandbox --format='value(connectionName)' ``` ## Step 5: Create Memorystore (Redis) ```bash theme={null} # Create Redis instance gcloud redis instances create stateset-sandbox \ --size=2 \ --region=$REGION \ --network=stateset-sandbox-vpc \ --tier=STANDARD_HA \ --redis-version=redis_7_0 \ --transit-encryption-mode=SERVER_AUTHENTICATION # Get host and port gcloud redis instances describe stateset-sandbox --region=$REGION \ --format='value(host,port)' ``` ## Step 6: Create Cloud Storage Bucket ```bash theme={null} # Create bucket gcloud storage buckets create gs://stateset-sandbox-artifacts-${PROJECT_ID} \ --location=$REGION \ --uniform-bucket-level-access \ --public-access-prevention # Set lifecycle rule cat < lifecycle.json { "rule": [{ "action": {"type": "Delete"}, "condition": {"age": 90, "matchesPrefix": ["artifacts/"]} }] } EOF gcloud storage buckets update gs://stateset-sandbox-artifacts-${PROJECT_ID} \ --lifecycle-file=lifecycle.json ``` ## Step 7: Set Up Secret Manager ```bash theme={null} # Create secrets echo -n "$(openssl rand -base64 32)" | \ gcloud secrets create jwt-secret --data-file=- echo -n "$(openssl rand -hex 32)" | \ gcloud secrets create database-encryption-key --data-file=- echo -n "your-stripe-secret-key" | \ gcloud secrets create stripe-secret-key --data-file=- # Create service account for secrets access gcloud iam service-accounts create sandbox-controller \ --display-name="Stateset Sandbox Controller" # Grant secrets access gcloud secrets add-iam-policy-binding jwt-secret \ --member="serviceAccount:sandbox-controller@${PROJECT_ID}.iam.gserviceaccount.com" \ --role="roles/secretmanager.secretAccessor" gcloud secrets add-iam-policy-binding database-encryption-key \ --member="serviceAccount:sandbox-controller@${PROJECT_ID}.iam.gserviceaccount.com" \ --role="roles/secretmanager.secretAccessor" gcloud secrets add-iam-policy-binding stripe-secret-key \ --member="serviceAccount:sandbox-controller@${PROJECT_ID}.iam.gserviceaccount.com" \ --role="roles/secretmanager.secretAccessor" # Grant GCS access gsutil iam ch serviceAccount:sandbox-controller@${PROJECT_ID}.iam.gserviceaccount.com:objectAdmin \ gs://stateset-sandbox-artifacts-${PROJECT_ID} ``` ## Step 8: Configure Workload Identity ```bash theme={null} # Allow Kubernetes service account to use GCP service account gcloud iam service-accounts add-iam-policy-binding \ sandbox-controller@${PROJECT_ID}.iam.gserviceaccount.com \ --role=roles/iam.workloadIdentityUser \ --member="serviceAccount:${PROJECT_ID}.svc.id.goog[stateset-sandbox/sandbox-controller]" ``` ## Step 9: Deploy StateSet Sandbox ### Create Namespace ```bash theme={null} kubectl apply -f k8s/namespace.yaml ``` ### Configure Service Account for Workload Identity ```yaml theme={null} # gcp-service-account.yaml apiVersion: v1 kind: ServiceAccount metadata: name: sandbox-controller namespace: stateset-sandbox annotations: iam.gke.io/gcp-service-account: sandbox-controller@YOUR_PROJECT_ID.iam.gserviceaccount.com ``` ### Create ConfigMap ```yaml theme={null} # gcp-configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: sandbox-controller-config namespace: stateset-sandbox data: # GCP-specific configuration SANDBOX_IMAGE: "us-central1-docker.pkg.dev/${PROJECT_ID}/stateset/sandbox:latest" # Database (Cloud SQL via private IP) DATABASE_URL: "postgres://sandbox-user:PASSWORD@CLOUD_SQL_PRIVATE_IP:5432/sandbox?sslmode=require" # Redis (Memorystore) REDIS_HOST: "MEMORYSTORE_HOST" REDIS_PORT: "6379" REDIS_TLS: "true" # Storage (GCS) STORAGE_PROVIDER: "gcs" STORAGE_BUCKET: "stateset-sandbox-artifacts-${PROJECT_ID}" GCS_PROJECT_ID: "${PROJECT_ID}" # Isolation DEFAULT_ISOLATION: "gvisor" SANDBOX_RUNTIME_CLASS: "gvisor" # Warm pool WARM_POOL_ENABLED: "true" WARM_POOL_SIZE: "5" # Exec agent runtime (optional) # Set to "go" to use the Go exec agent (execd) instead of the default Node.js agent. # The Go binary is included in the sandbox base image and has a significantly lower # memory footprint (~8MB RSS vs ~45MB for Node.js). # EXEC_AGENT_RUNTIME: "go" # CORS CORS_ORIGIN: "https://sandbox.yourdomain.com" APP_URL: "https://api.sandbox.yourdomain.com" # Logging (Cloud Logging integration) LOG_LEVEL: "info" NODE_ENV: "production" ``` ### Create External Secrets ```bash theme={null} # Install External Secrets Operator helm repo add external-secrets https://charts.external-secrets.io helm install external-secrets external-secrets/external-secrets \ --namespace external-secrets \ --create-namespace ``` ```yaml theme={null} # gcp-external-secrets.yaml apiVersion: external-secrets.io/v1beta1 kind: SecretStore metadata: name: gcp-secret-manager namespace: stateset-sandbox spec: provider: gcpsm: projectID: YOUR_PROJECT_ID --- apiVersion: external-secrets.io/v1beta1 kind: ExternalSecret metadata: name: sandbox-controller-secrets namespace: stateset-sandbox spec: refreshInterval: 1h secretStoreRef: name: gcp-secret-manager kind: SecretStore target: name: sandbox-controller-auth data: - secretKey: jwt-secret remoteRef: key: jwt-secret - secretKey: database-encryption-key remoteRef: key: database-encryption-key - secretKey: stripe-secret-key remoteRef: key: stripe-secret-key ``` ### Apply Custom Resource Definitions CRDs must be applied before the main deployment so the API server recognises StateSet resource types: ```bash theme={null} # Apply Custom Resource Definitions (before main deployment) kubectl apply -f k8s/crds/ # Verify CRDs are registered kubectl get crd | grep stateset # statesandboxes.sandbox.stateset.io # warmpools.sandbox.stateset.io # Optional: Enable CRD-based sandbox management # Set SANDBOX_BACKEND=crd in your configmap to use declarative sandbox management ``` ### Deploy Components ```bash theme={null} kubectl apply -f gcp-service-account.yaml kubectl apply -f gcp-configmap.yaml kubectl apply -f gcp-external-secrets.yaml kubectl apply -f k8s/rbac.yaml kubectl apply -f k8s/deployment.yaml kubectl apply -f k8s/service.yaml kubectl apply -f k8s/network-policy.yaml kubectl apply -f k8s/resource-quota.yaml ``` ### Create gVisor RuntimeClass ```yaml theme={null} # gvisor-runtimeclass.yaml apiVersion: node.k8s.io/v1 kind: RuntimeClass metadata: name: gvisor handler: runsc scheduling: nodeSelector: sandbox.gke.io/runtime: gvisor tolerations: - key: stateset.com/sandbox operator: Exists effect: NoSchedule ``` ```bash theme={null} kubectl apply -f gvisor-runtimeclass.yaml ``` ### CRD Usage Examples With the CRDs installed you can manage sandboxes and warm pools declaratively: ```yaml theme={null} # Example: Create a sandbox via CRD apiVersion: sandbox.stateset.io/v1alpha1 kind: StateSandbox metadata: name: my-sandbox namespace: stateset-sandbox spec: orgId: "org-123" image: "stateset/sandbox:latest" resources: cpu: "2" memory: "2Gi" timeoutSeconds: 600 isolation: container ``` ```bash theme={null} kubectl apply -f sandbox.yaml kubectl get ssb # short name kubectl get warmpools # or kubectl get wp ``` ## Step 10: Configure Load Balancer & SSL ### Managed SSL Certificate ```bash theme={null} # Create managed certificate gcloud compute ssl-certificates create stateset-sandbox-cert \ --domains=api.sandbox.yourdomain.com \ --global # Reserve static IP gcloud compute addresses create stateset-sandbox-ip --global ``` ### Ingress with GCE Load Balancer ```yaml theme={null} # gcp-ingress.yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: sandbox-controller namespace: stateset-sandbox annotations: kubernetes.io/ingress.global-static-ip-name: "stateset-sandbox-ip" networking.gke.io/managed-certificates: "sandbox-controller-cert" kubernetes.io/ingress.class: "gce" # Enable Cloud Armor (optional) # networking.gke.io/v1beta1.FrontendConfig: "sandbox-frontend-config" spec: rules: - host: api.sandbox.yourdomain.com http: paths: - path: /* pathType: ImplementationSpecific backend: service: name: sandbox-controller port: number: 80 --- apiVersion: networking.gke.io/v1 kind: ManagedCertificate metadata: name: sandbox-controller-cert namespace: stateset-sandbox spec: domains: - api.sandbox.yourdomain.com ``` ```bash theme={null} kubectl apply -f gcp-ingress.yaml # Get the external IP kubectl get ingress sandbox-controller -n stateset-sandbox # Update DNS to point to this IP ``` ### Cloud Armor Security Policy (Optional) ```bash theme={null} # Create security policy gcloud compute security-policies create stateset-sandbox-policy \ --description="Stateset Sandbox security policy" # Add rate limiting rule gcloud compute security-policies rules create 1000 \ --security-policy=stateset-sandbox-policy \ --action=rate-based-ban \ --rate-limit-threshold-count=100 \ --rate-limit-threshold-interval-sec=60 \ --ban-duration-sec=300 \ --conform-action=allow \ --exceed-action=deny-429 \ --enforce-on-key=IP # Add geo-blocking (optional) gcloud compute security-policies rules create 2000 \ --security-policy=stateset-sandbox-policy \ --action=deny-403 \ --expression="origin.region_code == 'CN'" # Create frontend config cat < The GCP cluster above runs sandboxed, untrusted code. Do not reuse an existing GKE cluster that also runs your own services: the isolation story in the [security guide](/stateset-sandbox/stateset-sandbox-security-guide) assumes a dedicated node pool, and a shared one gives a sandboxed process neighbours worth attacking. ## Next steps Read before this cluster takes untrusted code — network policy is what contains it. Health probes, metrics and runbooks for the cluster you just built. Capacity, warm pools and the settings that decide cost. The same deployment on the other two clouds. # Sandbox Deployments Source: https://docs.stateset.com/stateset-sandbox/stateset-sandbox-deployments Deployment guides for AWS, GCP, and Azure. StateSet Sandbox can be deployed on major cloud providers. Use the provider-specific guides for step-by-step instructions and infrastructure requirements. ## Choosing a provider All three run the same controller and runtime on managed Kubernetes. Pick on where the rest of your infrastructure lives, not on capability. | Provider | Cluster | Guide | | --------- | ------- | ----------------------------------------------------------------------- | | **AWS** | EKS | [AWS deployment](/stateset-sandbox/stateset-sandbox-deployment-aws) | | **GCP** | GKE | [GCP deployment](/stateset-sandbox/stateset-sandbox-deployment-gcp) | | **Azure** | AKS | [Azure deployment](/stateset-sandbox/stateset-sandbox-deployment-azure) | ## What every deployment needs Regardless of provider: * A managed Kubernetes cluster with room for the controller and runtime pods * A container registry the cluster can pull from * An ingress controller and TLS termination * Network policy allowing the controller to reach the runtime, and nothing else to Sandboxes execute untrusted code. Network policy is not optional hardening here — it is what stops a sandboxed process reaching the rest of your cluster. Read the [security guide](/stateset-sandbox/stateset-sandbox-security-guide) before the first production deploy, not after. ## Before you go to production The provider guides get a cluster running. These decide whether it survives contact with real workloads: | Concern | Where | | ----------------------------------------- | ------------------------------------------------------------------------- | | Capacity, scaling, resource profiles | [Production guide](/stateset-sandbox/stateset-sandbox-production-guide) | | Health probes, metrics, incident response | [Operations](/stateset-sandbox/stateset-sandbox-operations) | | Isolation model and hardening | [Security guide](/stateset-sandbox/stateset-sandbox-security-guide) | | Runtime selection | [Runtime selection](/stateset-sandbox/stateset-sandbox-runtime-selection) | ## Guides * [AWS Deployment](/stateset-sandbox/stateset-sandbox-deployment-aws) * [GCP Deployment](/stateset-sandbox/stateset-sandbox-deployment-gcp) * [Azure Deployment](/stateset-sandbox/stateset-sandbox-deployment-azure) ## Related Documentation * [Operations Guide](/stateset-sandbox/stateset-sandbox-operations) * [Production Guide](/stateset-sandbox/stateset-sandbox-production-guide) # Sandbox MCP Servers Source: https://docs.stateset.com/stateset-sandbox/stateset-sandbox-mcp Shopify, Gorgias, and Recharge operations as agent-callable tools. MCP servers for e-commerce platform operations, giving an agent tools to work directly against Shopify, Gorgias, Recharge, and related platforms. ## Available servers ### Shopify | Server | Scope | Tools | | ----------------- | ------------------------ | ------------------------------------------------- | | `shopify` | Fulfillment holds | `preview_orders`, `release_holds`, `add_tags` | | `shopify-full` | Comprehensive operations | **41** | | `shopify-refunds` | Partial refunds | `lookup_order`, `preview_refund`, `create_refund` | The narrow servers exist so you can grant an agent exactly one job. If an agent only needs to release fulfillment holds, give it `shopify` rather than `shopify-full` — 3 tools instead of 41, and no path to issuing a refund. ### `shopify-full` coverage **Orders** — `search_orders` with status, financial, fulfillment and date filters; `get_order` with line items, customer and transactions; `update_order`; `cancel_order` with reason, refund option and restock; `close_order` / `reopen_order`. **Fulfillment** — `get_fulfillment_orders`, `create_fulfillment` with tracking, `update_tracking`, `cancel_fulfillment`, `release_fulfillment_hold`. **Refunds** — `calculate_refund` to preview amounts, `create_refund`, `get_refunds`. **Products** — `search_products`, `get_product` with variants and images, `update_product`, `update_variant`. Plus customers, inventory, discounts, metafields, collections, gift cards, and draft orders. `create_refund`, `cancel_order`, and `create_fulfillment` take **real action on a live store**. `calculate_refund` and `preview_refund` exist precisely so an agent can check its arithmetic before committing — have it preview first. ## Other platforms Twenty servers ship in total, 254 tools between them. The same rule applies throughout: give an agent the narrow server when a narrow server exists. **Support** | Server | Scope | Tools | | -------------- | -------------------------------- | ----- | | `gorgias` | Support tickets | 9 | | `gorgias-full` | Comprehensive support operations | 34 | | `zendesk` | Support tickets | 9 | **Returns** | Server | Scope | Tools | | -------------- | --------------------------------------- | ----- | | `loop-returns` | Returns and exchanges | 9 | | `returnlogic` | Returns (ReturnLogic, now part of Loop) | 7 | **Subscriptions and payments** | Server | Scope | Tools | | ----------------- | ----------------------- | ----- | | `recharge` | Subscription management | 43 | | `stripe-payments` | Payment operations | 11 | **Fulfillment and shipping** | Server | Scope | Tools | | ------------- | ----------------------------------- | ----- | | `shiphero` | Warehouse fulfillment and inventory | 9 | | `shipstation` | Shipping and labels | 9 | | `shipfusion` | 3PL fulfillment | 9 | | `shiphawk` | Shipping and freight | 9 | | `aftership` | Shipment tracking | 6 | | `narvar` | Post-purchase experience | 6 | **Marketing and reviews** | Server | Scope | Tools | | ------------------- | ----------------------- | ----- | | `klaviyo-marketing` | Email and SMS marketing | 14 | | `attentive` | SMS marketing | 7 | | `yotpo` | Reviews and loyalty | 7 | | `okendo` | Reviews and Q\&A | 7 | ## Setup ```bash theme={null} cd mcp-servers npm install ``` Then register the server you want with your MCP host, supplying that platform's credentials via environment variables. Each server's README documents its required variables. ```json theme={null} { "mcpServers": { "shopify": { "command": "node", "args": ["/abs/path/mcp-servers/shopify/index.js"], "env": { "SHOPIFY_STORE": "acme.myshopify.com", "SHOPIFY_ACCESS_TOKEN": "shpat_…" } } } } ``` Registering the narrow `shopify` server rather than `shopify-full` is the whole security decision: the agent gets three tools instead of forty, and no path to issuing a refund. ## Choosing a scope | If the agent needs to… | Give it | | ------------------------------ | ----------------- | | Release fulfillment holds only | `shopify` | | Issue partial refunds only | `shopify-refunds` | | Operate the store broadly | `shopify-full` | | Handle support tickets | `gorgias` | | Manage subscriptions | `recharge` | These servers hold live store credentials and act on a real merchant's data. Scope the token you give them at the platform — a Shopify token with `write_orders` and nothing else cannot be talked into editing products, whatever the agent decides to try. For anything that must not happen on the model's judgement alone, put an [Agent Gate](/stateset-nsr-agent-gate) in front of the server so each call needs a proof before it runs. ## Next steps Requiring an authorization proof before a tool call runs. Where these servers sit. Isolation and credential handling. The wider set StateSet publishes. # Sandbox Operations Source: https://docs.stateset.com/stateset-sandbox/stateset-sandbox-operations Health checks, metrics, monitoring, and day-to-day operation of the sandbox. # Operations Guide This guide covers monitoring, metrics, health checks, alerting, and operational procedures for running StateSet Sandbox in production. ## Health Endpoints The controller exposes several health endpoints for monitoring and orchestration: ### GET /health Basic health check returning current status. ```bash theme={null} curl http://localhost:8080/health ``` Response: ```json theme={null} { "status": "healthy", "timestamp": "2024-01-21T12:00:00Z", "database": "connected", "redis": "connected" } ``` **Status values:** * `healthy` - All systems operational * `connected` / `disconnected` - Component status * `not_configured` - Optional component not enabled ### GET /ready Kubernetes readiness probe. Returns 200 when ready to accept traffic, 503 otherwise. Point the Kubernetes **liveness** probe at `/health` and the **readiness** probe at `/ready`, not both at `/health`. `/health` reports degraded dependencies without failing, so a liveness probe on it will not restart a wedged controller — and a readiness probe on it will keep sending traffic to a pod whose database has gone away. ```bash theme={null} curl http://localhost:8080/ready ``` Response: ```json theme={null} { "status": "ready", "checks": { "sandbox_manager": true, "redis": true } } ``` **Use in Kubernetes:** ```yaml theme={null} readinessProbe: httpGet: path: /ready port: 8080 initialDelaySeconds: 5 periodSeconds: 10 ``` ### GET /health/detailed Comprehensive health with component details. Useful for debugging. ```bash theme={null} curl http://localhost:8080/health/detailed ``` Response: ```json theme={null} { "timestamp": "2024-01-21T12:00:00Z", "environment": "production", "version": "1.0.0", "status": "healthy", "checks": { "database": { "status": "healthy" }, "redis": { "status": "healthy", "details": { "required": false, "connected": true } }, "sandbox_manager": { "status": "healthy" } } } ``` ### GET /metrics Prometheus metrics endpoint. Requires `INTERNAL_API_KEY` in production. ```bash theme={null} curl -H "X-Internal-Token: $INTERNAL_API_KEY" http://localhost:8080/metrics ``` ## Prometheus Metrics All metrics use the `stateset_sandbox_` prefix. ### HTTP Metrics | Metric | Type | Labels | Description | | ------------------------------- | --------- | -------------------- | ------------------- | | `http_requests_total` | Counter | method, path, status | Total HTTP requests | | `http_request_duration_seconds` | Histogram | method, path, status | Request latency | ### Sandbox Lifecycle | Metric | Type | Labels | Description | | ------------------------------------ | --------- | --------------- | ------------------------- | | `sandbox_creations_total` | Counter | status | Sandbox creation attempts | | `sandbox_startup_duration_seconds` | Histogram | phase | Startup time by phase | | `active_sandboxes` | Gauge | - | Current active sandboxes | | `sandbox_executions_total` | Counter | status | Command executions | | `sandbox_execution_duration_seconds` | Histogram | - | Command execution time | | `sandbox_terminations_total` | Counter | reason, org\_id | Terminations by reason | ### Warm Pool | Metric | Type | Labels | Description | | ------------------------------ | ------- | ------------------------------- | --------------------- | | `warm_pool_hits_total` | Counter | result | hit, miss, compatible | | `warm_pool_profile_hits_total` | Counter | result, cpus, memory, isolation | Hits by profile | ### Kubernetes Operations | Metric | Type | Labels | Description | | -------------------------------- | --------- | -------------------- | ------------------- | | `k8s_operations_total` | Counter | operation, status | K8s API calls | | `k8s_operation_duration_seconds` | Histogram | operation | K8s API latency | | `pod_scheduling_delay_seconds` | Histogram | isolation, scheduled | Pod scheduling time | | `pod_startup_events_total` | Counter | event\_type, reason | Pod events | ### API Keys & Auth | Metric | Type | Labels | Description | | ------------------------------- | ------- | ------------------ | ------------------------- | | `api_key_validations_total` | Counter | status, cache\_hit | Validation attempts | | `api_key_update_failures_total` | Counter | error\_type | Failed last\_used updates | | `api_key_usage_total` | Counter | key\_id, operation | Usage by key | ### Plan & Billing | Metric | Type | Labels | Description | | ----------------------------- | ------- | -------------------------------- | ---------------- | | `plan_limit_violations_total` | Counter | org\_id, plan\_type, limit\_type | Limit violations | | `org_resource_utilization` | Gauge | org\_id, resource\_type | Resource usage | | `billing_events_total` | Counter | event\_type, plan\_type | Billing events | ### Errors | Metric | Type | Labels | Description | | ---------------------------------- | ------- | ------------------------------- | ---------------------- | | `operation_errors_total` | Counter | operation, error\_code, org\_id | Operation errors | | `background_task_failures_total` | Counter | task\_name, error\_type | Background task errors | | `background_task_executions_total` | Counter | task\_name, status | Task executions | ### MicroVM & Advanced (when enabled) | Metric | Type | Labels | Description | | ------------------------------------------- | --------- | ----------------- | --------------- | | `microvm_startup_duration_seconds` | Histogram | phase, provider | MicroVM startup | | `microvm_snapshot_operations_total` | Counter | operation, status | Snapshot ops | | `microvm_snapshot_restore_duration_seconds` | Histogram | profile | Restore time | | `exec_latency_seconds` | Histogram | phase, method | Exec breakdown | | `gpu_allocations_total` | Counter | gpu\_type, status | GPU allocations | | `numa_scheduling_decisions_total` | Counter | decision | NUMA decisions | ### eBPF Observability (when enabled) | Metric | Type | Labels | Description | | ------------------------ | ------- | -------------------- | ----------------- | | `network_rx_bytes_total` | Counter | sandbox\_id, org\_id | Bytes received | | `network_tx_bytes_total` | Counter | sandbox\_id, org\_id | Bytes transmitted | | `syscall_count_total` | Counter | sandbox\_id, syscall | Syscall counts | | `ebpf_enabled` | Gauge | - | eBPF status (1/0) | | `network_collector_type` | Gauge | collector\_type | Active collector | ## Prometheus Configuration ```yaml theme={null} # prometheus.yml scrape_configs: - job_name: 'sandbox-controller' static_configs: - targets: ['sandbox-controller:8080'] metrics_path: /metrics authorization: type: Bearer credentials: '' scrape_interval: 15s ``` ## Grafana Dashboards ### Key Panels **Overview:** * Active sandboxes gauge * Sandbox creation rate * Error rate * P99 latency **Performance:** * Cold start histogram * Warm pool hit rate * Exec latency distribution * K8s API latency **Resources:** * CPU utilization by org * Memory utilization by org * Network bandwidth * Storage usage ### Example Queries ```promql theme={null} # Sandbox creation success rate (5m window) sum(rate(stateset_sandbox_sandbox_creations_total{status="success"}[5m])) / sum(rate(stateset_sandbox_sandbox_creations_total[5m])) # P99 cold start latency histogram_quantile(0.99, sum(rate(stateset_sandbox_sandbox_startup_duration_seconds_bucket[5m])) by (le) ) # Warm pool hit rate sum(rate(stateset_sandbox_warm_pool_hits_total{result="hit"}[5m])) / sum(rate(stateset_sandbox_warm_pool_hits_total[5m])) # Error rate by operation sum(rate(stateset_sandbox_operation_errors_total[5m])) by (operation) # Active sandboxes over time stateset_sandbox_active_sandboxes ``` ## Alerting Rules ### Prometheus AlertManager ```yaml theme={null} # alerts.yml groups: - name: sandbox-controller rules: # High error rate - alert: HighErrorRate expr: | sum(rate(stateset_sandbox_operation_errors_total[5m])) / sum(rate(stateset_sandbox_http_requests_total[5m])) > 0.05 for: 5m labels: severity: warning annotations: summary: "High error rate detected" description: "Error rate is {{ $value | humanizePercentage }}" # Controller down - alert: ControllerDown expr: up{job="sandbox-controller"} == 0 for: 1m labels: severity: critical annotations: summary: "Sandbox controller is down" # Database connection issues - alert: DatabaseUnhealthy expr: | stateset_sandbox_background_task_failures_total{task_name="database_health"} > 0 for: 2m labels: severity: critical annotations: summary: "Database connection issues" # Warm pool exhausted - alert: WarmPoolExhausted expr: | rate(stateset_sandbox_warm_pool_hits_total{result="miss"}[5m]) / rate(stateset_sandbox_warm_pool_hits_total[5m]) > 0.5 for: 10m labels: severity: warning annotations: summary: "Warm pool miss rate too high" description: "Consider increasing WARM_POOL_SIZE" # High latency - alert: HighLatency expr: | histogram_quantile(0.99, sum(rate(stateset_sandbox_http_request_duration_seconds_bucket[5m])) by (le) ) > 5 for: 5m labels: severity: warning annotations: summary: "P99 latency exceeds 5 seconds" # Sandbox creation failures - alert: SandboxCreationFailures expr: | rate(stateset_sandbox_sandbox_creations_total{status="failure"}[5m]) > 0.1 for: 5m labels: severity: warning annotations: summary: "Sandbox creation failures detected" ``` ## Logging ### Log Levels | Level | Use | | ------- | ------------------------------------- | | `fatal` | Unrecoverable errors causing shutdown | | `error` | Errors that affect functionality | | `warn` | Unexpected but handled conditions | | `info` | Normal operational events | | `debug` | Detailed debugging information | | `trace` | Very verbose tracing | ### Log Format Logs are JSON-formatted for easy parsing: ```json theme={null} { "level": "info", "time": "2024-01-21T12:00:00.000Z", "msg": "Sandbox created", "sandbox_id": "sb-abc123", "org_id": "org-xyz", "duration_ms": 1234 } ``` ### Log Aggregation **Kubernetes with Loki:** ```yaml theme={null} # promtail config scrape_configs: - job_name: sandbox-controller kubernetes_sd_configs: - role: pod relabel_configs: - source_labels: [__meta_kubernetes_pod_label_app] regex: sandbox-controller action: keep pipeline_stages: - json: expressions: level: level msg: msg sandbox_id: sandbox_id ``` ### Useful Log Queries ```logql theme={null} # Errors in last hour {app="sandbox-controller"} |= "error" | json | level="error" # Specific sandbox logs {app="sandbox-controller"} | json | sandbox_id="sb-abc123" # Slow requests {app="sandbox-controller"} | json | duration_ms > 5000 ``` ## Operational Runbooks ### High Error Rate 1. **Check metrics** for error patterns: ```promql theme={null} sum(rate(stateset_sandbox_operation_errors_total[5m])) by (operation, error_code) ``` 2. **Review logs** for stack traces: ```bash theme={null} kubectl logs -l app=sandbox-controller --since=10m | grep error ``` 3. **Check dependencies**: * Database: `curl /health | jq .database` * Redis: `curl /health | jq .redis` * K8s API: `kubectl get pods -n stateset-sandbox` 4. **Common causes**: * Database connection pool exhausted * K8s API rate limiting * Node resource pressure * Network issues ### Slow Cold Starts 1. **Identify phase** causing delay: ```promql theme={null} histogram_quantile(0.99, sum(rate(stateset_sandbox_sandbox_startup_duration_seconds_bucket[5m])) by (phase, le) ) ``` 2. **Check warm pool**: ```promql theme={null} rate(stateset_sandbox_warm_pool_hits_total{result="miss"}[5m]) ``` 3. **Check scheduling delays**: ```promql theme={null} histogram_quantile(0.99, sum(rate(stateset_sandbox_pod_scheduling_delay_seconds_bucket[5m])) by (le) ) ``` 4. **Remediation**: * Increase `WARM_POOL_SIZE` * Add more warm pool profiles * Check node resources * Verify image is pre-pulled ### Database Issues 1. **Check connection**: ```bash theme={null} curl http://localhost:8080/health/detailed | jq .checks.database ``` 2. **Check pool stats**: ```sql theme={null} SELECT state, count(*) FROM pg_stat_activity WHERE datname = 'sandbox' GROUP BY state; ``` 3. **Check for locks**: ```sql theme={null} SELECT * FROM pg_locks WHERE NOT granted; ``` 4. **Remediation**: * Restart controller to reset pool * Check database server health * Review connection limits ### Warm Pool Empty 1. **Check pool status**: ```promql theme={null} stateset_sandbox_warm_pool_hits_total ``` 2. **Check pod creation**: ```bash theme={null} kubectl get pods -n stateset-sandbox -l stateset.com/warm-pool=true ``` 3. **Review events**: ```bash theme={null} kubectl get events -n stateset-sandbox --sort-by='.lastTimestamp' | head -20 ``` 4. **Remediation**: * Check cluster capacity * Review warm pool configuration * Check for image pull issues * Verify RBAC permissions ### Memory Pressure 1. **Check controller memory**: ```bash theme={null} kubectl top pod -l app=sandbox-controller -n stateset-sandbox ``` 2. **Check for leaks**: ```promql theme={null} process_resident_memory_bytes{job="sandbox-controller"} ``` 3. **Remediation**: * Increase memory limits * Review connection pool sizes * Check for large response payloads * Consider horizontal scaling ## Scaling ### Horizontal Scaling The controller supports horizontal scaling with some considerations: 1. **Database**: Shared state via PostgreSQL 2. **Redis**: Required for distributed warm pool 3. **Leader election**: Not currently implemented (future) ```yaml theme={null} apiVersion: apps/v1 kind: Deployment spec: replicas: 3 template: spec: containers: - name: controller resources: requests: cpu: "500m" memory: "512Mi" limits: cpu: "2" memory: "2Gi" ``` ### Vertical Scaling For single-instance deployments: | Sandboxes | CPU | Memory | | --------- | --- | ------ | | 1-50 | 0.5 | 512Mi | | 50-200 | 1 | 1Gi | | 200-500 | 2 | 2Gi | | 500+ | 4 | 4Gi | ## Backup & Recovery ### Database Backup ```bash theme={null} # Backup pg_dump "$DATABASE_URL" > backup.sql # Restore psql "$DATABASE_URL" < backup.sql ``` ### Critical Data Ensure backup of: * API keys table (encrypted) * Organizations and users * Billing/usage records * Audit logs (if enabled) ### Recovery Procedure 1. Deploy new controller instance 2. Restore database from backup 3. Verify connectivity 4. Run health checks 5. Re-enable traffic ## Security Operations ### Rotating Secrets **JWT Secret:** 1. Generate new secret 2. Update in Kubernetes secret 3. Restart controller (all JWTs invalidated) **Database Encryption Key:** 1. Cannot rotate without re-encrypting data 2. Plan data migration if needed **API Keys:** ```bash theme={null} # Revoke compromised key curl -X DELETE -H "Authorization: Bearer $ADMIN_KEY" \ http://localhost:8080/api/admin/api-keys/ ``` ### Audit Log Review If `AUDIT_LOGS_ENABLED=true`: ```sql theme={null} SELECT * FROM audit_logs WHERE action = 'sandbox.create' AND created_at > NOW() - INTERVAL '24 hours' ORDER BY created_at DESC; ``` ### Incident Response 1. **Contain**: Suspend affected organization 2. **Investigate**: Review audit logs and metrics 3. **Remediate**: Revoke keys, patch vulnerabilities 4. **Recover**: Restore normal operation 5. **Document**: Post-incident review ## Next steps Capacity planning and warm-pool profiles behind the metrics here. The controls whose audit trail feeds these alerts. The controller endpoints these health checks sit alongside. Where to wire the Prometheus and Grafana configuration for each cloud. # Sandbox Production Guide Source: https://docs.stateset.com/stateset-sandbox/stateset-sandbox-production-guide Running the StateSet Sandbox in production. # StateSet Sandbox - Production Guide ## Overview StateSet Sandbox is a secure, isolated execution environment for running code on behalf of AI agents. It provides containerized sandboxes with full lifecycle management, checkpointing, artifact storage, and usage tracking. *** ## Architecture ``` ┌─────────────────────────────────────────────────────────────────────────┐ │ DEVELOPER │ └─────────────────────────────────────────────────────────────────────────┘ │ │ │ │ 1. Register │ 2. Use SDK │ 3. Dashboard ▼ ▼ ▼ ┌─────────────────────────────────────────────────────────────────────────┐ │ SANDBOX CONTROLLER │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌─────────────┐ │ │ │ Registration │ │ Sandbox │ │ Checkpoint │ │ Webhook │ │ │ │ Service │ │ Manager │ │ Manager │ │ Manager │ │ │ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ └──────┬──────┘ │ └─────────┼─────────────────┼─────────────────┼─────────────────┼─────────┘ │ │ │ │ ▼ ▼ ▼ ▼ ┌─────────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────────┐ │ PostgreSQL │ │ Kubernetes │ │ Cloud │ │ External │ │ (Managed DB) │ │ (Pods) │ │ Storage │ │ Webhooks │ │ │ │ │ │ (GCS/S3) │ │ │ │ • organizations │ │ • sandbox │ │ • artifacts │ │ • your-app.com │ │ • api_keys │ │ pods │ │ • checkpts │ │ │ │ • usage_events │ │ │ │ │ │ │ │ • checkpoints │ │ │ │ │ │ │ └─────────────────┘ └─────────────┘ └─────────────┘ └─────────────────┘ ``` ### Component Details | Component | Technology | Purpose | | ---------- | -------------------- | --------------------------- | | Controller | Rust + Axum | API server, orchestration | | Database | PostgreSQL (managed) | Persistent storage | | Sandboxes | Kubernetes Pods | Isolated execution | | Storage | GCS/S3 | Artifact & checkpoint files | | Dashboard | Next.js 14 | Web UI for management | | SDK | TypeScript | Client library | ### Exec Agent Runtime The sandbox base image includes both Node.js and Go exec agents. Set `EXEC_AGENT_RUNTIME=go` for a lower memory footprint (\~8 MB vs \~45 MB RSS). The two runtimes are wire-compatible, so no controller changes are needed. *** ## Database Tables | Table | Purpose | | --------------------------- | ------------------------------ | | `organizations` | Customer accounts | | `users` | Organization members | | `api_keys` | Authentication tokens (hashed) | | `checkpoints` | Sandbox state snapshots | | `artifacts` | Uploaded/downloaded files | | `usage_events` | Per-operation usage records | | `usage_aggregates` | Rolled-up usage stats | | `vcpu_allocations` | CPU allocation tracking | | `billing_records` | Invoice line items | | `plans` | Subscription tiers | | `stripe_subscription_items` | Stripe integration | | `organization_secrets` | Encrypted secrets | | `secret_access_log` | Secret access audit | | `schema_migrations` | Migration tracking | *** ## User Flow ### 1. Registration Developer signs up via API or dashboard: ```bash theme={null} curl -X POST https://api.sandbox.stateset.app/api/v1/register \ -H "Content-Type: application/json" \ -d '{ "first_name": "Ada", "last_name": "Lovelace", "email": "dev@company.com", "organization_name": "Acme Corp", "password": "YourSecurePassword123!", "use_case": "CI automation" }' ``` **Response:** ```json theme={null} { "organization": { "id": "org_abc123", "name": "Acme Corp", "slug": "acme-corp", "plan": "hobby" }, "user": { "id": "user_abc123", "email": "dev@company.com", "role": "owner" }, "api_key": { "key": "sk-sandbox-xxxxxxxxxxxxxxxxxxxx", "key_prefix": "sk-sandbox-xxxx", "name": "Default API Key" } } ``` ### 2. SDK Installation Node.js: ```bash theme={null} npm install @stateset/sandbox-sdk ``` Python: ```bash theme={null} pip install stateset-sandbox ``` Additional preview SDKs are available in this repo: * Rust: `sdk-rust/README.md` * Ruby: `sdk-ruby/README.md` * Go: `sdk-go/README.md` * PHP: `sdk-php/README.md` * Java: `sdk-java/README.md` * Kotlin: `sdk-kotlin/README.md` * Swift: `sdk-swift/README.md` ### 3. Basic Usage ```typescript theme={null} import { StateSetSandbox } from '@stateset/sandbox-sdk'; // Initialize client const client = new StateSetSandbox({ baseUrl: 'https://api.sandbox.stateset.app', authToken: 'sk-sandbox-xxxxxxxxxxxx' }); // Create a sandbox const sandbox = await client.create({ timeout_seconds: 300, // 5 minutes cpus: '1', // 1 vCPU memory: '1Gi', // 1GB RAM env: { NODE_ENV: 'production' } }); console.log(`Sandbox created: ${sandbox.sandbox_id}`); console.log(`Expires at: ${sandbox.expires_at}`); // Execute commands const result = await client.execute(sandbox.sandbox_id, { command: ['node', '-e', 'console.log("Hello from sandbox!")'] }); console.log(result.stdout); // "Hello from sandbox!" console.log(result.exit_code); // 0 // Write files await client.writeFiles(sandbox.sandbox_id, [ { path: '/workspace/app.js', content: 'console.log("app");' } ]); // Read files const content = await client.readFile(sandbox.sandbox_id, '/workspace/app.js'); // Stop sandbox when done await client.stop(sandbox.sandbox_id); ``` Advanced endpoints (checkpoints, artifacts, webhooks, templates) are available via `StateSetSandboxExtended`: ```typescript theme={null} import { StateSetSandboxExtended } from '@stateset/sandbox-sdk'; const client = new StateSetSandboxExtended({ baseUrl: 'https://api.sandbox.stateset.app', authToken: 'sk-sandbox-xxxxxxxxxxxx' }); ``` ### 4. Advanced: Checkpoints Save and restore sandbox state: ```typescript theme={null} // Save current state const checkpoint = await client.createCheckpoint(sandbox.sandbox_id, { name: 'after-npm-install', description: 'All dependencies installed' }); console.log(`Checkpoint created: ${checkpoint.id}`); // Later: Restore to that state (same sandbox) await client.restoreCheckpoint(sandbox.sandbox_id, checkpoint.id, { restore_files: true, restore_env: true, overwrite: true }); // Clone a checkpoint const cloned = await client.cloneCheckpoint(checkpoint.id, 'feature-branch-base'); // Compare checkpoints const diff = await client.compareCheckpoints( checkpoint1.id, checkpoint2.id ); ``` ### 5. Advanced: Artifacts Upload and download files: ```typescript theme={null} // Upload file from sandbox to cloud storage const artifact = await client.uploadArtifact(sandbox.sandbox_id, { path: '/workspace/output/report.pdf', remote_path: 'reports/2024/report.pdf', // stored under artifacts//... content_type: 'application/pdf', expires_in: 86400 * 7 // 7 days }); // Download artifact to sandbox await client.downloadArtifact( sandbox.sandbox_id, artifact.id, '/workspace/downloads/report.pdf' ); // Get presigned URL for direct download const { url } = await client.getArtifactUrl(artifact.id, 3600); console.log(`Artifact URL: ${url}`); ``` ### 6. Advanced: Webhooks Get notified of sandbox events: ```typescript theme={null} // Register webhook const webhook = await client.createWebhook({ url: 'https://your-app.com/webhooks/sandbox', events: [ 'sandbox.created', 'sandbox.ready', 'sandbox.error', 'sandbox.stopped', 'command.completed', 'checkpoint.created' ], secret: 'whsec_your_secret_key' }); // Test webhook const testResult = await client.testWebhook(webhook.id); console.log(`Test successful: ${testResult.success}`); // List webhook deliveries const deliveries = await client.getWebhookDeliveries(webhook.id, 10); ``` **Webhook Payload:** ```json theme={null} { "id": "evt_abc123", "event": "sandbox.ready", "timestamp": "2024-01-15T12:00:00Z", "sandboxId": "sbx_xyz789", "orgId": "org_abc123", "data": { "podIp": "10.0.1.5", "startupMs": 1234 } } ``` **Signature Verification:** ```typescript theme={null} import { createHmac, timingSafeEqual } from 'crypto'; function verifyWebhook(payload: string, signature: string, secret: string): boolean { if (!signature?.startsWith('sha256=')) return false; const expected = createHmac('sha256', secret).update(payload).digest(); const provided = Buffer.from(signature.slice('sha256='.length), 'hex'); // Constant-time comparison: a plain === reveals how many leading bytes matched // through its timing, which is enough to forge a signature byte by byte. // timingSafeEqual throws when lengths differ, so compare lengths first. return provided.length === expected.length && timingSafeEqual(provided, expected); } ``` ### 7. Templates Create sandboxes from pre-configured templates: ```typescript theme={null} // List available templates const templates = await client.listTemplates(); // Create from template const sandbox = await client.createFromTemplate('python-data-science', { timeout_seconds: 600, env: { DATASET_URL: 'https://example.com/data.csv' } }); ``` **Available Templates:** | Template | Description | Pre-installed | | --------------------- | -------------- | ------------------------- | | `python-basic` | Python 3.11 | pip, venv | | `node-basic` | Node.js 22 | npm, yarn | | `go-basic` | Go 1.22 | go mod | | `rust-basic` | Rust stable | cargo | | `computer-use` | GUI automation | xdotool, Xvfb, Firefox | | `python-data-science` | Data analysis | pandas, numpy, matplotlib | | `node-express-api` | Web APIs | express, typescript | | `claude-agent` | AI agents | claude-code, MCP servers | ### 8. REPL Sessions Interactive Python REPL with persistent session state. Variables, imports, and objects survive across execute calls — ideal for data exploration, iterative development, and AI agent workflows. **Enable:** Set `REPL_ENABLED=true` on the controller. ```typescript theme={null} const session = await client.createReplSession(sandboxId, { language: 'python' }); const result = await client.executeRepl(sandboxId, session.id, { code: 'x = 42' }); // Variables persist across calls const result2 = await client.executeRepl(sandboxId, session.id, { code: 'print(x * 2)' }); await client.destroyReplSession(sandboxId, session.id); ``` *** ## API Reference ### Authentication All API requests require authentication via header: ``` Authorization: ApiKey sk-sandbox-xxxxxxxxxxxx ``` The organization is inferred from the API key or JWT claims; no org header is required. Or with JWT: ``` Authorization: Bearer eyJhbGciOiJIUzI1NiIs... ``` ### Endpoints #### Registration & API Keys | Method | Endpoint | Description | | ------ | ---------------------- | ----------------------------- | | POST | `/api/v1/register` | Create organization + API key | | GET | `/api/v1/api-keys` | List API keys | | POST | `/api/v1/api-keys` | Create new API key | | DELETE | `/api/v1/api-keys/:id` | Revoke API key | #### Sandboxes | Method | Endpoint | Description | | ------ | ------------------------------------ | -------------------------------- | | POST | `/api/v1/sandbox/create` | Create sandbox | | GET | `/api/v1/sandbox/:id` | Get sandbox details | | GET | `/api/v1/sandbox/:id/status` | Get sandbox status (lightweight) | | POST | `/api/v1/sandbox/:id/execute` | Execute command | | POST | `/api/v1/sandbox/:id/files` | Write files | | GET | `/api/v1/sandbox/:id/files?path=...` | Read file | | POST | `/api/v1/sandbox/:id/stop` | Stop sandbox | | DELETE | `/api/v1/sandbox/:id` | Delete sandbox | | GET | `/api/v1/sandboxes` | List sandboxes | #### Checkpoints | Method | Endpoint | Description | | ------ | ----------------------------------------- | ------------------- | | GET | `/api/v1/checkpoints` | List checkpoints | | GET | `/api/v1/checkpoints/:id` | Get checkpoint | | POST | `/api/v1/sandbox/:id/checkpoints` | Create checkpoint | | POST | `/api/v1/sandbox/:id/checkpoints/restore` | Restore checkpoint | | POST | `/api/v1/checkpoints/:id/clone` | Clone checkpoint | | POST | `/api/v1/checkpoints/compare` | Compare checkpoints | | DELETE | `/api/v1/checkpoints/:id` | Delete checkpoint | #### Artifacts | Method | Endpoint | Description | | ------ | ---------------------------------------- | -------------------- | | GET | `/api/v1/artifacts` | List artifacts | | GET | `/api/v1/artifacts/:id` | Get artifact details | | GET | `/api/v1/artifacts/:id/url` | Get presigned URL | | POST | `/api/v1/sandbox/:id/artifacts/upload` | Upload artifact | | POST | `/api/v1/sandbox/:id/artifacts/download` | Download artifact | | DELETE | `/api/v1/artifacts/:id` | Delete artifact | #### Webhooks | Method | Endpoint | Description | | ------ | --------------------------------- | -------------------- | | GET | `/api/v1/webhooks` | List webhooks | | POST | `/api/v1/webhooks` | Create webhook | | DELETE | `/api/v1/webhooks/:id` | Delete webhook | | POST | `/api/v1/webhooks/:id/test` | Test webhook | | GET | `/api/v1/webhooks/:id/deliveries` | Get delivery history | #### Usage & Billing | Method | Endpoint | Description | | ------ | ------------------------------- | ----------------------- | | GET | `/api/v1/usage/current` | Get usage summary | | GET | `/api/v1/usage/history` | Get usage history | | GET | `/api/v1/usage/sandboxes` | Get usage by sandbox | | GET | `/api/v1/pricing` | Get pricing info | | POST | `/api/v1/pricing/estimate` | Estimate cost | | POST | `/api/v1/pricing/bulk-estimate` | Bulk estimate costs | | GET | `/api/v1/pricing/plans` | List subscription plans | | GET | `/api/v1/billing/subscription` | Get subscription status | | POST | `/api/v1/billing/upgrade` | Upgrade plan | | GET | `/api/v1/billing/invoices` | List invoices | | POST | `/api/v1/billing/portal` | Open billing portal | #### Templates | Method | Endpoint | Description | | ------ | -------------------------------------- | -------------------- | | GET | `/api/v1/templates` | List templates | | GET | `/api/v1/templates/:id` | Get template details | | GET | `/api/v1/templates/categories` | List categories | | POST | `/api/v1/sandbox/create-from-template` | Create from template | #### REPL Sessions | Method | Endpoint | Description | | ------ | ------------------------------------------------------ | ------------------- | | POST | `/api/v1/sandbox/:id/repl/sessions` | Create REPL session | | GET | `/api/v1/sandbox/:id/repl/sessions` | List REPL sessions | | POST | `/api/v1/sandbox/:id/repl/sessions/:sessionId/execute` | Execute code | | DELETE | `/api/v1/sandbox/:id/repl/sessions/:sessionId` | Destroy session | *** ## Webhook Events | Event | Description | | --------------------- | ----------------------------- | | `sandbox.created` | Sandbox pod created | | `sandbox.ready` | Sandbox is ready for commands | | `sandbox.stopped` | Sandbox stopped | | `sandbox.error` | Sandbox encountered error | | `sandbox.timeout` | Sandbox timed out | | `command.started` | Command execution started | | `command.completed` | Command finished successfully | | `command.failed` | Command failed | | `file.written` | File written to sandbox | | `checkpoint.created` | Checkpoint created | | `checkpoint.restored` | Checkpoint restored | | `checkpoint.cloned` | Checkpoint cloned | | `checkpoint.deleted` | Checkpoint deleted | | `artifact.uploaded` | Artifact uploaded | | `artifact.deleted` | Artifact deleted | | `resource.warning` | Resource usage warning | | `resource.critical` | Resource usage critical | | `mcp.started` | MCP server started | | `mcp.stopped` | MCP server stopped | *** ## Pricing ### Plans | Plan | Price | Sandboxes/Month | Concurrent | Max Duration | Storage | | ---------- | ------- | --------------- | ---------- | ------------ | ------- | | Free | \$0 | 50 | 2 | 5 min | 1 GB | | Pro | \$29/mo | 1,000 | 10 | 1 hour | 50 GB | | Team | \$99/mo | 10,000 | 50 | 2 hours | 500 GB | | Enterprise | Custom | Unlimited | Custom | Custom | Custom | ### Usage-Based Pricing | Resource | Rate | | -------- | ------------------ | | CPU | \$0.05 / vCPU-hour | | Memory | \$0.02 / GB-hour | | Network | \$0.10 / GB egress | | Storage | \$0.20 / GB-month | *** ## Dashboard Access at: `https://sandbox.stateset.com/dashboard` ### Pages | Page | Path | Description | | ----------- | ------------------------ | ------------------------- | | Overview | `/dashboard` | Active sandboxes, metrics | | Sandboxes | `/dashboard/sandboxes` | List and manage sandboxes | | API Keys | `/dashboard/api-keys` | Create/revoke keys | | Checkpoints | `/dashboard/checkpoints` | Manage checkpoints | | Artifacts | `/dashboard/artifacts` | File storage | | Webhooks | `/dashboard/webhooks` | Event notifications | | Audit Logs | `/dashboard/audit-logs` | Activity history | | Usage | `/dashboard/usage` | Usage and billing | | Settings | `/dashboard/settings` | Organization settings | *** ## Security ### API Key Security * Keys are hashed (SHA-256) before storage * Original key shown only once at creation * Keys can have scopes and expiration * Rate limiting per key ### Sandbox Isolation * Each sandbox runs in isolated Kubernetes pod * Network policies restrict pod communication * Resource limits enforced (CPU, memory) * Read-only root filesystem * Non-root user execution ### Data Protection * All data encrypted in transit (TLS) * Database encrypted at rest * Secrets stored encrypted with KMS * Audit logging for compliance *** ## Environment Variables ### Controller Configuration | Variable | Description | Default | | ------------------------------- | -------------------------------------------------- | -------------------- | | `DATABASE_URL` | PostgreSQL connection string | Required | | `AUTO_MIGRATE` | Run database migrations on startup (`true` or `1`) | `false` | | `JWT_SECRET` | Secret for JWT signing | Required | | `NAMESPACE` | Kubernetes namespace | Deployment namespace | | `SANDBOX_IMAGE` | Docker image for sandboxes | Required | | `DEFAULT_CPUS` | Default CPU allocation | `0.5` | | `DEFAULT_MEMORY` | Default memory allocation | `512Mi` | | `DEFAULT_TIMEOUT` | Default timeout (seconds) | `300` | | `MAX_SANDBOXES_PER_ORG` | Concurrent sandbox limit | `5` | | `SANDBOX_EXEC_BACKEND` | Exec backend (`k8s` or `kubectl`) | `kubectl` | | `STRIPE_SECRET_KEY` | Stripe API key for billing | Optional | | `STORAGE_PROVIDER` | `local`, `s3`, `gcs`, or `azure` | `local` | | `STORAGE_BUCKET` | Cloud storage bucket name | Required if cloud | | `EXEC_AGENT_RUNTIME` | Exec agent runtime (`node` or `go`) | `node` | | `SANDBOX_BACKEND` | Sandbox backend (`imperative` or `crd`) | `imperative` | | `REPL_ENABLED` | Enable REPL sessions | `false` | | `REPL_JUPYTER_PORT` | Jupyter port inside sandbox | `8888` | | `REPL_SESSION_TIMEOUT_SECONDS` | Session idle timeout | `3600` | | `REPL_MAX_SESSIONS_PER_SANDBOX` | Max sessions per sandbox | `5` | *** ## Metrics Access `/metrics` is protected in production. Configure your monitoring stack to include the auth header you use for metrics access. If you use Prometheus Operator, apply `k8s/prometheus/service-monitor.yaml` and add your metrics auth header to the scrape config: ```bash theme={null} kubectl apply -f k8s/prometheus/service-monitor.yaml ``` If you keep network policies enabled, label the monitoring namespace to allow scraping: ```bash theme={null} kubectl label namespace monitoring your-domain.com/monitoring=true ``` *** ## Warm Pool Profiles Use warm pool profiles to pre‑warm the most common CPU/memory shapes and reduce cold starts. Profiles are matched on `cpus` + `memory` with optional `isolation`. Example configuration: ``` WARM_POOL_ENABLED=true WARM_POOL_PROFILES='[ {"cpus":"1","memory":"2Gi","size":6,"isolation":"container"}, {"cpus":"2","memory":"2Gi","size":4,"isolation":"microvm"} ]' ``` If you pass `isolation` on sandbox creation, ensure the matching warm pool profile sets the same `isolation`. Internal stats endpoint (production requires `X-Internal-Token`): ```bash theme={null} curl -H "X-Internal-Token: $INTERNAL_API_KEY" \ "https://api.sandbox.yourdomain.com/api/v1/pool/stats?cpus=1&memory=2Gi&isolation=microvm" ``` *** ## Tunnel Configuration Tunnels expose a sandbox port through the API gateway. Optional settings: * `TUNNEL_BASE_URL=https://api.sandbox.yourdomain.com` * `TUNNEL_DEFAULT_TTL_SECONDS=3600` * `TUNNEL_MAX_TTL_SECONDS=86400` ## Artifact Storage (Cloud) ### Google Cloud Storage (GCS) Required settings: * `STORAGE_PROVIDER=gcs` * `STORAGE_BUCKET=your-bucket` `remote_path` is treated as a relative path and always stored under `artifacts//`. If you are not using Workload Identity, mount a service account key and set: * `GCS_PROJECT_ID=your-gcp-project-id` * `GCS_KEY_FILE=/var/secrets/gcp/key.json` Example secret creation: ```bash theme={null} kubectl create secret generic gcs-credentials \ --from-file=key.json=/path/to/service-account.json \ -n your-namespace ``` Mount the secret and set env vars in your controller deployment: ```yaml theme={null} env: - name: GCS_PROJECT_ID value: your-gcp-project-id - name: GCS_KEY_FILE value: /var/secrets/gcp/key.json volumeMounts: - name: gcs-credentials mountPath: /var/secrets/gcp readOnly: true volumes: - name: gcs-credentials secret: secretName: gcs-credentials ``` ### Azure Blob Storage (SAS) Required settings: * `STORAGE_PROVIDER=azure` * `STORAGE_BUCKET=your-container` * `AZURE_STORAGE_ACCOUNT=your-account` * `AZURE_STORAGE_SAS_TOKEN=sv=...&ss=b&srt=co&sp=rwld&se=...` Recommended: store `AZURE_STORAGE_SAS_TOKEN` in a Kubernetes Secret and inject it via `env` or `envFrom`. Optional: * `AZURE_STORAGE_ENDPOINT=https://your-account.blob.core.windows.net` (sovereign/private clouds) Verify artifacts end-to-end: ```bash theme={null} STATESET_API_URL=https://api.sandbox.stateset.app \ STATESET_API_KEY=sk_sandbox_xxx \ ./scripts/verify-gcs-artifacts.sh ``` *** ## Deployment ### Kubernetes Resources ```yaml theme={null} # Namespace apiVersion: v1 kind: Namespace metadata: name: your-namespace # Controller Deployment apiVersion: apps/v1 kind: Deployment metadata: name: sandbox-controller namespace: your-namespace spec: replicas: 3 template: spec: serviceAccountName: sandbox-controller containers: - name: controller image: REGISTRY_URL/sandbox-controller:latest ports: - containerPort: 8080 env: - name: DATABASE_URL valueFrom: secretKeyRef: name: sandbox-db-url key: url - name: db-proxy image: DB_PROXY_IMAGE args: - "--structured-logs" - "--port=5432" - "YOUR_DATABASE_CONNECTION" ``` ### Custom Resource Definitions (CRDs) Apply CRDs before deploying: ```bash theme={null} kubectl apply -f k8s/crds/ ``` Optional: Set `SANDBOX_BACKEND=crd` for declarative sandbox management. This enables `kubectl get ssb` (sandboxes) and `kubectl get wp` (warm pool entries). ### Health Checks * Liveness: `GET /health` * Readiness: `GET /ready` *** ## Troubleshooting ### Common Issues **401 Unauthorized** * Check API key format: `ApiKey sk-sandbox-xxx` * Verify key exists in the database and is not revoked **Sandbox Creation Failed** * Check Kubernetes connectivity * Verify namespace exists * Check resource quotas **Database Connection Failed** * Verify database proxy is running (if used) * Check workload identity / service account bindings * Verify database credentials ### Logs ```bash theme={null} # Controller logs kubectl logs -n your-namespace -l app=sandbox-controller -c controller # Database proxy logs kubectl logs -n your-namespace -l app=sandbox-controller -c db-proxy # Sandbox pod logs kubectl logs -n your-namespace ``` *** ## Support * GitHub Issues: [https://github.com/stateset/sandbox/issues](https://github.com/stateset/sandbox/issues) * Documentation: [https://docs.stateset.io/sandbox](https://docs.stateset.io/sandbox) * Email: [support@stateset.io](mailto:support@stateset.io) ## Next steps Health, metrics, alerting and runbooks. Isolation and hardening before you take real traffic. Provider-specific cluster setup. What warm-pool profiles cost, and how they trade start-up time for spend. # Sandbox Quickstart Source: https://docs.stateset.com/stateset-sandbox/stateset-sandbox-quickstart-guide Create one sandbox on a provisioned deployment, verify command output, and confirm teardown. Run one command in an isolated runtime, keep its execution result, and stop the runtime when finished. This walkthrough uses the controller’s HTTP contract directly. **A provisioned deployment is required.** Check the dated [host report](/api-reference/hosts) and obtain your workspace URL and API key before starting. A published reference does not establish that the hosted service is available or that your workspace has runtime capacity. ## Prerequisites Use Bash, `curl` **7.76+**, and `jq`. Obtain an API key with permission to create, inspect, execute in, and stop a sandbox. Confirm the deployment’s supported isolation modes and resource limits. This example requests a Linux container for five minutes; runtime and quota policies can reject the request. Set these in the same terminal, replacing the deployment URL with the one supplied to you: ```bash theme={null} export SANDBOX_URL="https://YOUR-PROVISIONED-HOST/api/v1" export STATESET_SANDBOX_API_KEY="YOUR_SANDBOX_API_KEY" ``` API keys use **`Authorization: ApiKey`**. The controller also accepts `Bearer` for a JWT; putting an API key after `Bearer` does not turn it into a JWT. The source-backed correction is recorded in this repository’s [Sandbox overlay](https://github.com/stateset/docs/blob/main/spec/overlays/sandbox.json). ## Create, execute, and stop Save this as `sandbox-smoke.sh`. It captures the returned ID, polls with a finite budget, verifies the command result, and attempts cleanup on exit once an ID is known. ```bash theme={null} #!/usr/bin/env bash set -euo pipefail : "${SANDBOX_URL:?Set the provisioned base URL including /api/v1}" : "${STATESET_SANDBOX_API_KEY:?Set the sandbox API key}" SANDBOX_URL="${SANDBOX_URL%/}" request() { curl --silent --show-error --fail-with-body --max-time 30 \ --header "Authorization: ApiKey $STATESET_SANDBOX_API_KEY" "$@" } SANDBOX_ID="" cleanup() { if [ -n "$SANDBOX_ID" ]; then if ! request --request POST "$SANDBOX_URL/sandbox/$SANDBOX_ID/stop" > sandbox-stop.json; then printf 'Cleanup failed. Inspect sandbox %s and stop it manually.\n' "$SANDBOX_ID" >&2 return 1 fi printf 'Stop request accepted for %s; verify teardown below.\n' "$SANDBOX_ID" fi } trap cleanup EXIT request --request POST "$SANDBOX_URL/sandbox/create" \ --header 'Content-Type: application/json' \ --data '{"desktop_os":"linux","cpus":"1","memory":"1Gi","isolation":"container","timeout_seconds":300}' \ > sandbox-created.json SANDBOX_ID=$(jq -er '.sandbox_id | select(type == "string" and length > 0)' sandbox-created.json) printf '%s\n' "$SANDBOX_ID" > sandbox-id.txt ready=false for attempt in {1..30}; do request "$SANDBOX_URL/sandbox/$SANDBOX_ID/status" > sandbox-status.json status=$(jq -er '.status' sandbox-status.json) case "$status" in running) ready=true; break ;; creating) sleep 2 ;; *) printf 'Unexpected sandbox state: %s\n' "$status" >&2; exit 1 ;; esac done if [ "$ready" != true ]; then printf 'Sandbox did not become ready within 30 observations.\n' >&2 exit 1 fi request --request POST "$SANDBOX_URL/sandbox/$SANDBOX_ID/execute" \ --header 'Content-Type: application/json' \ --data '{"command":["printf","sandbox-ok"],"working_dir":"/workspace","timeout":10000}' \ > sandbox-execution.json jq -e '.exit_code == 0 and .stdout == "sandbox-ok"' sandbox-execution.json printf 'Command output verified for sandbox %s\n' "$SANDBOX_ID" ``` ```bash theme={null} bash sandbox-smoke.sh ``` **Success:** the script prints a real sandbox ID, observes `running`, verifies exit code `0` and output `sandbox-ok`, and receives a successful stop response. It saves the responses locally for inspection. These are observed criteria, not guarantees of provisioning speed. If creation times out before returning an ID, inspect your workspace’s sandbox list before creating another runtime. The cleanup trap cannot stop a resource whose identity it never received. ## Verify teardown After the script exits, read the status with the saved ID: ```bash theme={null} SANDBOX_ID=$(cat sandbox-id.txt) curl --silent --show-error --max-time 30 \ --write-out '\nHTTP %{http_code}\n' \ --header "Authorization: ApiKey $STATESET_SANDBOX_API_KEY" \ "$SANDBOX_URL/sandbox/$SANDBOX_ID/status" ``` A terminal `terminated` state or `404` with the same workspace credentials establishes that the runtime is no longer accessible at that identity. `terminating` is still in progress; recheck within your operational deadline. A `401`, `403`, network error, or `5xx` does not prove teardown. Confirm with the workspace operator if the result remains uncertain. ## Troubleshooting | Failure | Next action | | -------------------------- | --------------------------------------------------------------------------- | | `401` | Check the service-specific key and `ApiKey` prefix | | `403` | Check scope, organization membership, and sandbox permissions | | Creation rejected | Read field errors and confirm supported resources and isolation mode | | `429` | Inspect quota and existing runtimes before creating another | | Readiness budget exhausted | Inspect the saved status and provisioning logs | | Command returns nonzero | Read `sandbox-execution.json`; do not treat HTTP success as command success | | Stop fails | Use `sandbox-id.txt` to inspect and stop the same runtime; retain the error | ## Next steps * [Files and longer tasks](/guides/sandbox-first-runtime) — extend the verified runtime workflow. * [Sandbox API](/api-reference/sandbox/overview) — schemas, lifetime, execution, and files. * [Sandbox MCP](/stateset-sandbox/stateset-sandbox-mcp) — connect an agent to its runtime tools. * [Support](/support#report-an-api-problem) — share the failed step and redacted result. # Runtime Selection Source: https://docs.stateset.com/stateset-sandbox/stateset-sandbox-runtime-selection Choosing between WebAssembly, gVisor, and Kata — what each isolation boundary costs and when to pay for it. The sandbox offers three isolation runtimes. They trade the same three things against each other: **how strong the boundary is**, **how fast a sandbox starts**, and **how much of Linux the workload gets**. If you don't choose, you get **gVisor**. It is the right default for agentic workloads — real Linux compatibility with a syscall boundary, at low startup cost. Read on only if your workload is unusually short, unusually hostile, or unusually compliance-bound. ## At a glance | | **WebAssembly** | **gVisor** | **Kata Containers** | | ------------------- | ------------------ | ------------------------------------- | ---------------------- | | Boundary | Wasm VM (no OS) | User-space kernel intercepts syscalls | Hardware VM | | Shares host kernel | No kernel at all | Yes, via interception | **No** | | Startup | Milliseconds | Fast | Slowest | | Density per node | Thousands | High | Lowest | | Linux compatibility | WASI subset only | Feels like Linux | Full | | Cost | Lowest | Low | Highest | | Runtime name | `wasmedge`, `spin` | `runsc` | `kata-qemu`, `kata-fc` | ## Choosing ``` Untrusted third-party binary, needs root, or VM isolation required for compliance? └── yes ──▶ Kata Containers Pure compute, sub-second, already compiles to .wasm? └── yes ──▶ WebAssembly Otherwise ─────▶ gVisor (default) ``` ### WebAssembly Code runs in a lightweight stack-based virtual machine rather than a container — there is no OS to boot, which is where the millisecond startup comes from. **Use it for** ultra-short tasks (under a second), pure logic and compute — validating a JSON schema, evaluating an expression, a numeric transform — and for scale where startup time dominates total execution time. **The cost:** WASI only. No full Linux syscall surface, no arbitrary processes, no `git` or `npm`. Your workload must be compiled to `.wasm`, or run inside a Wasm-compiled interpreter (Python or JS in Wasm), which is a real build-pipeline commitment. ### gVisor A user-space kernel — the "Sentry" — intercepts syscalls, so the container never talks directly to the host kernel. Startup is fast and memory overhead is low, while `bash`, the filesystem, and compilers all behave normally. **Use it for** standard CLI coding agents, workloads that need ordinary Linux tooling (`git`, `curl`, `grep`, `npm`), MCP-connected toolchains, and any "do work → write files → exit" flow. gVisor is a strong sandbox but it still **shares the host kernel**, reached through interception rather than directly. Kata's VM boundary is a categorically different guarantee. If your threat model includes a determined attacker with a kernel exploit, gVisor is not the answer. **The cost:** some syscalls are emulated or restricted. Workloads that probe unusual kernel interfaces can behave differently than on bare Linux. ### Kata Containers Each container runs inside its own lightweight VM using hardware virtualization. No shared kernel — the strongest boundary available. **Use it for** high-risk untrusted customer binaries, workloads that need root and might attempt to escape a container, and compliance regimes that specifically require VM-level isolation. **The cost:** higher startup latency, more memory and CPU per sandbox, lower density. You are trading velocity and money for a boundary you may not need. ### Selecting one `isolation` on a create request picks the profile; the boundary itself comes from the RuntimeClass the cluster has configured for it. ```bash theme={null} curl --request POST "https://api.sandbox.stateset.app/api/v1/sandbox/create" \ --header "Authorization: ApiKey $STATESET_SANDBOX_API_KEY" \ --header "Content-Type: application/json" \ --data '{ "isolation": "microvm", "cpus": 2, "memory": "4Gi", "timeout_seconds": 600 }' ``` On a self-hosted cluster the mapping is a controller setting, not a per-request one: ```bash theme={null} # Every sandbox on this controller runs under gVisor SANDBOX_RUNTIME_CLASS=runsc ``` Asking for `microvm` does not give you a MicroVM unless the cluster has a Kata RuntimeClass configured and named in `SANDBOX_RUNTIME_CLASS`. The request-level field selects a warm-pool profile and labels the pod; the boundary is cluster-wide. Check the controller's setting before assuming a workload is VM-isolated — see the [security guide](/stateset-sandbox/stateset-sandbox-security-guide). ## The tiered model Rather than standardising on one runtime, route by workload class: | Class | Runtime | Rationale | | ----------------------- | ------------------------------ | ------------------------------------------------- | | Ultra-fast compute | WebAssembly (WasmEdge / Spin) | Startup dominates; massive scale for simple tasks | | General-purpose agentic | gVisor (`runsc`) | Developer-friendly compatibility, low overhead | | High-security zone | Kata (`kata-qemu` / `kata-fc`) | Hardened isolation for hostile code | Pick per workload, not per platform. A single deployment can run all three: Wasm for a validation function, gVisor for the coding agent that calls it, Kata for the one customer whose contract requires VM isolation. ## Related * [Sandboxes](/stateset-sandbox/stateset-sandboxes) — what the sandbox is for * [Architecture](/stateset-sandbox/stateset-sandbox-architecture) — how the runtimes are wired in * [Security Guide](/stateset-sandbox/stateset-sandbox-security-guide) — the full threat model * [Production Guide](/stateset-sandbox/stateset-sandbox-production-guide) # Sandbox SDK (Node.js) Source: https://docs.stateset.com/stateset-sandbox/stateset-sandbox-sdk-node Node.js SDK usage for creating and managing sandboxes. Create sandboxes, run commands in them, and move files in and out. Get an API key at [sandbox.stateset.com](https://sandbox.stateset.com). ## Install ```bash theme={null} npm install @stateset/sandbox-sdk ``` ## Initialize the Client ```typescript theme={null} import { StateSetSandbox } from '@stateset/sandbox-sdk'; const client = new StateSetSandbox({ baseUrl: 'https://api.sandbox.stateset.app', authToken: 'sk-sandbox-YOUR_API_KEY', orgId: 'org_YOUR_ORG_ID' }); ``` ## Create and Execute ```typescript theme={null} const sandbox = await client.create({ timeout_seconds: 300 }); const result = await client.execute(sandbox.sandbox_id, { command: ['python3', '-c', 'print("Hello from Stateset!")'] }); ``` ## File Operations ```typescript theme={null} await client.writeFiles(sandbox.sandbox_id, [ { path: '/workspace/hello.txt', content: Buffer.from('Hello!').toString('base64') } ]); const file = await client.readFiles(sandbox.sandbox_id, ['/workspace/hello.txt']); ``` File content crosses the wire as **base64** in both directions, as above — sending raw text writes the literal characters of your string rather than the bytes you meant. ## Cleanup ```typescript theme={null} await client.stop(sandbox.sandbox_id); ``` A sandbox runs, and bills, until it is stopped or `timeout_seconds` expires. Stop it in a `finally` block so a thrown error does not leave one up: ```typescript theme={null} const sandbox = await client.create({ timeout_seconds: 300 }); try { const result = await client.execute(sandbox.sandbox_id, { command: ['python3', '-c', 'print("hello")'], }); console.log(result.stdout); } finally { await client.stop(sandbox.sandbox_id); } ``` ## Handling failures `execute` resolves for a command that ran and failed — a non-zero exit is a result, not an exception. Check `exit_code` yourself; only transport and API errors reject. ```typescript theme={null} const result = await client.execute(sandbox.sandbox_id, { command: ['npm', 'run', 'build'], }); if (result.exit_code !== 0) { throw new Error(`build failed (${result.exit_code}): ${result.stderr}`); } ``` ## Next steps The wider flow, including REPL sessions and checkpoints. Every endpoint this SDK wraps. The same surface for Python services. What a sandboxed process can and cannot reach. # Sandbox SDK (Python) Source: https://docs.stateset.com/stateset-sandbox/stateset-sandbox-sdk-python Python SDK usage for creating and managing sandboxes. Use the Python SDK to create sandboxes, execute commands, and manage files. ## Install ```bash theme={null} pip install stateset-sandbox ``` ## Initialize the Client ```python theme={null} from stateset_sandbox import SandboxClient client = SandboxClient( base_url="https://api.sandbox.stateset.app", api_key="sk-sandbox-YOUR_API_KEY", org_id="org_YOUR_ORG_ID", ) ``` ## Create and Execute ```python theme={null} sandbox = client.create(timeout_seconds=300) result = client.execute( sandbox_id=sandbox["sandbox_id"], command=["python3", "-c", "print('Hello from Stateset!')"], ) ``` ## File Operations ```python theme={null} client.write_files( sandbox_id=sandbox["sandbox_id"], files=[{"path": "/workspace/hello.txt", "content": "SGVsbG8h", "encoding": "base64"}], ) file = client.read_files(sandbox_id=sandbox["sandbox_id"], paths=["/workspace/hello.txt"]) ``` File content crosses the wire as **base64** in both directions — `SGVsbG8h` above is `Hello!`. Passing raw text writes the literal characters of your string rather than the bytes you meant. ## Cleanup ```python theme={null} client.stop(sandbox_id=sandbox["sandbox_id"]) ``` A sandbox runs, and bills, until it is stopped or `timeout_seconds` expires. Stop it in a `finally` block so a raised exception does not leave one up: ```python theme={null} sandbox = client.create(timeout_seconds=300) try: result = client.execute( sandbox_id=sandbox["sandbox_id"], command=["python3", "-c", "print('hello')"], ) print(result["stdout"]) finally: client.stop(sandbox_id=sandbox["sandbox_id"]) ``` ## Handling failures `execute` returns for a command that ran and failed — a non-zero exit is a result, not an exception. Check `exit_code` yourself; only transport and API errors raise. ```python theme={null} result = client.execute( sandbox_id=sandbox["sandbox_id"], command=["npm", "run", "build"], ) if result["exit_code"] != 0: raise RuntimeError(f"build failed ({result['exit_code']}): {result['stderr']}") ``` ## Next steps The wider flow, including REPL sessions and checkpoints. Every endpoint this SDK wraps. The same surface for Node services. What a sandboxed process can and cannot reach. # Sandbox Security Guide Source: https://docs.stateset.com/stateset-sandbox/stateset-sandbox-security-guide Isolation models, key handling, and hardening for the sandbox. # Security Best Practices Guide This guide covers security considerations for deploying and using StateSet Sandbox, including isolation options, secret management, network policies, and compliance. ## Security Architecture ``` ┌──────────────────────────────────────────────────────────────────────────┐ │ API Gateway │ │ ┌─────────────────────────────────────────────────────────────────────┐ │ │ │ TLS Termination │ Rate Limiting │ API Key Auth │ CORS │ WAF │ │ │ └─────────────────────────────────────────────────────────────────────┘ │ └──────────────────────────────────────────────────────────────────────────┘ │ ▼ ┌──────────────────────────────────────────────────────────────────────────┐ │ Controller │ │ ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌───────────────────┐ │ │ │ Input │ │ Auth/Authz │ │ Secret │ │ Audit Logging │ │ │ │ Validation │ │ │ │ Management │ │ │ │ │ └────────────┘ └────────────┘ └────────────┘ └───────────────────┘ │ └──────────────────────────────────────────────────────────────────────────┘ │ ▼ ┌──────────────────────────────────────────────────────────────────────────┐ │ Sandbox Isolation │ │ ┌─────────────────────┐ ┌─────────────────────────────────────────┐ │ │ │ Container │ │ gVisor / Kata / Firecracker │ │ │ │ ├─ seccomp │ │ ├─ Separate kernel/VM per sandbox │ │ │ │ ├─ AppArmor │ │ ├─ Memory isolation │ │ │ │ ├─ Resource limits │ │ └─ Full syscall filtering │ │ │ │ └─ Network policies │ │ │ │ │ └─────────────────────┘ └─────────────────────────────────────────┘ │ └──────────────────────────────────────────────────────────────────────────┘ ``` ## Isolation Levels Isolation is enforced at the Kubernetes pod level via RuntimeClass (and your cluster's configured runtimes). * **Container**: default runtime (no RuntimeClass) * **gVisor**: a gVisor RuntimeClass (often named `gvisor`) * **MicroVM**: a Kata/Firecracker RuntimeClass (often named `kata-qemu`) In the Node controller, `SANDBOX_RUNTIME_CLASS` is applied to all sandbox pods: ```bash theme={null} # Example: run sandboxes under gVisor SANDBOX_RUNTIME_CLASS=gvisor ``` The sandbox create API accepts an `isolation` field (`container` | `gvisor` | `microvm`) which is currently used for warm pool profile matching and labeling; the actual isolation boundary is provided by the RuntimeClass. Setting `isolation: "microvm"` on the create API does **not** give you MicroVM isolation. That field selects a warm-pool profile and labels the pod; the boundary itself comes from the RuntimeClass named in `SANDBOX_RUNTIME_CLASS`, cluster-wide. If that variable is unset, every sandbox runs under the default container runtime no matter what the API call asked for — check the value on the controller, not the request. ## API Key Security ### Scopes A key carries scopes, and every mutating route checks them. There are four: | Scope | Grants | | ---------------- | --------------------------------------------------------------------- | | `sandbox:read` | Reading sandboxes, files, logs and metrics | | `sandbox:create` | Creating sandboxes | | `sandbox:write` | Everything that changes a sandbox — execute, files, sessions, secrets | | `sandbox:*` | All of the above | ```bash theme={null} # A key that can run work but never provision new sandboxes curl --request POST "https://api.sandbox.stateset.app/api/v1/api-keys" \ --header "Authorization: ApiKey $API_KEY" \ --header "Content-Type: application/json" \ --data '{ "name": "ci-runner", "scopes": ["sandbox:read", "sandbox:write"], "expires_in_days": 90 }' ``` **A typo in a scope name produces a full-access key.** Unrecognised scopes are dropped rather than rejected, and a request whose scopes are *all* dropped falls back to the default — which is `sandbox:*`. So `{"scopes": ["sandbox:reed"]}` does not fail and does not create a read-only key; it creates an unrestricted one. Read the `scopes` array back from the create response and assert it is what you asked for. That check is one line and it is the only thing standing between a typo and an unrestricted credential. Scope checks apply to **API keys only**. A request authenticated with a user session passes every scope check, by design — scopes narrow machine credentials, not people. Do not model human permissions with them; that is what the per-sandbox roles above are for. ### Key Management 1. **Generate strong keys**: ```bash theme={null} openssl rand -base64 32 ``` 2. **Use environment variables** (never hardcode): ```bash theme={null} export STATESET_API_KEY="sk_..." ``` 3. **Rotate keys regularly**: ```bash theme={null} # Create new key curl -X POST https://api.sandbox.stateset.app/api/v1/api-keys \ -H "Authorization: ApiKey $OLD_KEY" \ -d '{"name": "production-v2"}' # Update your systems with new key # Revoke old key curl -X DELETE https://api.sandbox.stateset.app/api/v1/api-keys/$OLD_KEY_ID \ -H "Authorization: ApiKey $NEW_KEY" ``` 4. **Use scoped keys** when possible: * Read-only keys for monitoring * Limited keys for specific sandboxes ### Key Storage | Environment | Recommendation | | ----------- | --------------------------------------------------------------- | | Development | `.env` file (gitignored) | | CI/CD | Pipeline secrets (GitHub Actions, GitLab CI) | | Production | Secret manager (Vault, AWS Secrets Manager, GCP Secret Manager) | | Kubernetes | Kubernetes Secrets (encrypted at rest) | ## Secret Management StateSet stores secrets at the organization level and injects them into sandbox pods as environment variables at creation time. ### Create / Update Secrets (BYOK) ```bash theme={null} # Create a secret (values are never returned after creation) curl -X POST https://api.sandbox.stateset.app/api/v1/secrets \ -H "Authorization: ApiKey $API_KEY" \ -H "Content-Type: application/json" \ -d '{ "name": "ANTHROPIC_API_KEY", "value": "sk-ant-REDACTED", "scope": "sandbox" }' # Update a secret curl -X PUT https://api.sandbox.stateset.app/api/v1/secrets/ANTHROPIC_API_KEY \ -H "Authorization: ApiKey $API_KEY" \ -H "Content-Type: application/json" \ -d '{ "value": "sk-ant-REDACTED" }' ``` Security properties: * Encrypted at rest in Postgres using AES-256-GCM (`DATABASE_ENCRYPTION_KEY` is required in production when `DATABASE_URL` is set) * Never logged or returned by list endpoints * Persist until deleted (not tied to a single sandbox lifetime) * Can be restricted to specific sandboxes using `allowed_sandbox_patterns` ### Secret Access Logging Enable audit logging for secret access: ```bash theme={null} AUDIT_LOGS_ENABLED=true ``` Query audit logs: ```sql theme={null} SELECT * FROM secret_access_log WHERE secret_name = 'OPENAI_API_KEY' ORDER BY accessed_at DESC; ``` ## Network Security ### Default Network Policy In the provided Kubernetes manifests (`k8s/network-policy.yaml`), sandbox pods: * Allow ingress only from the controller * Allow egress only to: * DNS (kube-dns) * TCP 443 to a Cloudflare IP allowlist (to reach `api.anthropic.com`) This is intentionally restrictive and IP-based (NetworkPolicy cannot match by DNS name). If you need broader egress or DNS-aware allowlisting, use an egress proxy/service mesh or a CNI with DNS-aware policies (e.g., Cilium). ### Custom Network Policies Create Kubernetes NetworkPolicy for additional restrictions: ```yaml theme={null} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: sandbox-egress namespace: stateset-sandbox spec: podSelector: matchLabels: app: stateset-sandbox policyTypes: - Egress egress: # Allow DNS - to: - namespaceSelector: {} podSelector: matchLabels: k8s-app: kube-dns ports: - protocol: UDP port: 53 # Allow HTTPS only - to: - ipBlock: cidr: 0.0.0.0/0 except: - 10.0.0.0/8 - 172.16.0.0/12 - 192.168.0.0/16 - 169.254.0.0/16 ports: - protocol: TCP port: 443 ``` ### Webhook URL Validation Webhooks cannot target: * localhost * Private IP addresses * Cluster-internal services Webhook requests: * Follow up to 3 redirects * Block cross-host redirects * Block HTTPS→HTTP downgrades ## Input Validation ### Path Traversal Prevention All file paths are validated: * No `..` components allowed * Must be within sandbox workspace * Absolute paths normalized ```python theme={null} # Rejected "/workspace/../etc/passwd" "../../secrets" # Allowed "/workspace/src/app.py" "/workspace/data.json" ``` ### Command Injection Prevention Prefer passing commands as arrays to avoid shell interpolation. If you send a string command, the controller executes it via `sh -c`. ```python theme={null} # Recommended (no shell) sdk.execute(sandbox_id, ["ls", "-la", "/workspace"]) # String form is supported, but uses a shell: sdk.execute(sandbox_id, "ls -la /workspace") ``` ### Request Size Limits | Limit | Default | Max | | ----------------------- | ------- | ------------------------------------ | | File content (per file) | 10MB | 10MB (hard limit) | | JSON request body | 10MB | 10MB (hard limit) | | Files per request | 100 | 100 (hard limit) | | Stripe webhook payload | 1MB | 5MB (via `STRIPE_WEBHOOK_MAX_BYTES`) | ## Pod Security ### Security Context Sandbox pods run with restricted security context: ```yaml theme={null} securityContext: runAsUser: 1001 runAsGroup: 1001 runAsNonRoot: true readOnlyRootFilesystem: true # When enabled allowPrivilegeEscalation: false capabilities: drop: - ALL ``` ### Service Account Sandboxes use a dedicated service account without cluster access: ```yaml theme={null} automountServiceAccountToken: false enableServiceLinks: false ``` Configuration: ```bash theme={null} SANDBOX_AUTOMOUNT_SERVICE_ACCOUNT_TOKEN=false SANDBOX_ENABLE_SERVICE_LINKS=false ``` ### Resource Limits All sandboxes have enforced limits: ```yaml theme={null} resources: limits: cpu: "2" memory: "4Gi" ephemeral-storage: "10Gi" requests: cpu: "100m" memory: "128Mi" ``` ## Database Security ### Encryption at Rest Sensitive values stored in the database (e.g., organization secrets and webhook secrets) are encrypted with AES-256-GCM. API keys are stored as one-way hashes (not reversible); only prefixes are returned for display. Configuration: ```bash theme={null} DATABASE_ENCRYPTION_KEY=<64-hex-character-key> ``` Generate key: ```bash theme={null} openssl rand -hex 32 ``` ### Connection Security ```bash theme={null} # Use SSL for database connections DATABASE_URL="postgres://user:pass@host:5432/sandbox?sslmode=require" ``` ### Access Control Use separate database users: * Application user with limited permissions * Migration user for schema changes * Read-only user for analytics ## Authentication & Authorization ### JWT Configuration ```bash theme={null} # Strong secret (minimum 32 characters) JWT_SECRET=$(openssl rand -base64 32) # Short token lifetime for security JWT_EXPIRY=3600 # 1 hour ``` ### Admin Access Protect admin endpoints: ```bash theme={null} ADMIN_API_KEY= # Or use username/password ADMIN_USERNAME=admin ADMIN_PASSWORD_HASH= ``` Generate bcrypt hash: ```javascript theme={null} const bcrypt = require('bcrypt'); console.log(bcrypt.hashSync('your-password', 10)); ``` ### Internal Endpoints Protect internal endpoints (/metrics, /health/detailed): ```bash theme={null} INTERNAL_API_KEY= ``` ## Audit Logging For pushing security events to your own systems as they happen — `security.alert`, `security.suspended`, `resource.critical` — see [Sandbox webhooks](/stateset-sandbox/stateset-sandbox-webhooks). ### Enable Audit Logs ```bash theme={null} AUDIT_LOGS_ENABLED=true ``` ### Logged Events * Sandbox creation/termination * Command execution * File operations * Secret access * API key operations * Admin actions ### Log Format ```json theme={null} { "timestamp": "2024-01-21T12:00:00.000Z", "event": "sandbox.created", "org_id": "org-123", "user_id": "user-456", "sandbox_id": "sb-abc789", "ip_address": "203.0.113.50", "user_agent": "Stateset-SDK/1.0.0", "details": { "cpus": "2", "memory": "4Gi" } } ``` ### Log Retention Configure retention policy based on compliance requirements: * SOC2: 1 year minimum * HIPAA: 6 years * GDPR: Document justification for retention period ## Production Hardening Checklist ### Environment Configuration * [ ] `NODE_ENV=production` * [ ] `DEV_MODE=false` * [ ] `LOG_LEVEL=info` (not debug/trace) ### Authentication * [ ] `JWT_SECRET` is unique, 32+ characters * [ ] `ADMIN_API_KEY` is set and strong * [ ] `INTERNAL_API_KEY` is set for metrics * [ ] API keys are rotated regularly ### Database * [ ] `DATABASE_ENCRYPTION_KEY` is set (64 hex chars) * [ ] SSL enabled for database connection * [ ] Database credentials are from secret manager * [ ] Regular backups configured ### Network * [ ] CORS restricted to your domains: `CORS_ORIGIN=https://app.example.com` * [ ] TLS/HTTPS enforced * [ ] `WEBHOOK_ALLOW_PRIVATE=false` * [ ] `WS_ALLOW_QUERY_AUTH=false` (legacy/deprecated; query-token auth is disabled for Events/WebSocket) ### Sandbox Security * [ ] `SANDBOX_READ_ONLY_ROOT=true` * [ ] `SANDBOX_AUTOMOUNT_SERVICE_ACCOUNT_TOKEN=false` * [ ] `SANDBOX_ENABLE_SERVICE_LINKS=false` * [ ] Appropriate isolation level set * [ ] Resource limits configured ### Monitoring * [ ] `AUDIT_LOGS_ENABLED=true` * [ ] Metrics collection configured * [ ] Alerting for security events * [ ] Log aggregation configured ## Incident Response ### Suspicious Activity 1. **Identify**: Check audit logs for anomalies 2. **Contain**: Suspend affected organization ```bash theme={null} curl -X POST https://api.sandbox.stateset.app/api/admin/orgs/$ORG_ID/suspend \ -H "Authorization: Bearer $ADMIN_KEY" ``` 3. **Investigate**: Review logs, metrics, sandbox contents 4. **Remediate**: Revoke compromised keys, patch vulnerabilities 5. **Document**: Post-incident report ### Key Compromise 1. **Revoke** the compromised key immediately 2. **Generate** new keys 3. **Audit** recent activity with the compromised key 4. **Notify** affected users 5. **Review** how the key was exposed ### Data Breach Response 1. **Assess** scope of breach 2. **Contain** the breach (suspend orgs, terminate sandboxes) 3. **Notify** affected parties per GDPR/regulatory requirements 4. **Document** timeline and response 5. **Improve** security controls ## Compliance Considerations ### SOC2 * Enable audit logging * Encrypt data at rest * Implement access controls * Monitor for anomalies * Document security policies ### GDPR * Document data processing purposes * Implement data retention policies * Support data export and deletion * Log consent and access * Appoint DPO if required ### HIPAA * Sign BAA with cloud providers * Enable audit logging * Encrypt all PHI * Implement access controls * Train staff on HIPAA ## Security Contacts * Security issues: [security@stateset.com](mailto:security@stateset.com) * Bug bounty: [https://stateset.com/security/bounty](https://stateset.com/security/bounty) * Security documentation: [https://stateset.com/security](https://stateset.com/security) ## Next steps Choosing between container, gVisor and MicroVM, and what each costs you in start-up time. Audit logging, alerting, and the incident procedures this guide's controls feed. Capacity, warm pools and the environment variables referenced above. Applying this hardening on EKS, GKE or AKS. # Sandbox webhooks Source: https://docs.stateset.com/stateset-sandbox/stateset-sandbox-webhooks Twenty-four events the sandbox controller emits, how to subscribe, and how to verify the signature on each delivery. Webhooks push sandbox events to your server as they happen, so an agent orchestrator does not poll `GET /sandbox/{id}/status` in a loop waiting for a pod to come up or a command to finish. ## Events **Sandbox lifecycle** | Event | Fires when | Payload carries | | ----------------- | -------------------------------- | ---------------------------- | | `sandbox.created` | The pod has been created | `sandbox_id`, `org_id` | | `sandbox.ready` | The sandbox accepts commands | `sandbox_id`, startup timing | | `sandbox.stopped` | The sandbox terminated | `sandbox_id`, reason | | `sandbox.error` | An error occurred in the sandbox | `sandbox_id`, error details | | `sandbox.timeout` | The timeout limit was reached | `sandbox_id`, duration | **Command execution** | Event | Fires when | Payload carries | | ------------------- | ------------------------------- | --------------------------------- | | `command.started` | Execution began | `sandbox_id`, command | | `command.completed` | The command exited successfully | `sandbox_id`, `exit_code`, output | | `command.failed` | The command failed | `sandbox_id`, `exit_code`, error | **Files and artifacts** | Event | Fires when | Payload carries | | ------------------- | ----------------------------------- | ------------------------ | | `file.written` | A file was written into the sandbox | `sandbox_id`, path, size | | `artifact.uploaded` | An artifact reached storage | `sandbox_id`, path, url | | `artifact.deleted` | An artifact was deleted | `sandbox_id`, path | **Checkpoints** | Event | Fires when | Payload carries | | --------------------- | --------------------------------- | ----------------------------- | | `checkpoint.created` | A checkpoint was saved | `sandbox_id`, `checkpoint_id` | | `checkpoint.restored` | A checkpoint was restored | `sandbox_id`, `checkpoint_id` | | `checkpoint.cloned` | A checkpoint became a new sandbox | `source_id`, `new_sandbox_id` | | `checkpoint.deleted` | A checkpoint was deleted | `checkpoint_id` | **Resources and security** | Event | Fires when | Payload carries | | ----------------------------- | --------------------------------------------------- | ----------------------------- | | `resource.warning` | Usage crossed 80% | `sandbox_id`, resource, value | | `resource.critical` | Usage crossed 95% | `sandbox_id`, resource, value | | `mcp.started` / `mcp.stopped` | An MCP server inside the sandbox started or stopped | `sandbox_id`, `server_name` | | `security.alert` | A security event was detected | `sandbox_id`, `alert_type` | | `security.suspended` | The organization was suspended | `org_id`, reason | ## Subscribe Subscribe to specific events, or to `"*"` for all of them. Give a `secret` — it is what lets you verify that a delivery came from StateSet rather than from anyone who learned your URL. ```bash cURL theme={null} curl -X POST https://api.sandbox.stateset.app/api/webhooks \ -H "Authorization: ApiKey sk_your_api_key" \ -H "Content-Type: application/json" \ -d '{ "url": "https://your-app.com/webhooks/sandbox", "events": ["sandbox.ready", "command.completed", "command.failed", "sandbox.stopped"], "secret": "whsec_your_webhook_secret", "retry_count": 3, "retry_delay_ms": 1000, "timeout_ms": 30000 }' ``` ```typescript Node.js theme={null} import { StateSetSandbox } from '@stateset/sandbox-sdk'; const sdk = new StateSetSandbox({ apiKey: 'sk_your_api_key' }); const webhook = await sdk.registerWebhook({ url: 'https://your-app.com/webhooks/sandbox', events: ['sandbox.ready', 'command.completed', 'command.failed', 'sandbox.stopped'], secret: 'whsec_your_webhook_secret', retry_count: 3, }); ``` ```python Python theme={null} from stateset_sandbox import CreateWebhookOptions, StateSetSandbox sdk = StateSetSandbox( base_url="https://api.sandbox.stateset.app", auth_token="sk_your_api_key", ) webhook = sdk.create_webhook(CreateWebhookOptions( url="https://your-app.com/webhooks/sandbox", events=["sandbox.ready", "command.completed", "command.failed", "sandbox.stopped"], secret="whsec_your_webhook_secret", retry_count=3, )) ``` | Field | Type | Default | | | ---------------- | ------------------ | -------- | ------------------------------------------ | | `url` | string | required | HTTPS endpoint that receives deliveries | | `events` | string\[] or `"*"` | `"*"` | Which events to receive | | `secret` | string | — | Signs each delivery; see below | | `headers` | object | — | Extra headers to include on every delivery | | `retry_count` | number | `3` | Retries on a non-2xx or timeout, 0–10 | | `retry_delay_ms` | number | `1000` | Base delay between retries | | `timeout_ms` | number | `30000` | Delivery timeout, 1000–60000 | ## The payload Every delivery has the same envelope; `data` is event-specific. ```json theme={null} { "id": "pay_lx7k8j_abc123", "event": "sandbox.ready", "timestamp": "2026-08-30T12:00:00.000Z", "sandboxId": "sb-abc123", "orgId": "org-xyz", "data": { "image": "stateset/sandbox:latest", "cpus": "2", "memory": "2Gi", "timeout_seconds": 600 } } ``` `id` is unique per delivery and stable across retries — use it to make your handler idempotent, because a retry after a timeout means you may see the same delivery twice. ## Verify the signature When a subscription has a `secret`, every delivery carries an HMAC-SHA256 of the raw body in the `X-Webhook-Signature` header, as `sha256=`. Verify it against the **raw request body**, not a re-serialised object — whitespace differences change the digest. ```javascript Node.js theme={null} const crypto = require('crypto'); function verify(rawBody, signature, secret) { const expected = 'sha256=' + crypto.createHmac('sha256', secret).update(rawBody).digest('hex'); return crypto.timingSafeEqual(Buffer.from(signature), Buffer.from(expected)); } app.post('/webhooks/sandbox', express.raw({ type: 'application/json' }), (req, res) => { if (!verify(req.body, req.headers['x-webhook-signature'] || '', process.env.WEBHOOK_SECRET)) { return res.status(401).json({ error: 'Invalid signature' }); } const event = JSON.parse(req.body); res.status(200).json({ received: true }); }); ``` ```python Python theme={null} import hmac, hashlib def verify(raw_body: bytes, signature: str, secret: str) -> bool: expected = "sha256=" + hmac.new(secret.encode(), raw_body, hashlib.sha256).hexdigest() return hmac.compare_digest(signature, expected) @app.route("/webhooks/sandbox", methods=["POST"]) def webhook(): if not verify(request.data, request.headers.get("X-Webhook-Signature", ""), WEBHOOK_SECRET): return {"error": "Invalid signature"}, 401 return {"received": True}, 200 ``` ```go Go theme={null} func verify(payload []byte, signature, secret string) bool { mac := hmac.New(sha256.New, []byte(secret)) mac.Write(payload) expected := "sha256=" + hex.EncodeToString(mac.Sum(nil)) return hmac.Equal([]byte(signature), []byte(expected)) } ``` Compare with a constant-time function — `timingSafeEqual`, `hmac.compare_digest`, `hmac.Equal` — not `==`. A plain string comparison leaks how many leading bytes matched. ## Related * [Sandbox API reference](/stateset-sandbox/stateset-sandbox-api-reference) — the status endpoints webhooks let you stop polling * [Sandbox security guide](/stateset-sandbox/stateset-sandbox-security-guide) — secrets, isolation and audit logging * [Sandbox operations](/stateset-sandbox/stateset-sandbox-operations) # StateSet iCommerce Sandbox Engine Source: https://docs.stateset.com/stateset-sandbox/stateset-sandboxes Isolated execution for AI agents — provision, run, capture, tear down. A sandbox is an isolated runtime an agent executes inside. It gets controlled access to tools and data, its output streams back for audit, and it disappears when you're done. This is where you run agent code you don't want touching your infrastructure directly. ## The lifecycle ``` create ──▶ execute ──▶ read/write files ──▶ stop │ │ timeout_seconds is the ceiling ───────────────┘ ``` ```bash theme={null} # 1. Provision curl -X POST https://api.sandbox.stateset.app/api/v1/sandbox/create \ -H "Authorization: ApiKey $STATESET_SANDBOX_API_KEY" \ -H "Content-Type: application/json" \ -d '{"timeout_seconds": 300}' # 2. Run something curl -X POST https://api.sandbox.stateset.app/api/v1/sandbox/$SANDBOX_ID/execute \ -H "Authorization: ApiKey $STATESET_SANDBOX_API_KEY" \ -H "Content-Type: application/json" \ -d '{"command": "python3 -c \"print(2+2)\""}' # 3. Tear down — do not wait for the timeout curl -X POST https://api.sandbox.stateset.app/api/v1/sandbox/$SANDBOX_ID/stop \ -H "Authorization: ApiKey $STATESET_SANDBOX_API_KEY" ``` Authentication is **`Authorization: ApiKey `** — not `Bearer`. The rest of the platform uses `Bearer`, which makes this the single most common integration mistake here. ## Files `POST /api/v1/sandbox/:id/files/write` and `/files/read` move data in and out without going through a shell command. ## Why isolation matters An agent that can execute code is, by construction, an agent that can execute the *wrong* code. A sandbox bounds the consequences: a bad run wastes a container and a timeout rather than reaching your systems. | Use | Why a sandbox | | -------------------- | ---------------------------------------------- | | **Agent evaluation** | Test a workflow before it touches production | | **Production runs** | Execution with a clear audit trail | | **Multi-tenant** | Workloads isolated per org | | **Untrusted code** | Model-generated code runs somewhere disposable | ## Always stop explicitly ```js theme={null} const sandbox = await create({ timeout_seconds: 300 }); try { await run(sandbox); } finally { await stop(sandbox.id); // even if run() throws } ``` A sandbox left running bills to its full `timeout_seconds`. The `finally` block is the difference between a failed run and an expensive one. ## Where sandboxes are used They back agent execution for [ResponseCX](/stateset-response/responsecx-platform), the [Console](/stateset-console/stateset-console-overview), and [iCommerce agent workflows](/stateset-icommerce/stateset-icommerce-agent-sandbox). ## Next First sandbox, end to end. Every endpoint. Controller, runtime, isolation model. Isolation guarantees and hardening. # Sequencer Architecture: Deep Dive Source: https://docs.stateset.com/stateset-sequencer-architecture The full architecture of the StateSet Sequencer, a Verifiable Event Sync (VES) v1.0 implementation for deterministic event ordering, cryptographic # StateSet Sequencer Architecture This document describes the high-level architecture of the StateSet Sequencer, a Verifiable Event Sync (VES) v1.0 implementation for deterministic event ordering, cryptographic verification, and agent-to-agent payment sequencing. ## System Overview ``` ┌──────────────────────────────────────────────────────────────────────────┐ │ Stateset Sequencer │ │ │ ┌──────────────────┐ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────────┐ │ │ │ HTTP / gRPC │ │ │ │ │ │ │ │ │ AI Agent 1 │──────────────▶│ │ Ingest │───▶│ Sequencer │───▶│ Event Store │ │ │ (Local SQLite) │ │ │ Service │ │ │ │ (PostgreSQL) │ │ │ │ │ │ │ │ │ │ │ │ └──────────────────┘ │ └──────┬──────┘ └──────┬──────┘ └────────┬────────┘ │ │ │ │ │ │ ┌──────────────────┐ │ ▼ │ │ │ │ │ HTTP / gRPC │ ┌─────────────┐ ┌──────┴──────┐ │ │ │ AI Agent 2 │──────────────▶│ │ Agent Key │ │ Schema │ │ │ │ (Local SQLite) │ │ │ Registry │ │ Registry │ │ │ │ │ │ │ │ │ │ │ │ └──────────────────┘ │ └─────────────┘ └─────────────┘ │ │ │ ▼ │ ┌──────────────────┐ │ ┌─────────────┐ ┌─────────────────┐ │ │ │ HTTP / gRPC │ │ Projector │ │ Commitment │ │ │ AI Agent N │──────────────▶│ │ │───▶│ Engine │ │ │ (Local SQLite) │ │ │ (Domain │ │ (Merkle) │ │ │ │ │ │ Handlers) │ │ │ │ └──────────────────┘ │ └──────┬──────┘ └────────┬────────┘ │ │ │ │ │ │ │ │ ┌─────────────┐ ┌──────┴──────┐ │ │ │ │ │ x402 │ │ Dead Letter │ │ │ │ │ │ Payment │ │ Queue │ │ │ │ │ │ Engine │ └─────────────┘ │ │ │ │ └──────┬──────┘ │ │ │ │ │ │ │ │ │ ┌──────┴──────────────────────────────────────┴────────────────┐ │ │ │ │ Operational Infrastructure │ │ │ │ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌───────────────┐ │ │ │ │ │ │ Cache │ │ Circuit │ │ Pool │ │ Metrics & │ │ │ │ │ │ │ Manager │ │ Breaker │ │ Monitor │ │ Telemetry │ │ │ │ │ │ └──────────┘ └──────────┘ └──────────┘ └───────────────┘ │ │ │ │ └─────────────────────────────────────────────────────────────┘ │ │ │ │ │ │ ┌─────────────────────────────────────────┐ │ │ │ │ Compliance Proof Engine │ │ │ │ │ (Verification + Storage) │ │ │ │ └──────────────────┬──────────────────────┘ │ │ │ │ │ │ │ ┌─────────────────────────────────────────┐ ┌─────────────────┐ │ │ │ │ Validity Proof Registry │ │ Anchor Service │ │ │ │ │ (External SNARK/ZK proofs) │ │ (Ethereum L2) │ │ │ │ └─────────────────────────────────────────┘ └────────┬────────┘ │ │ └────────────────────────────────────────────────────────┼────────────────┘ │ │ │ ┌────────────────────┐ ▼ │ │ │ ┌─────────────────┐ └─▶│ stateset-stark │ │ Set Chain │ │ (STARK Prover) │ │ (SetPaymentBatch│ │ │ │ + StatesetAnchor) └────────────────────┘ └─────────────────┘ ``` ### Component Relationships ``` ┌───────────────────────────────────────────────────────────────────────────────────────────────┐ │ Full Stack Architecture │ ├───────────────────────────────────────────────────────────────────────────────────────────────┤ │ │ │ ┌─────────────┐ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │ │ │ AI Agent │────▶│ stateset-sequencer│────▶│ stateset-stark │────▶│ Set Chain │ │ │ │ (CLI) │ │ (Event Sync + │ │ (ZK Proofs) │ │ (L2: Anchors + │ │ │ │ │ │ Payments) │ │ │ │ Payments) │ │ │ └─────────────┘ └──────────────────┘ └──────────────────┘ └──────────────────┘ │ │ │ │ │ │ │ │ ▼ ▼ ▼ ▼ │ │ ┌─────────────┐ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │ │ │ SQLite │ │ PostgreSQL │ │ STARK Proofs │ │ On-Chain │ │ │ │ Outbox │ │ Event Store + │ │ (~100-200KB) │ │ Anchors + │ │ │ │ │ │ Payment Intents │ │ │ │ Settlements │ │ │ └─────────────┘ └──────────────────┘ └──────────────────┘ └──────────────────┘ │ │ │ │ Protocols: HTTP REST | gRPC v1+v2 (streaming) | x402 (payments) | VES v1.0 (events) │ │ │ └───────────────────────────────────────────────────────────────────────────────────────────────┘ ``` ## Core Components ### 1. Ingest Service **Location:** `src/api/handlers/ingest.rs`, `src/server.rs` The entry point for all events via HTTP REST and gRPC. Responsible for: * **Authentication**: Validates API keys, JWT tokens, or agent Ed25519 signatures * **Schema Validation**: Validates payloads against registered JSON Schemas (configurable: disabled, warn, strict) * **Signature Verification**: Verifies Ed25519 agent signatures with domain-separated hashing * **Deduplication**: Rejects duplicate `event_id` and `command_id` values * **Batching**: Groups events for efficient processing with parallel partitioning * **Rate Limiting**: Sliding-window per-tenant rate limiting ``` Request → Auth → Rate Limit → Validate Schema → Verify Sig → Dedupe → Sequencer ``` ### 2. Agent Key Registry **Location:** `src/auth/agent_keys.rs`, `src/infra/postgres/agent_key_registry.rs` Manages agent public keys for signature verification: * **Key Registration**: `POST /api/v1/agents/keys` (REST) and `RegisterAgentKey` (gRPC) * **Key Lookup**: `(tenant_id, agent_id, key_id) -> public_key` with LRU caching * **Key Types**: Ed25519 (signing) and X25519 (encryption) * **Validity Windows**: Keys have `valid_from` and `valid_to` timestamps * **Revocation**: Keys can be revoked to invalidate future signatures * **Proof of Possession**: Registration requires a signature proving key ownership ```rust theme={null} pub struct AgentKeyEntry { pub public_key: [u8; 32], // Ed25519 or X25519 public key pub key_type: KeyType, // Signing or Encryption pub status: KeyStatus, // Active, Revoked, Expired pub valid_from: Option>, pub valid_to: Option>, } ``` ### 3. Sequencer **Location:** `src/infra/postgres/sequencer.rs`, `src/infra/postgres/ves_sequencer.rs` Assigns monotonic sequence numbers to events: * **Monotonic Ordering**: Each `(tenant_id, store_id)` has independent sequence counter * **Gap-Free**: Sequence numbers are contiguous with no gaps * **Atomic Assignment**: Uses a single PostgreSQL transaction with `SELECT ... FOR UPDATE` on the per-stream counter * **Receipt Generation**: Produces signed receipts for each sequenced event (configurable via `VES_SEQUENCER_SIGNING_KEY`) * **Sequencer Identity**: Optional pinned sequencer ID via `VES_SEQUENCER_ID` ```sql theme={null} -- Sequence counter table CREATE TABLE sequence_counters ( tenant_id UUID NOT NULL, store_id UUID NOT NULL, current_sequence BIGINT NOT NULL DEFAULT 0, PRIMARY KEY (tenant_id, store_id) ); ``` ### 4. Event Store **Location:** `src/infra/postgres/event_store.rs` Append-only storage for sequenced events: * **Immutability**: Events are never modified or deleted * **Encryption-at-Rest**: Optional AES-256-GCM payload encryption (modes: disabled, optional, required) * **Indexing**: Efficient queries by sequence, entity, and time * **Range Reads**: Fetch events by sequence number range * **Read/Write Splitting**: Reads served from replica pool when configured ```sql theme={null} CREATE TABLE events ( tenant_id UUID NOT NULL, store_id UUID NOT NULL, sequence_number BIGINT NOT NULL, event_id UUID UNIQUE NOT NULL, entity_type VARCHAR(64) NOT NULL, entity_id VARCHAR(256) NOT NULL, event_type VARCHAR(128) NOT NULL, payload JSONB NOT NULL, payload_hash BYTEA NOT NULL, created_at TIMESTAMPTZ NOT NULL, PRIMARY KEY (tenant_id, store_id, sequence_number) ); ``` ### 5. Projector **Location:** `src/projection/handlers.rs`, `src/projection/runner.rs` Applies events to domain projections: * **Domain Handlers**: Entity-specific projection logic * **Optimistic Concurrency**: Version checking prevents conflicts * **Invariant Validation**: Rejects events violating business rules * **Checkpoint Tracking**: Tracks last processed sequence per store * **Dead Letter Queue**: Failed projections are moved to DLQ for retry Supported entity types: * **Order**: `order.created`, `order.confirmed`, `order.shipped`, etc. * **Inventory**: `inventory.initialized`, `inventory.adjusted`, `inventory.reserved` * **Product**: `product.created`, `product.updated`, `product.deactivated` * **Customer**: `customer.created`, `customer.updated`, `customer.address_added` * **Return**: `return.requested`, `return.approved`, `return.refunded` * **x402 Payment**: `x402_payment.created`, `x402_payment.sequenced`, `x402_payment.settled` * **x402 Batch**: `x402_batch.created`, `x402_batch.committed`, `x402_batch.settled` ### 6. Commitment Engine **Location:** `src/infra/postgres/commitment.rs`, `src/infra/ves_commitment.rs` Creates Merkle tree commitments over event batches: * **Merkle Roots**: SHA-256 trees over event payload hashes * **State Roots**: Track state transitions (prev\_root -> new\_root) * **Inclusion Proofs**: Generate proofs for individual events * **Batch Storage**: Persist commitments for later verification * **VES Commitments**: Separate commitment engine for VES v1.0 events ```rust theme={null} pub struct BatchCommitment { pub batch_id: Uuid, pub tenant_id: TenantId, pub store_id: StoreId, pub prev_state_root: [u8; 32], pub new_state_root: [u8; 32], pub events_root: [u8; 32], // Merkle root of payload hashes pub event_count: u32, pub sequence_range: (u64, u64), pub committed_at: DateTime, pub chain_tx_hash: Option<[u8; 32]>, } ``` ### 7. Anchor Service **Location:** `src/anchor.rs` Submits commitments to Ethereum L2: * **StateSetAnchor Contract**: On-chain batch commitment storage * **SetPaymentBatch Contract**: On-chain x402 payment batch settlement * **Transaction Building**: Constructs and signs anchor transactions using Alloy * **Verification**: Confirms anchoring status on-chain * **Gas Management**: Handles gas estimation and pricing * **Circuit Breaker Protected**: External calls guarded by circuit breaker ### 8. Compliance Proof Engine **Location:** `src/domain/ves_compliance.rs`, `src/infra/ves_compliance.rs` Stores and verifies zero-knowledge compliance proofs generated by `stateset-stark`: * **Proof Storage**: Stores STARK proofs in `ves_compliance_proofs` table * **Public Input Validation**: Ensures canonical public inputs match event data * **Policy Verification**: Validates proof matches declared policy * **Idempotency**: Deduplicates by `(event_id, proof_type, policy_hash)` ```sql theme={null} CREATE TABLE ves_compliance_proofs ( id UUID PRIMARY KEY, event_id UUID NOT NULL REFERENCES ves_events(event_id), proof_type VARCHAR(64) NOT NULL, -- e.g., "stark.compliance.v1" proof_version INTEGER NOT NULL, policy_id VARCHAR(128) NOT NULL, -- e.g., "aml.threshold" policy_params JSONB NOT NULL, -- e.g., {"threshold": 10000} policy_hash BYTEA NOT NULL, -- SHA256 of policy proof_hash BYTEA NOT NULL, -- SHA256 of proof bytes proof_bytes BYTEA, -- Full STARK proof (~100-200KB) public_inputs JSONB NOT NULL, -- Canonical JCS format witness_commitment BYTEA, -- Rescue hash of private witness verified_at TIMESTAMPTZ, created_at TIMESTAMPTZ NOT NULL, UNIQUE (event_id, proof_type, proof_version, policy_hash) ); ``` ### 9. Validity Proof Registry **Location:** `src/domain/ves_validity.rs`, `src/infra/ves_validity.rs` External proof registry for SNARK/ZK proofs attesting to batch properties: * **Proof Submission**: External provers submit validity proofs for committed batches * **Proof Storage**: Persists proof bytes and public inputs * **Proof Hashing**: SHA-256 hash of proof for integrity * **Stream Matching**: Trigger enforces proofs reference valid batches ```sql theme={null} CREATE TABLE ves_validity_proofs ( id UUID PRIMARY KEY, batch_id UUID NOT NULL REFERENCES ves_commitments(batch_id), proof_type VARCHAR(64) NOT NULL, proof_version INTEGER NOT NULL, proof_hash BYTEA NOT NULL, proof_bytes BYTEA, public_inputs JSONB NOT NULL, verified_at TIMESTAMPTZ, created_at TIMESTAMPTZ NOT NULL ); ``` ### 10. x402 Payment Engine **Location:** `src/domain/x402_payment.rs`, `src/infra/postgres/x402_repository.rs`, `src/infra/x402_batch_worker.rs`, `src/api/handlers/x402.rs` Implements the x402 protocol for agent-to-agent payment sequencing and batched L2 settlement: * **Payment Intent Sequencing**: Signed payment intents assigned sequence numbers * **Signature Verification**: Ed25519 signatures with `X402_PAYMENT_V1` domain separator * **Nonce-Based Replay Protection**: Per-agent nonce tracking * **Idempotency**: Optional idempotency keys for at-most-once delivery * **Batch Assembly**: Configurable batch size (default 100, max 1000) and time thresholds * **Merkle Commitments**: Merkle root computation over batched payment intents * **Multi-Chain Settlement**: Settlement on Set Chain L2 via `SetPaymentBatch` contract * **Multi-Asset Support**: USDC, USDT, ssUSD, wssUSD, DAI, ETH ``` AI Agent | | Creates X402PaymentIntent (signed) v Sequencer | | 1. Validates signature | 2. Checks nonce (replay protection) | 3. Assigns sequence number | 4. Batches into X402PaymentBatch v Set Chain L2 (SetPaymentBatch contract) | | Executes aggregated transfers v Settlement complete (receipt with Merkle inclusion proof) ``` **Supported Networks:** | Network | Chain ID | Type | | ----------------- | -------- | ------- | | Set Chain | 84532001 | Mainnet | | Set Chain Testnet | 84532002 | Testnet | | Arc | 5042001 | Mainnet | | Base | 8453 | Mainnet | | Ethereum | 1 | Mainnet | | Arbitrum | 42161 | Mainnet | | Optimism | 10 | Mainnet | **Supported Assets:** | Asset | Decimals | Description | | ------ | -------- | ------------------------------- | | USDC | 6 | USD Coin | | USDT | 6 | Tether | | ssUSD | 6 | StateSet USD (yield-bearing) | | wssUSD | 6 | Wrapped StateSet USD (ERC-4626) | | DAI | 18 | DAI stablecoin | | ETH | 18 | Native ETH | **Payment Intent Lifecycle:** ``` Pending -> Sequenced -> Batched -> Settled -> Failed -> Expired ``` ### 11. Schema Registry **Location:** `src/domain/schema.rs`, `src/infra/postgres/schema_store.rs`, `src/api/handlers/schemas.rs` JSON Schema validation system for event payloads: * **Schema Versioning**: Monotonically increasing version per `(tenant_id, event_type)` * **Compatibility Modes**: Forward, Backward, Full, or None * **Validation Modes**: Disabled, Optional (warn), Required, Strict * **Status Lifecycle**: Active -> Deprecated -> Archived * **LRU Caching**: Configurable cache size and TTL for hot schemas * **Detailed Errors**: Validation errors include JSON paths and messages ```rust theme={null} pub struct Schema { pub id: SchemaId, pub tenant_id: TenantId, pub event_type: EventType, pub version: u32, pub schema_json: serde_json::Value, pub status: SchemaStatus, // Active, Deprecated, Archived pub compatibility: SchemaCompatibility, // Forward, Backward, Full, None pub description: Option, pub created_at: DateTime, } ``` The v1 and v2 gRPC surfaces are served side by side and are not interchangeable: v2 carries the compliance-proof fields that v1 has no place for. Pin a version explicitly in your client rather than relying on a default, or a proof-bearing response will arrive at a v1 client with the proof silently absent. ## gRPC API (v1 + v2) **Location:** `src/grpc/`, `proto/sequencer.proto`, `proto/sequencer_v2.proto` The sequencer exposes dual gRPC services alongside the REST API: ### gRPC v2 Service (Full VES v1.0 Protocol) | RPC | Type | Description | | ------------------- | ----------------------- | -------------------------------------- | | `Push` | Unary | Push a batch of events for sequencing | | `PullEvents` | Unary | Pull events (simple polling) | | `GetSyncState` | Unary | Get current sync state for store | | `GetInclusionProof` | Unary | Get Merkle inclusion proof | | `GetCommitment` | Unary | Get batch commitment | | `GetEntityHistory` | Unary | Get entity event history | | `GetHealth` | Unary | Health check | | `StreamEvents` | Server streaming | Continuous event delivery with filters | | `SyncStream` | Bidirectional streaming | Full-duplex agent sync | | `SubscribeEntity` | Server streaming | Subscribe to entity updates | ### Key Management Service (gRPC) | RPC | Description | | ------------------ | ---------------------------------------------------- | | `RegisterAgentKey` | Register Ed25519/X25519 key with proof of possession | | `GetAgentKeys` | List agent keys with optional filters | | `RevokeAgentKey` | Revoke an agent key | ### Bidirectional Sync Protocol The `SyncStream` RPC enables full-duplex communication: ``` Client -> Server: Push, Pull, EventAck, Heartbeat Server -> Client: PushResponse, PullResponse, SequencedEvent, SyncState, Heartbeat ``` Agents can push events, receive real-time updates, and acknowledge processing in a single persistent connection with heartbeat-based liveness detection. ## Operational Infrastructure ### Authentication System **Location:** `src/auth/` Multi-method authentication with composable validators: | Method | Description | | ------------- | ------------------------------------------------------------ | | API Keys | SHA-256 hashed, scoped to tenant/store, stored in PostgreSQL | | JWT Tokens | HS256/HS384/HS512 with configurable issuer/audience | | Agent Keys | Ed25519 signature verification for VES events | | Bootstrap Key | Initial admin key from `BOOTSTRAP_ADMIN_API_KEY` env var | * **Rate Limiting**: Sliding-window algorithm, configurable per-minute limit * **Permissions Model**: Read, Write, Admin scopes per key * **gRPC Auth Interceptor**: Shared authenticator for gRPC services ### Cache Manager **Location:** `src/infra/cache.rs` Multi-layer LRU caching with configurable TTL per cache type: | Cache | Default Max | Description | | --------------- | ------------ | ------------------------- | | Commitments | configurable | Merkle commitment lookups | | Proofs | configurable | Inclusion proof results | | VES Commitments | configurable | VES-specific commitments | | VES Proofs | configurable | VES-specific proofs | | Agent Keys | configurable | Agent public key lookups | | Schemas | configurable | JSON Schema definitions | ### Pool Monitor **Location:** `src/infra/pool_monitor.rs` Real-time database connection pool health tracking (15-second polling): * **Health States**: Healthy (\< 50%), Moderate (50-80%), Stressed (80-95%), Critical (> 95%) * **Metrics**: Active/idle connections, acquisition latency, slow acquisition tracking * **Integrated**: Exposed via `/health/detailed` endpoint and Prometheus metrics ### Circuit Breaker Registry **Location:** `src/infra/circuit_breaker.rs` Failure resilience for external service calls (L2 anchoring, chain settlement): * **States**: Closed (normal) -> Open (fail-fast) -> HalfOpen (testing recovery) * **Exponential Backoff**: Configurable multiplier with jitter * **Slow Call Detection**: Configurable threshold for degraded performance * **Per-Service Tracking**: Independent breaker per external service ### Dead Letter Queue **Location:** `src/infra/dead_letter.rs` Handles events that fail projection processing: * **Auto-Retry**: Exponential backoff (1 min initial, 1 hour max, 10 retries) * **Categorized Reasons**: Schema validation, invariant violation, state transition errors * **Non-Retryable**: Invariant violations and invalid state transitions skip retry * **Admin Operations**: Retry, purge, and inspect via admin CLI ### Payload Encryption-at-Rest **Location:** `src/infra/payload_encryption.rs`, `src/crypto/encrypt.rs` Automatic event payload encryption in the database: * **Modes**: Disabled, Optional, Required * **Algorithm**: AES-256-GCM * **HPKE Support**: Multi-recipient encryption via X25519-HKDF-SHA256 * **Key Rotation**: Supports key versioning and rotation ### Audit Logging **Location:** `src/infra/audit.rs` Comprehensive audit trail for administrative operations: * API key management (create, revoke, update) * Schema registry changes (register, deprecate, delete) * Agent key operations (register, rotate, revoke) * Authentication events (login, failure, token refresh) * Dead letter queue operations (retry, purge) ### Metrics & Telemetry **Location:** `src/metrics/`, `src/telemetry/` * **Prometheus Export**: `/metrics` endpoint with 40+ predefined metric names * **Component Metrics**: Background collection every 15 seconds (pool, circuit breaker stats) * **OpenTelemetry**: OTLP export for distributed tracing (`OTEL_EXPORTER_OTLP_ENDPOINT`) * **Structured Logging**: JSON or text format (`LOG_FORMAT`) * **Counters, Gauges, Histograms**: Full metric type support with labels ### Graceful Shutdown **Location:** `src/infra/graceful_shutdown.rs` Coordinated shutdown with request draining: * **Request Tracking**: Guard-based in-flight request monitoring * **Shutdown Signals**: Coordinated signal propagation to background tasks * **Deadline Enforcement**: Configurable drain timeout ## stateset-stark (ZK Compliance Proofs) **Repository:** `stateset-stark` A STARK proving system that enables cryptographic verification of compliance policies on encrypted event payloads without revealing the underlying data. ### Purpose When events contain encrypted payloads (e.g., order amounts), compliance rules (e.g., AML thresholds) need verification without exposing sensitive data. `stateset-stark` generates zero-knowledge proofs that: 1. The prover knows the plaintext payload 2. The payload satisfies the compliance policy 3. The payload matches the encrypted ciphertext hash ### Architecture ``` stateset-stark/ ├── crates/ │ ├── ves-stark-primitives/ # Goldilocks field, Rescue hash │ ├── ves-stark-air/ # AIR constraint definitions (167 constraints) │ ├── ves-stark-prover/ # Proof generation │ ├── ves-stark-verifier/ # Proof verification │ ├── ves-stark-batch/ # Batch proofs (Phase 2) │ ├── ves-stark-client/ # HTTP client for sequencer │ └── ves-stark-cli/ # Command-line tool ``` ### Cryptographic Foundation | Component | Choice | Notes | | ---------- | -------------------------------- | ----------------------------- | | Field | Goldilocks (p = 2^64 - 2^32 + 1) | 64-bit efficient arithmetic | | Hash | Rescue-Prime | STARK-friendly algebraic hash | | Commitment | Blake3-256 Merkle | Vector commitments | | Security | \~100 bits | Default proof options | ### Supported Policies | Policy ID | Constraint | Use Case | | ----------------- | -------------------- | --------------------------------- | | `aml.threshold` | `amount < threshold` | AML compliance (strict less-than) | | `order_total.cap` | `amount <= cap` | Order limits (less-than-or-equal) | ### Proof Generation Flow ``` ┌─────────────────────────────────────────────────────────────────────────┐ │ Compliance Proof Generation │ ├─────────────────────────────────────────────────────────────────────────┤ │ │ │ 1. Agent decrypts VES event payload (ephemeral, off-chain) │ │ │ │ │ ▼ │ │ 2. Extract witness data (e.g., order amount = 5000) │ │ │ │ │ ▼ │ │ 3. Fetch canonical public inputs from sequencer │ │ GET /api/v1/ves/compliance/{event_id}/inputs │ │ │ │ │ ▼ │ │ 4. Build ComplianceWitness + CompliancePublicInputs │ │ │ │ │ ▼ │ │ 5. Generate STARK proof (ves-stark-prover) │ │ - Build execution trace (105 columns, 128+ rows) │ │ - Apply AIR constraints (167 total) │ │ - Produce proof (~100-200KB) │ │ │ │ │ ▼ │ │ 6. Submit proof to sequencer │ │ POST /api/v1/ves/compliance/{event_id}/proofs │ │ │ │ │ ▼ │ │ 7. Sequencer verifies and stores proof │ │ │ └─────────────────────────────────────────────────────────────────────────┘ ``` ### Public Inputs (Canonical JCS Format) ```json theme={null} { "eventId": "550e8400-e29b-41d4-a716-446655440000", "tenantId": "tenant-uuid", "storeId": "store-uuid", "sequenceNumber": 42, "payloadKind": 1, "payloadPlainHash": "abc123...", "payloadCipherHash": "def456...", "eventSigningHash": "789abc...", "policyId": "aml.threshold", "policyParams": { "threshold": 10000 }, "policyHash": "computed-sha256..." } ``` ### CLI Usage ```bash theme={null} # Generate a compliance proof ves-stark prove --amount 5000 --limit 10000 --policy aml.threshold # Verify a proof ves-stark verify --proof proof.stark --inputs inputs.json --limit 10000 # Benchmark proving performance ves-stark benchmark -n 10 --max-amount 10000 --limit 10000 ``` ### Integration Points | Sequencer Endpoint | Purpose | | ----------------------------------------------------- | ----------------------------- | | `GET /api/v1/ves/compliance/{event_id}/inputs` | Fetch canonical public inputs | | `POST /api/v1/ves/compliance/{event_id}/proofs` | Submit generated proof | | `GET /api/v1/ves/compliance/{event_id}/proofs` | List proofs for event | | `GET /api/v1/ves/compliance/proofs/{proof_id}` | Get proof by ID | | `GET /api/v1/ves/compliance/proofs/{proof_id}/verify` | Verify proof | ## REST API Reference ### Event Ingestion | Method | Endpoint | Description | | ------ | --------------------------- | ---------------------------------------- | | `POST` | `/api/v1/events/ingest` | Legacy event ingestion | | `POST` | `/api/v1/ves/events/ingest` | VES v1.0 event ingestion with signatures | ### VES Commitments | Method | Endpoint | Description | | ------ | ----------------------------------- | ----------------------- | | `GET` | `/api/v1/ves/commitments` | List VES commitments | | `POST` | `/api/v1/ves/commitments` | Create VES commitment | | `POST` | `/api/v1/ves/commitments/anchor` | Commit and anchor | | `GET` | `/api/v1/ves/commitments/:batch_id` | Get specific commitment | ### VES Proofs & Anchoring | Method | Endpoint | Description | | ------ | ------------------------------------- | ------------------------- | | `GET` | `/api/v1/ves/proofs/:sequence_number` | Get VES inclusion proof | | `POST` | `/api/v1/ves/proofs/verify` | Verify VES proof | | `POST` | `/api/v1/ves/anchor` | Anchor VES commitment | | `GET` | `/api/v1/ves/anchor/:batch_id/verify` | Verify on-chain anchoring | ### VES Validity Proofs | Method | Endpoint | Description | | ------ | ---------------------------------------------- | --------------------- | | `GET` | `/api/v1/ves/validity/:batch_id/inputs` | Get public inputs | | `GET` | `/api/v1/ves/validity/:batch_id/proofs` | List validity proofs | | `POST` | `/api/v1/ves/validity/:batch_id/proofs` | Submit validity proof | | `GET` | `/api/v1/ves/validity/proofs/:proof_id` | Get proof by ID | | `GET` | `/api/v1/ves/validity/proofs/:proof_id/verify` | Verify validity proof | ### VES Compliance Proofs | Method | Endpoint | Description | | ------ | ------------------------------------------------ | ----------------------- | | `POST` | `/api/v1/ves/compliance/:event_id/inputs` | Get public inputs | | `GET` | `/api/v1/ves/compliance/:event_id/proofs` | List compliance proofs | | `POST` | `/api/v1/ves/compliance/:event_id/proofs` | Submit compliance proof | | `GET` | `/api/v1/ves/compliance/proofs/:proof_id` | Get proof by ID | | `GET` | `/api/v1/ves/compliance/proofs/:proof_id/verify` | Verify compliance proof | ### x402 Payment Protocol | Method | Endpoint | Description | | ------ | ------------------------------------------ | --------------------- | | `POST` | `/api/v1/x402/payments` | Submit payment intent | | `GET` | `/api/v1/x402/payments` | List payment intents | | `GET` | `/api/v1/x402/payments/:intent_id` | Get payment intent | | `GET` | `/api/v1/x402/payments/:intent_id/receipt` | Get payment receipt | | `POST` | `/api/v1/x402/batches` | Create payment batch | | `GET` | `/api/v1/x402/batches/:batch_id` | Get batch | | `POST` | `/api/v1/x402/batches/settle` | Settle batch on-chain | ### Schema Registry | Method | Endpoint | Description | | -------- | ----------------------------------------------- | ------------------------- | | `GET` | `/api/v1/schemas` | List schemas | | `POST` | `/api/v1/schemas` | Register schema | | `POST` | `/api/v1/schemas/validate` | Validate payload | | `GET` | `/api/v1/schemas/:schema_id` | Get schema | | `PUT` | `/api/v1/schemas/:schema_id/status` | Update schema status | | `DELETE` | `/api/v1/schemas/:schema_id` | Delete schema | | `GET` | `/api/v1/schemas/event-type/:event_type` | Get schemas by event type | | `GET` | `/api/v1/schemas/event-type/:event_type/latest` | Get latest schema | ### Legacy Events & Commitments | Method | Endpoint | Description | | ---------- | ------------------------------------------ | ------------------------- | | `GET` | `/api/v1/events` | List events | | `GET` | `/api/v1/head` | Get current head sequence | | `GET` | `/api/v1/entities/:entity_type/:entity_id` | Get entity history | | `GET/POST` | `/api/v1/commitments` | List/create commitments | | `GET` | `/api/v1/commitments/:batch_id` | Get commitment | | `GET` | `/api/v1/proofs/:sequence_number` | Get inclusion proof | | `POST` | `/api/v1/proofs/verify` | Verify proof | ### Health & Observability | Method | Endpoint | Description | | ------ | ------------------ | ---------------------------------------- | | `GET` | `/health` | Basic health check | | `GET` | `/health/detailed` | Detailed health (pool, circuit breakers) | | `GET` | `/ready` | Readiness probe | | `GET` | `/metrics` | Prometheus metrics | ## Data Flow ### Event Ingestion Flow ``` 1. Agent creates event locally (SQLite outbox) 2. Agent signs event with Ed25519 private key 3. Agent POSTs to /api/v1/ves/events/ingest (or pushes via gRPC) 4. Sequencer authenticates request (API key, JWT, or agent signature) 5. Schema validation (if configured) 6. Sequencer verifies Ed25519 signature against registered public key 7. Sequencer deduplicates by event_id and command_id 8. Sequencer assigns sequence number atomically 9. Event stored in PostgreSQL events table (encrypted-at-rest if configured) 10. Sequencer returns receipt with sequence number 11. Agent marks event as synced locally ``` ### x402 Payment Flow ``` 1. Agent creates X402PaymentIntent with payer/payee/amount/asset 2. Agent signs intent with Ed25519 key (X402_PAYMENT_V1 domain separator) 3. Agent POSTs to /api/v1/x402/payments 4. Sequencer validates signature, checks nonce, verifies expiration 5. Sequencer assigns x402 sequence number atomically 6. Intent stored with status=Sequenced 7. Batch worker assembles intents into X402PaymentBatch (configurable size/time) 8. Batch worker computes Merkle root over payment intents 9. Batch submitted to SetPaymentBatch contract on Set Chain L2 10. On confirmation, intents marked Settled with tx_hash and block_number 11. Payment receipt with Merkle inclusion proof available for verification ``` ### Commitment Flow ``` 1. Client requests commitment for sequence range 2. Commitment Engine reads events from Event Store 3. Engine builds Merkle tree from payload hashes 4. Engine computes state root transition 5. Commitment stored in commitments table 6. (Optional) Anchor Service submits to StatesetAnchor 7. Chain tx hash stored with commitment ``` ### Verification Flow ``` 1. Client requests inclusion proof for sequence N 2. Engine retrieves commitment containing N 3. Engine rebuilds Merkle tree for batch 4. Engine generates proof path for leaf N 5. Client verifies locally, then verifies the batch root is anchored on-chain ``` ### Compliance Proof Flow ``` 1. Agent creates encrypted VES event (payload encrypted with HPKE) 2. Agent syncs event to sequencer (receives sequence number) 3. Agent decrypts payload locally (ephemeral) 4. Agent extracts witness data (e.g., order.total = 5000) 5. Agent fetches canonical public inputs from sequencer 6. Agent generates STARK proof using stateset-stark: - Builds execution trace (witness decomposition, Rescue hash) - Applies 167 AIR constraints - Produces ~100-200KB proof 7. Agent submits proof to sequencer 8. Sequencer validates public inputs match event 9. Sequencer stores proof in ves_compliance_proofs 10. (Future) Sequencer cryptographically verifies proof ``` ## Database Schema ### Core Tables | Table | Purpose | | ------------------------ | --------------------------------------- | | `events` | Append-only event log (legacy) | | `ves_events` | VES v1.0 events with signatures | | `sequence_counters` | Per-store sequence tracking | | `ves_sequencer_receipts` | VES sequencer signed receipts | | `commitments` | Legacy Merkle commitment records | | `ves_commitments` | VES v1.0 Merkle commitments | | `ves_compliance_proofs` | STARK compliance proofs | | `ves_validity_proofs` | Batch validity proofs | | `agent_signing_keys` | Agent public key registry with rotation | | `entity_versions` | Entity version tracking (OCC) | | `projection_checkpoints` | Projector progress | | `agent_sync_state` | Agent sync state tracking | | `rejected_events_log` | Event rejection audit log | | `ves_rejections` | VES event rejection log | | `x402_payment_intents` | x402 payment authorizations | | `x402_payment_batches` | x402 aggregated payment batches | | `x402_sequence_counters` | Per-store x402 sequence tracking | | `x402_nonce_tracking` | x402 nonce replay protection | | `api_keys` | API key management | ### Migrations | Migration | Description | | ------------------------------- | ---------------------------------------------------- | | `001_production_postgres.sql` | Core event store, sequence counters, entity versions | | `002_ves_v1_tables.sql` | VES v1.0 events, receipts, commitments, agent keys | | `003_constraints.sql` | Unique indexes, chain TX hash validation | | `004_ves_validity_proofs.sql` | Validity proof registry with stream matching | | `005_ves_compliance_proofs.sql` | Compliance proof storage with stream matching | | `006_key_rotation_policies.sql` | Agent key rotation policies | | `007_encryption_groups.sql` | Encryption group management | | `008_command_dedupe.sql` | Command deduplication indexes | | `009_api_keys.sql` | API key management tables | | `010_ves_sequence_counters.sql` | VES-specific sequence counters | | `011_x402_payments.sql` | x402 payment intents and batches | ### Indexes ```sql theme={null} -- Fast event lookups CREATE INDEX idx_events_entity ON events(tenant_id, store_id, entity_type, entity_id); CREATE INDEX idx_events_type ON events(tenant_id, store_id, event_type); CREATE INDEX idx_events_created ON events(tenant_id, store_id, created_at); -- Agent key lookups CREATE INDEX idx_agent_keys_lookup ON agent_keys(tenant_id, agent_id, key_id); ``` ## Cryptographic Design ### Signing Hash Construction Per VES v1.0 Section 8.3: ``` signing_hash = SHA256( "VES_EVENTSIG_V1" || // Domain separator event_id || tenant_id || store_id || agent_id || entity_type || entity_id || event_type || payload_plain_hash || occurred_at ) agent_signature = Ed25519.Sign(agent_private_key, signing_hash) ``` ### x402 Payment Signing Hash ``` signing_hash = SHA256( "X402_PAYMENT_V1" || // Domain separator intent_id || payer_address || payee_address || amount || asset || network || chain_id || nonce || valid_until ) payer_signature = Ed25519.Sign(agent_private_key, signing_hash) ``` ### Merkle Tree Construction ``` [Root] / \ [H01] [H23] / \ / \ [H0] [H1] [H2] [H3] | | | | E0 E1 E2 E3 (Event payload hashes) Domain-separated hashing: - Leaf: SHA256("VES_LEAF_V1" || payload_hash) - Node: SHA256("VES_NODE_V1" || left || right) ``` ### Encryption (HPKE) Multi-recipient encryption for VES-ENC-1: | Parameter | Value | | --------- | ------------------ | | Mode | Base | | KEM | X25519-HKDF-SHA256 | | KDF | HKDF-SHA256 | | AEAD | AES-256-GCM | ## Offline-First Architecture Agents operate offline using SQLite: ``` ┌─────────────────────────────────────────┐ │ Local Agent │ │ │ │ ┌─────────────┐ ┌───────────────┐ │ │ │ Business │───▶│ Outbox │ │ │ │ Logic │ │ (SQLite) │ │ │ └─────────────┘ └───────┬───────┘ │ │ │ │ │ ▼ │ │ ┌───────────────┐ │ │ │ Sync State │ │ │ │ Tracker │ │ │ └───────┬───────┘ │ └─────────────────────────────┼───────────┘ │ ▼ (when online) ┌───────────────────┐ │ Remote Sequencer │ │ (HTTP or gRPC) │ └───────────────────┘ ``` **SQLite Tables:** ```sql theme={null} -- Local event outbox CREATE TABLE outbox ( local_seq INTEGER PRIMARY KEY, event_id TEXT UNIQUE NOT NULL, payload TEXT NOT NULL, signature TEXT NOT NULL, pushed_at TEXT, remote_seq INTEGER ); -- Sync state tracking CREATE TABLE sync_state ( key TEXT PRIMARY KEY, value TEXT NOT NULL ); ``` ## Scalability Considerations ### Horizontal Scaling * **Stateless API**: Multiple sequencer instances behind load balancer * **Read/Write Pool Splitting**: Separate connection pools for reads (replica) and writes (primary) * **Database Pooling**: Configurable pool size, acquire timeout, idle timeout, max lifetime * **Sequence Partitioning**: Each `(tenant_id, store_id)` is independent ### Performance Optimizations * **Batch Inserts**: Events ingested in batches with parallel partitioning * **Read Replicas**: Entity history and read queries served from replica pool * **Multi-Layer Caching**: LRU caches for commitments, proofs, schemas, and agent keys * **Proof Memoization**: Common proof paths cached with TTL * **Connection Pool Monitoring**: Automatic health degradation detection ### Resilience * **Circuit Breakers**: External service calls (L2 anchoring) protected with exponential backoff * **Dead Letter Queue**: Failed projections queued with automatic retry * **Graceful Shutdown**: Request draining on SIGTERM * **Rate Limiting**: Per-tenant sliding-window rate limiter ### Capacity Planning | Component | Recommended | | ---------- | ------------------------------------- | | PostgreSQL | 16+ GB RAM, SSD storage | | Sequencer | 2-4 CPU cores, 4 GB RAM | | Write Pool | 10-20 connections per instance | | Read Pool | 10-20 connections per instance | | Events/sec | \~1000-5000 depending on payload size | ## Security Model ### Trust Boundaries 1. **Agent -> Sequencer**: TLS + Ed25519 signatures + API key/JWT auth 2. **Sequencer -> Database**: Network isolation + credentials + session timeouts 3. **Sequencer -> L2 Chain**: Private key for signing + circuit breaker 4. **Agent -> Agent (payments)**: Ed25519 signed payment intents + nonce replay protection ### Key Management * Agent private keys: Never leave agent, stored securely * Sequencer signing key: For receipt signing (`VES_SEQUENCER_SIGNING_KEY`) * Sequencer anchor key: For L2 transactions (`SEQUENCER_PRIVATE_KEY`) * API keys: SHA-256 hashed, stored in PostgreSQL * Database credentials: Environment variables, not in code * Encryption keys: AES-256-GCM for payload encryption-at-rest ### Audit Trail All administrative operations logged via the audit system: * API key lifecycle (create, revoke, update) * Schema registry changes * Agent key operations * Authentication events * Dead letter queue operations See [SECURITY.md](docs/SECURITY.md) for detailed security guidance. ## Configuration ### Feature Flags (Cargo Features) | Feature | Default | Description | | ------------------- | ------- | ----------------------------------- | | `full` | Yes | Enables all features | | `grpc` | Yes | gRPC service (tonic/prost) | | `telemetry` | Yes | OpenTelemetry distributed tracing | | `anchoring` | Yes | L2 blockchain anchoring (alloy) | | `schema-validation` | Yes | JSON Schema validation (jsonschema) | | `sqlite` | No | SQLite backend for local agents | | `encryption` | No | Payload encryption at rest | ### Key Environment Variables | Variable | Description | | ----------------------------- | ---------------------------------------- | | `DATABASE_URL` | PostgreSQL connection URL (primary) | | `READ_DATABASE_URL` | PostgreSQL connection URL (read replica) | | `PORT` | HTTP listen port (default: 8080) | | `GRPC_PORT` | gRPC listen port (default: PORT + 1) | | `GRPC_DISABLED` | Disable gRPC server | | `AUTH_MODE` | `required` (default) or `disabled` | | `BOOTSTRAP_ADMIN_API_KEY` | Initial admin API key | | `JWT_SECRET` | JWT signing secret | | `VES_SEQUENCER_ID` | Pinned sequencer UUID | | `VES_SEQUENCER_SIGNING_KEY` | Ed25519 key for receipt signing | | `PAYLOAD_ENCRYPTION_MODE` | `disabled`, `optional`, `required` | | `SCHEMA_VALIDATION_MODE` | `disabled`, `warn`, `required`, `strict` | | `RATE_LIMIT_PER_MINUTE` | Request rate limit | | `L2_RPC_URL` | Ethereum L2 RPC endpoint | | `SET_REGISTRY_ADDRESS` | StateSetAnchor contract address | | `SEQUENCER_PRIVATE_KEY` | Anchor transaction signing key | | `OTEL_EXPORTER_OTLP_ENDPOINT` | OpenTelemetry OTLP endpoint | | `LOG_FORMAT` | `json` or `text` (default) | | `CORS_ALLOW_ORIGINS` | Comma-separated origins or `*` | | `DB_MIGRATE_ON_STARTUP` | Run migrations on boot (default: true) | | `CACHE_*` | Cache size and TTL overrides | | `MAX_DB_CONNECTIONS` | Write pool max connections | | `READ_MAX_DB_CONNECTIONS` | Read pool max connections | | `DB_STATEMENT_TIMEOUT_MS` | PostgreSQL statement timeout | | `DB_IDLE_IN_TX_TIMEOUT_MS` | Idle-in-transaction timeout | ## Technology Stack | Component | Technology | | ----------------- | --------------------------------------------------------- | | Language | Rust (Edition 2021) | | Web Framework | Axum 0.7 | | gRPC Framework | Tonic 0.12, Prost 0.13 | | Async Runtime | Tokio | | Database | PostgreSQL 16+ (sqlx 0.8) | | Local Storage | SQLite (sqlx) | | Cryptography | ed25519-dalek 2, sha2, aes-gcm | | Key Exchange | x25519-dalek 2, hpke 0.12 | | Merkle Trees | rs\_merkle (custom domain separation) | | Blockchain | Alloy 0.8 (Ethereum/EVM, Solidity ABI) | | Schema Validation | jsonschema 0.26 | | Authentication | jsonwebtoken 9 | | Serialization | serde, serde\_json, serde\_json\_canonicalizer (RFC 8785) | | Observability | OpenTelemetry 0.24, tracing, Prometheus export | | Testing | proptest, mockall, criterion, fake | ## Binaries | Binary | Path | Description | | -------------------------- | ------------------ | ------------------------------------- | | `stateset-sequencer` | `src/main.rs` | Main sequencer server (HTTP + gRPC) | | `stateset-sequencer-admin` | `src/bin/admin.rs` | Admin CLI for key management, DLQ ops | ## Module Structure ``` src/ ├── main.rs # Entry point ├── lib.rs # Library exports ├── server.rs # HTTP + gRPC server bootstrap, config ├── anchor.rs # On-chain anchoring (Alloy/Ethereum) │ ├── api/ │ ├── mod.rs # REST router definition │ └── handlers/ │ ├── agent_keys.rs # Agent key registration │ ├── anchoring.rs # Commitment anchoring │ ├── commitments.rs # Commitment CRUD │ ├── events.rs # Event listing/retrieval │ ├── health.rs # Health, readiness, detailed checks │ ├── ingest.rs # Event ingestion pipeline │ ├── proofs.rs # Merkle proof generation/verification │ ├── schemas.rs # Schema registry management │ ├── x402.rs # x402 payment protocol │ └── ves/ │ ├── mod.rs # VES v1.0 handler organization │ ├── anchoring.rs # VES commitment anchoring │ ├── commitments.rs # VES commitment management │ ├── compliance_proofs.rs # VES compliance proofs │ ├── inclusion_proofs.rs # VES inclusion proofs │ └── validity_proofs.rs # VES validity proofs │ ├── auth/ │ ├── mod.rs # Auth module (Authenticator, ApiKeyValidator, JwtValidator) │ ├── agent_keys.rs # Agent key registry │ └── middleware.rs # Axum auth middleware + gRPC interceptor │ ├── crypto/ │ ├── hash.rs # Domain-separated SHA-256 hashing │ ├── signing.rs # Ed25519 operations │ └── encrypt.rs # HPKE + AES-256-GCM encryption │ ├── domain/ │ ├── types.rs # Core types (TenantId, StoreId, AgentId, etc.) │ ├── event.rs # EventEnvelope │ ├── commitment.rs # BatchCommitment, MerkleProof │ ├── schema.rs # Schema, SchemaId, SchemaCompatibility │ ├── ves_event.rs # VES v1.0 events │ ├── ves_commitment.rs # VES batch commitments │ ├── ves_compliance.rs # VES compliance proof types │ ├── ves_validity.rs # VES validity proof types │ └── x402_payment.rs # x402 payment intents, batches, receipts │ ├── grpc/ │ ├── mod.rs # gRPC module │ ├── service.rs # gRPC v1 service implementation │ ├── service_v2.rs # gRPC v2 service (streaming, key management) │ └── interceptor.rs # gRPC auth interceptor │ ├── infra/ │ ├── mod.rs # Infrastructure module + trait exports │ ├── traits.rs # Core service traits (EventStore, Sequencer, etc.) │ ├── error.rs # SequencerError, contextual errors │ ├── audit.rs # Audit logging │ ├── batch.rs # Batch operations, deduplication │ ├── cache.rs # Multi-layer LRU caching │ ├── circuit_breaker.rs # Circuit breaker pattern │ ├── dead_letter.rs # Dead letter queue │ ├── graceful_shutdown.rs # Graceful shutdown coordination │ ├── payload_encryption.rs # Encryption-at-rest configuration │ ├── pool_monitor.rs # Connection pool health monitoring │ ├── retry.rs # Exponential backoff retry │ ├── schema_validation.rs # Schema validation mode handling │ ├── x402_batch_worker.rs # x402 batch assembly background worker │ ├── ves_commitment.rs # PgVesCommitmentEngine │ ├── ves_compliance.rs # PgVesComplianceProofStore │ ├── ves_validity.rs # PgVesValidityProofStore │ ├── postgres/ │ │ ├── sequencer.rs # PgSequencer (legacy) │ │ ├── ves_sequencer.rs # VesSequencer (VES v1.0) │ │ ├── event_store.rs # PgEventStore │ │ ├── commitment.rs # PgCommitmentEngine │ │ ├── agent_key_registry.rs # PgAgentKeyRegistry │ │ ├── schema_store.rs # PgSchemaStore │ │ └── x402_repository.rs # PgX402Repository │ └── sqlite/ │ └── outbox.rs # SqliteOutbox (local agents) │ ├── metrics/ │ └── mod.rs # MetricsRegistry, ComponentMetrics, Prometheus export │ ├── migrations/ │ ├── mod.rs # Migration runner │ ├── postgres/ # 11 PostgreSQL migrations │ └── sqlite/ # 1 SQLite migration │ ├── projection/ │ ├── runner.rs # Projection executor │ └── handlers.rs # Domain projection handlers │ ├── proto/ # Generated protobuf code │ ├── mod.rs # Proto module (v1 + v2) │ └── v2/ # gRPC v2 generated code │ ├── telemetry/ │ └── mod.rs # OpenTelemetry setup, OTLP export │ └── bin/ └── admin.rs # Admin CLI binary ``` ## Related Documentation * [Getting Started](GETTING_STARTED.md) - Quick start guide * [System Overview](SYSTEM_OVERVIEW.md) - High-level system overview * [VES Specification](docs/VES_SPEC.md) - Full protocol spec * [API Reference](docs/API_REFERENCE.md) - REST API documentation * [Event Types](docs/EVENT_TYPES.md) - Supported event types * [Deployment Guide](docs/DEPLOYMENT.md) - Production deployment * [Runbook](docs/RUNBOOK.md) - Operational runbook * [Security Guide](docs/SECURITY.md) - Security best practices * [ZK Integration Guide](docs/ZK_INTEGRATION_GUIDE.md) - STARK proof integration * [Agent Integration](docs/AGENT_INTEGRATION.md) - Agent SDK integration guide * [Anchoring Overview](docs/ANCHORING_OVERVIEW.md) - On-chain anchoring details ## Related Repositories | Repository | Purpose | | ---------------- | ------------------------------------------------------------ | | `stateset-stark` | STARK proving system for compliance proofs | | `@stateset/cli` | AI agent CLI with local SQLite outbox | | `set-chain` | Ethereum L2 for anchoring commitments and payment settlement | ## Next steps The system this architecture describes. What stateset-stark produces. Where sequenced batches settle. Running it day to day. # StateSet VES System Overview Source: https://docs.stateset.com/stateset-sequencer/stateset-sequencer The world's first operating system for autonomous agents with native USDC wallets and cross-web state management **Early preview.** This section describes a surface that is not generally available; interfaces here may change without notice. Talk to us before building against it. # Verifiable Event Sync (VES) System Overview A complete zero-knowledge commerce infrastructure enabling AI agents to interact with cryptographic verification, STARK proofs, and on-chain anchoring. ## System Architecture ``` ┌─────────────────────────────────────────────────────────────────────────────────────┐ │ AI Agent Commerce Platform │ ├─────────────────────────────────────────────────────────────────────────────────────┤ │ │ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │ │ AI Agent │ │ AI Agent │ │ AI Agent │ │ AI Agent │ │ │ │ (Orders) │ │ (Inventory) │ │ (Payments) │ │ (Returns) │ │ │ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │ │ │ │ │ │ │ │ └──────────────────┴────────┬─────────┴──────────────────┘ │ │ │ │ │ ▼ │ │ ┌─────────────────────────────────────────────────────────────────────────────┐ │ │ │ Stateset CLI (MCP Server) │ │ │ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │ │ │ │ │ Outbox │ │ Ed25519 │ │ HPKE │ │ Event Capture │ │ │ │ │ │ (SQLite) │ │ Signing │ │ Encryption │ │ & Serialization │ │ │ │ │ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────────────┘ │ │ │ └────────────────────────────────────┬────────────────────────────────────────┘ │ │ │ │ └───────────────────────────────────────┼────────────────────────────────────────────┘ │ VES Protocol v1.0 ▼ ┌─────────────────────────────────────────────────────────────────────────────────────┐ │ stateset-sequencer (Rust) │ ├─────────────────────────────────────────────────────────────────────────────────────┤ │ │ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌───────────────┐ │ │ │ Event Ingest │ │ Sequencing │ │ Merkle Trees │ │ Commitments │ │ │ │ (REST/gRPC) │ │ (Deterministic)│ │ (rs_merkle) │ │ (Batches) │ │ │ └────────┬────────┘ └────────┬────────┘ └────────┬────────┘ └───────┬───────┘ │ │ │ │ │ │ │ │ └────────────────────┴────────────────────┴───────────────────┘ │ │ │ │ │ ┌──────────────────────────────────────┴──────────────────────────────────────┐ │ │ │ Event Store (PostgreSQL/SQLite) │ │ │ └──────────────────────────────────────────────────────────────────────────────┘ │ │ │ │ └─────────────────────────────────────────┼──────────────────────────────────────────┘ │ Batch Events ▼ ┌─────────────────────────────────────────────────────────────────────────────────────┐ │ stateset-stark (Rust) │ ├─────────────────────────────────────────────────────────────────────────────────────┤ │ │ │ ┌─────────────────────────────────────────────────────────────────────────────┐ │ │ │ STARK Prover │ │ │ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │ │ │ │ │ Witness │ │ Trace │ │ AIR │ │ Winterfell │ │ │ │ │ │ Builder │ │ Generator │ │ Constraints │ │ Prover │ │ │ │ │ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────────────┘ │ │ │ └─────────────────────────────────────────────────────────────────────────────┘ │ │ │ │ ┌─────────────────────────────────────────────────────────────────────────────┐ │ │ │ Supported Policies │ │ │ │ • aml.threshold - Proves amount < threshold (AML compliance) │ │ │ │ • order_total.cap - Proves amount <= cap (Order limits) │ │ │ └─────────────────────────────────────────────────────────────────────────────┘ │ │ │ │ │ │ STARK Proofs │ │ ▼ │ │ ┌─────────────────────────────────────────────────────────────────────────────┐ │ │ │ STARK Verifier │ │ │ │ • Proof verification in ~600µs │ │ │ │ • Public inputs validation │ │ │ │ • Policy compliance checking │ │ │ └─────────────────────────────────────────────────────────────────────────────┘ │ │ │ │ └─────────────────────────────────────────┼──────────────────────────────────────────┘ │ Verified Proofs ▼ ┌─────────────────────────────────────────────────────────────────────────────────────┐ │ set/anchor (Rust) │ ├─────────────────────────────────────────────────────────────────────────────────────┤ │ │ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │ │ Sequencer │ │ Registry │ │ Health │ │ │ │ API Client │ │ Client │ │ Monitoring │ │ │ └────────┬────────┘ └────────┬────────┘ └─────────────────┘ │ │ │ │ │ │ └────────────────────┴─────────────────────────┐ │ │ │ │ └──────────────────────────────────────────────────────────┼─────────────────────────┘ │ On-chain TX ▼ ┌─────────────────────────────────────────────────────────────────────────────────────┐ │ Set L2 (EVM-Compatible Chain) │ ├─────────────────────────────────────────────────────────────────────────────────────┤ │ │ │ ┌─────────────────────────────────────────────────────────────────────────────┐ │ │ │ SetRegistry.sol │ │ │ │ │ │ │ │ struct BatchCommitment { │ │ │ │ bytes32 eventsRoot; // Merkle root of events │ │ │ │ bytes32 prevStateRoot; // Previous state │ │ │ │ bytes32 newStateRoot; // New state after batch │ │ │ │ uint64 sequenceStart; // First sequence number │ │ │ │ uint64 sequenceEnd; // Last sequence number │ │ │ │ uint32 eventCount; // Events in batch │ │ │ │ } │ │ │ │ │ │ │ │ struct StarkProofCommitment { │ │ │ │ bytes32 proofHash; // Hash of STARK proof │ │ │ │ bytes32 policyHash; // Policy used │ │ │ │ bool allCompliant; // Compliance status │ │ │ │ } │ │ │ │ │ │ │ │ Functions: │ │ │ │ • commitBatch() - Anchor batch commitment │ │ │ │ • commitStarkProof() - Anchor STARK proof │ │ │ │ • verifyInclusion() - Verify event in batch │ │ │ │ │ │ │ └─────────────────────────────────────────────────────────────────────────────┘ │ │ │ └─────────────────────────────────────────────────────────────────────────────────────┘ ``` ## Data Flow ### Step 1: AI Agent Creates Event ```javascript theme={null} // AI Agent creates a commerce event through the CLI const event = { entityType: 'order', entityId: 'order-001', eventType: 'OrderCreated', payload: { orderId: 'order-001', customerId: 'customer-001', items: [{ sku: 'WIDGET-001', quantity: 2, price: 29.99 }], total: 59.98 } }; // CLI captures, signs, and encrypts the event outbox.append(event); ``` ### Step 2: Event Signing & Encryption (CLI) The CLI performs VES v1.0 protocol operations: 1. **Payload Hash**: `SHA-256(domain_prefix || canonical_json(payload))` 2. **Ed25519 Signature**: Signs event envelope with agent's private key 3. **HPKE Encryption**: Encrypts payload for authorized recipients 4. **Cipher Hash**: `SHA-256(domain_prefix || ciphertext)` ``` Event Envelope: ├── eventId: UUID ├── vesVersion: 1 ├── payloadKind: 1 (encrypted) ├── payloadPlainHash: "0x..." ├── payloadCipherHash: "0x..." ├── agentKeyId: 1 ├── agentSignature: "0x..." └── payloadEncrypted: { ... } ``` ### Step 3: Sequencing (stateset-sequencer) The sequencer: 1. Validates agent signature 2. Assigns deterministic sequence number 3. Adds event to Merkle tree 4. Creates batch when threshold reached ``` Sequenced Event: ├── envelope: { ... } ├── sequenceNumber: 42 ├── sequencedAt: "2024-12-22T20:15:00Z" └── receiptHash: "0x..." ``` ### Step 4: STARK Proof Generation (stateset-stark) For each batch, generate a STARK proof: ```bash theme={null} # Prove compliance: amount < 10000 (AML threshold) ves-stark prove \ --amount 5000 \ --limit 10000 \ --policy aml.threshold \ --inputs public_inputs.json \ --output proof.json ``` **Proof Characteristics:** * Proof Size: \~36KB (individual), \~53KB (batch) * Proving Time: \~20-25ms * Verification Time: \~600µs * Security Level: 128-bit ### Step 5: On-Chain Anchoring (set/anchor → SetRegistry) The anchor service submits to Set L2: ```solidity theme={null} // SetRegistry.commitBatch() commitBatch( batchId, // Unique batch identifier tenantId, // Tenant UUID as bytes32 storeId, // Store UUID as bytes32 eventsRoot, // Merkle root of events prevStateRoot, // State before batch newStateRoot, // State after batch sequenceStart, // 0 sequenceEnd, // 7 eventCount // 8 ); // SetRegistry.commitStarkProof() commitStarkProof( batchId, proofHash, // SHA-256 of STARK proof policyHash, // Policy identifier hash policyLimit, // 10000 allCompliant, // true proofSize, // 53074 provingTimeMs // 25 ); ``` ### Step 6: Verification by Other Agents Any AI agent can verify: 1. **Event Inclusion**: Merkle proof against on-chain root 2. **Compliance**: STARK proof verification 3. **State Transition**: Verify prev\_state → new\_state ```bash theme={null} # Verify a STARK proof ves-stark verify \ --proof proof.json \ --inputs public_inputs.json \ --limit 10000 \ --policy aml.threshold # Output: Proof VALID (verified in 622.133µs) ``` ## Repository Structure | Repository | Path | Description | | ---------------------- | ---------------------------------- | ------------------------- | | **stateset-sequencer** | `icommerce-app/stateset-sequencer` | VES protocol sequencer | | **stateset-stark** | `icommerce-app/stateset-stark` | STARK prover/verifier | | **set** | `icommerce-app/set` | L2 chain & anchor service | | **CLI** | `stateset-icommerce/cli` | AI agent MCP server | ## Crate Structure (stateset-stark) ``` stateset-stark/crates/ ├── ves-stark-primitives/ # Field arithmetic, Rescue hash ├── ves-stark-air/ # AIR constraints for policies ├── ves-stark-prover/ # Witness & proof generation ├── ves-stark-verifier/ # Proof verification ├── ves-stark-batch/ # zkRollup batch proofs ├── ves-stark-cli/ # Command-line interface └── ves-stark-client/ # HTTP client for sequencer ``` ## CLI Commands ### STARK Prover CLI ```bash theme={null} # Generate public inputs ves-stark gen-inputs --limit 10000 --policy aml.threshold -o inputs.json # Generate compliance proof ves-stark prove --amount 5000 --limit 10000 --policy aml.threshold \ --inputs inputs.json --output proof.json --json # Verify proof ves-stark verify --proof proof.json --inputs inputs.json \ --limit 10000 --policy aml.threshold # Inspect proof metadata ves-stark inspect --proof proof.json # Run benchmark ves-stark benchmark -n 10 --max-amount 10000 --limit 10000 # Generate batch proof (zkRollup style) ves-stark batch-prove -n 8 --limit 10000 --output batch_proof.json # Run sequencer simulation ves-stark sequencer -n 16 --batch-size 8 --limit 10000 \ --output-dir ./proofs ``` ### Sync CLI ```bash theme={null} # Initialize sync configuration stateset-sync init # Generate agent keys stateset-sync keys:generate # Push events to sequencer stateset-sync push # Pull events from sequencer stateset-sync pull # Show sync status stateset-sync status ``` The performance numbers below are measured on the reference deployment described in this page. Treat them as a shape — where the cost sits, and what scales with what — rather than as a figure to quote in a capacity plan for different hardware. ## Performance Metrics | Operation | Time | Size | | --------------------------- | ------- | ------ | | Individual Proof Generation | \~20ms | \~36KB | | Batch Proof (8 events) | \~25ms | \~53KB | | Proof Verification | \~600µs | - | | Merkle Proof Verification | 1ms | 1KB | ## Security Properties 1. **Privacy**: Event payloads encrypted with HPKE 2. **Authenticity**: Ed25519 signatures on all events 3. **Ordering**: Deterministic sequencing prevents reordering 4. **Compliance**: Zero-knowledge proofs for policy enforcement 5. **Finality**: On-chain anchoring provides immutability 6. **Verifiability**: Anyone can verify proofs without trusted setup ## Running the Demo ```bash theme={null} # Run the complete demonstration ./run-ves-demo.sh ``` See `run-ves-demo.sh` for the full demonstration script. ## Next steps The design behind the components above. The agents this sequences for. The wallet flow it settles. The settlement layer underneath. # Sequencer Operations Source: https://docs.stateset.com/stateset-sequencer/stateset-sequencer-operations Running the VES sequencer — the ingest pipeline, batch lifecycle, what to monitor, and the dead-letter queue. ## The ingest pipeline Every event passes through the same ordered stages. Knowing the order tells you where a rejection came from: ``` Request → Auth → Rate Limit → Validate Schema → Verify Signature → Dedupe → Sequence ``` | Stage | Rejects when | | -------------------- | -------------------------------------------------- | | **Auth** | Missing or invalid `ApiKey` | | **Rate limit** | Sliding-window per-tenant budget exceeded | | **Validate schema** | Payload doesn't match the registered schema | | **Verify signature** | Signature doesn't match the agent's registered key | | **Dedupe** | Replay of an already-sequenced event | A rejection at *dedupe* is usually a client retry working correctly, not an error. A rejection at *verify signature* on previously-working traffic usually means a key rotation that didn't complete. ## Batch lifecycle 1. Events are accepted and assigned sequence numbers within their stream. 2. A batch closes and its Merkle tree is built over event leaves. 3. A commitment is emitted — `prev_state_root`, `new_state_root`, `events_root`, sequence range. 4. The commitment is anchored on [Set L2](/set/stateset-set-l2). 5. Receipts and inclusion proofs become available to clients. Steps 3 and 4 are distinct, and the gap matters. After step 3 you have a **sequenced** event; only after step 4 is it **verifiable** by a third party. A receipt issued between the two proves sequencing, not settlement. ## What to monitor | Signal | Why | | ---------------------------------- | -------------------------------------------------------------------------- | | **Event ingest rate** | Baseline for everything else | | **Batch size** | Batches closing small means low throughput or an aggressive close interval | | **Commit latency** | Time from event accepted to commitment emitted | | **Anchor lag** | Commitments emitted but not yet on-chain — this is the verifiability gap | | **Dead-letter depth** | Events the pipeline permanently rejected | | **Rate-limit rejections** | A client hitting its budget, or an agent in a loop | | **Storage health and replay time** | Replay is how you rebuild state; if it's slow, recovery is slow | Proof-conflict counters are worth alerting on specifically — `ves_compliance_proof_conflict`, `ves_compliance_proof_public_inputs_conflict`, and `ves_validity_proof_conflict`. A conflict means two proofs disagree for the same event, which is a correctness signal rather than a capacity one. ## Dead letters Events the sequencer permanently rejected land in a dead-letter queue rather than vanishing. Treat non-zero depth as a real alert: something upstream is producing events the pipeline won't accept, and every one of them is missing from the anchored history. ## Anchor lag is the one that matters Ingest and batching can be healthy while anchoring is stalled. When that happens, events keep sequencing and nothing is externally verifiable — which is invisible until someone tries to verify. ```bash theme={null} curl -H "Authorization: ApiKey $SEQ_API_KEY" \ http://localhost:8080/api/v1/anchor/status ``` Alert on anchor lag independently of ingest health. ## Health probes | Endpoint | Checks | | ------------- | -------------------------- | | `GET /health` | The process is up | | `GET /ready` | Plus database connectivity | Use `/ready` for load-balancer membership; `/health` alone will keep a sequencer in rotation while it can't reach Postgres. ## Admin surface | Endpoint | Purpose | | ----------------------------------------------- | ----------------- | | `GET /v1/admin/overview` | Fleet-level state | | `GET /v1/admin/tenants` · `/stores` · `/agents` | Per-entity views | ## Related * [Architecture deep dive](/stateset-sequencer-architecture) * [Quickstart](/stateset-sequencer/stateset-sequencer-quickstart) * [Set L2 Verification](/set/stateset-set-l2-verification) # Sequencer Quickstart Source: https://docs.stateset.com/stateset-sequencer/stateset-sequencer-quickstart Run the VES sequencer, ingest a signed event, and read back an inclusion proof. The sequencer orders events, commits them to Merkle roots, and anchors those commitments so anyone can verify inclusion without trusting it. This gets one running and walks a single event through. ## Run it ```bash theme={null} docker-compose up -d curl http://localhost:8080/health # process up curl http://localhost:8080/ready # + database connectivity ``` Authentication is **`Authorization: ApiKey `**. A bootstrap admin key exists for local development: ```bash theme={null} curl -H "Authorization: ApiKey dev_admin_key" \ "http://localhost:8080/api/v1/head?tenant_id=&store_id=" ``` `dev_admin_key` is a development bootstrap credential. Replace it before anything reachable from outside your machine. ## 1. Register an agent key Events are signed, so the sequencer needs the agent's public key first. ```bash theme={null} curl -X POST http://localhost:8080/api/v1/agents/register \ -H "Authorization: ApiKey $SEQ_API_KEY" \ -H 'content-type: application/json' \ -d '{ … }' ``` Keys are registered with **proof of possession** — the agent demonstrates it holds the private key rather than merely asserting a public one. See `/api/v1/agents/keys`. ## 2. Ingest an event ```bash theme={null} curl -X POST http://localhost:8080/api/v1/events/ingest \ -H "Authorization: ApiKey $SEQ_API_KEY" \ -H 'content-type: application/json' \ -d '{ … }' ``` The event gets a sequence number in its stream, and its leaf enters the next batch. ## 3. Read the head ```bash theme={null} curl -H "Authorization: ApiKey $SEQ_API_KEY" \ "http://localhost:8080/api/v1/head?tenant_id=$TENANT&store_id=$STORE" ``` `head` is the current sequence position for that tenant and store — how you know whether your event landed. ## 4. Get the commitment and an inclusion proof ```bash theme={null} curl -H "Authorization: ApiKey $SEQ_API_KEY" \ http://localhost:8080/api/v1/commitments/$BATCH_ID ``` That returns the batch's `events_root`, state roots, and sequence range. To verify an individual event, pair it with an inclusion proof from the gRPC v2 service (`GetInclusionProof`) and recompute the root locally. Recompute against the root **read from the chain**, not the one this API returns. Comparing an API-supplied root to an API-supplied proof verifies nothing — the on-chain value is what makes it trust-minimised. Full walkthrough: [verification example](/set/stateset-set-l2-verification-example). ## 5. Anchor ```bash theme={null} curl -X POST http://localhost:8080/api/v1/anchor \ -H "Authorization: ApiKey $SEQ_API_KEY" curl -H "Authorization: ApiKey $SEQ_API_KEY" \ http://localhost:8080/api/v1/anchor/status ``` Anchoring submits the commitment to [Set L2](/set/stateset-set-l2). Until it lands, you have a sequenced event; after, you have a verifiable one. ## Run the full demo ```bash theme={null} ./scripts/run_demo.sh # or run_e2e_demo.sh ``` ## Next What VES is and why it exists. Every component in detail. Running it in production. Prove inclusion yourself. # x402 payments reference Source: https://docs.stateset.com/stateset-sequencer/stateset-sequencer-x402 Complete reference for sequencer x402 payment endpoints. Reference for the sequencer endpoints that handle x402 payment intents. ## Properties Tenant UUID that owns the payment intent. Store UUID used for sequencing and batching. Agent UUID that signed the payment intent. Payer address for the intent. Payee address for the intent. Amount in smallest units (e.g., 1,000,000 = 1 USDC). ## Examples ### Submit an intent ```bash theme={null} curl -X POST https://api.sequencer.stateset.com/api/v1/x402/payments \ -H "Content-Type: application/json" \ -d '{ "tenant_id": "uuid", "store_id": "uuid", "agent_id": "uuid", "payer_address": "0x...", "payee_address": "0x...", "amount": 1000000, "asset": "usdc", "network": "set_chain", "valid_until": 1705320000, "nonce": 42, "signing_hash": "0x...", "payer_signature": "0x..." }' ``` ## Response The ID of the sequenced payment intent. Current intent status (e.g., `sequenced`). ## Related references * [x402 Payments with iCommerce](/stateset-icommerce/stateset-icommerce-x402-payments) * [Set L2 Verification](/set/stateset-set-l2-verification) `signing_hash` is not free-form — it must be SHA-256 over the exact domain-separated preimage (`X402_PAYMENT_V1`, field order fixed, integers big-endian), signed with Ed25519. Build it from the [walkthrough](/stateset-x402-walkthrough#step-2--build-and-sign-the-intent); an intent whose hash was computed any other way is rejected. ## Nonces Each `nonce` is reserved per payer on first use. A resubmitted intent fails with `Nonce already used for this payer` — that is replay protection working, not an error to retry through. Sign a fresh intent with a new nonce instead. ## Related * [x402 walkthrough](/stateset-x402-walkthrough) — the full flow, 402 to verified receipt * [Sequencer architecture](/stateset-sequencer-architecture) — batching and anchoring * [Set L2](/set/stateset-set-l2) — where the batch commitment lands # Sync Server MCP Source: https://docs.stateset.com/stateset-sync-mcp Connect an MCP host to a Sync Server tenant — four transports, per-tenant policy, and admin controls. The Sync Server speaks MCP (JSON-RPC 2.0, protocol version `2025-03-26`) so an agent can drive order, inventory, product, and returns operations across 180+ integrations for a given tenant. ## What it exposes The registry holds **186 general-purpose tools** across 29 categories, plus a small number of tenant-specific operations built for individual accounts. | Category | Tools | | Category | Tools | | ---------- | ----- | - | ----------------- | ----- | | Monitoring | 21 | | Reconciliation | 8 | | Inventory | 16 | | Analytics | 8 | | Shopify | 14 | | Connectors | 7 | | NetSuite | 11 | | Schedules | 6 | | Webhooks | 10 | | Dead-letter queue | 6 | | Jobs | 10 | | Walmart | 5 | | Orders | 9 | | Tracking | 5 | | Flows | 9 | | Returns | 5 | | Billing | 9 | | Sync | 3 | | Tenant | 8 | | Agent | 3 | The remainder are one or two tools each for Amazon, eBay, Etsy, TikTok, MercadoLibre, Shopware, WooCommerce, products, and audit. `tools/list` does not return all of them. It defaults to `availableOnly`, hiding any tool whose integration is not configured for the tenant, and the read-only mode and per-tenant blocklist below narrow it further. Count against your own tenant rather than against this table. ## Transports Four, depending on where the agent runs: | Transport | Endpoint | Auth | Use for | | ------------------------- | ---------------------------------------------- | --------------------------------------- | ------------------------------------------------------- | | **HTTP** | `POST /v1/tenants/{tenant_id}/mcp` | `x-stateset-api-key` or `Bearer` | Server-to-server, hosted AI, back-end agents | | **Streamable HTTP / SSE** | `GET /v1/tenants/{tenant_id}/mcp/sse` | Same | Server-push notifications — progress, resource updates | | **stdio** | `stateset-sync-server mcp-stdio --tenant ` | Env (`APP_CONFIG_PATH`, `DATABASE_URL`) | Desktop clients launching the binary as a child process | | **Multi-tenant HTTP** | `POST /v1/mcp` | `x-stateset-admin-key` | One operator connection addressing any loaded tenant | The multi-tenant transport takes the tenant per request via `params._meta.tenant`. ## Connect ### Remote (recommended for a hosted server) ```json theme={null} { "mcpServers": { "stateset-sync": { "command": "npx", "args": [ "-y", "mcp-remote", "https://api.sync.stateset.com/v1/tenants//mcp", "--header", "x-stateset-api-key: ss_sync_..." ] } } } ``` ### stdio (local binary) ```json theme={null} { "mcpServers": { "stateset-sync": { "command": "stateset-sync-server", "args": ["mcp-stdio", "--tenant", "acme"], "env": { "APP_CONFIG_PATH": "…", "DATABASE_URL": "…" } } } } ``` Both expose the same tool surface — pick based on where the agent runs, not on capability. ## Safety controls The Sync Server has the most developed per-tenant guardrails of any StateSet MCP server, because its tools reach live commerce platforms. ### Read-only mode Freeze a tenant so every mutating tool is refused while reads keep working. Useful during an incident. ### Tool blocklist Finer-grained than read-only — disable specific tools for a specific tenant and leave the rest available: ```bash theme={null} # Disable cancel_order for tenant 'acme' curl -X POST "https://sync.example.com/v1/admin/tenants/acme/disabled-tools/cancel_order" \ -H "x-stateset-admin-key: $ADMIN_KEY" # List what's blocked curl https://sync.example.com/v1/admin/tenants/acme/disabled-tools \ -H "x-stateset-admin-key: $ADMIN_KEY" # → { "tools": ["cancel_order", "create_order"], "count": 2 } # Re-enable curl -X DELETE "https://sync.example.com/v1/admin/tenants/acme/disabled-tools/cancel_order" \ -H "x-stateset-admin-key: $ADMIN_KEY" ``` Both read-only mode and the blocklist are **DB-persisted** (`mcp_tenant_policy`) and rehydrate into the in-memory registries at startup — so a pod restart mid-incident cannot silently unfreeze a tenant or re-enable a blocked tool. ### Result cache An opt-in per-tenant cache on tool results, for read-heavy agent loops. ## Operations | Surface | Purpose | | -------------------------- | ------------------------------------------ | | Runtime inspection (admin) | What's loaded and active per tenant | | Runtime diagnostics | Health of the MCP surface itself | | SSE subscriptions | Progress and resource-update notifications | Observability config ships with the repo — `docs/observability/mcp-alerts.yaml` and `mcp-dashboard.json`. ## claude.ai custom connector The server supports claude.ai Custom Connectors over **OAuth 2.1**, with a documented issuer surface and a dashboard handoff contract. This is the path for connecting a tenant to claude.ai without distributing an API key. ## Related * [Sync Server API Basics](/stateset-sync-server-api-basics) * [Sync Server: What's New](/stateset-sync-server-whats-new) * [All MCP servers](/mcp-servers) # Sync Server ACP Guide Source: https://docs.stateset.com/stateset-sync-server-acp-guide Agentic Commerce Protocol support in the StateSet Sync Server. # Agent Commerce Protocol (ACP) Guide The Agent Commerce Protocol (ACP) enables AI agents to place orders directly into your commerce infrastructure. With a single API call, an AI agent can create an order that automatically flows through to fulfillment and ERP systems. ## Overview ``` ┌─────────────────────────────────────────────────────────────────────┐ │ AI AGENT LAYER │ │ (ChatGPT, Claude, Custom Agents, Voice Assistants, Chatbots) │ └─────────────────────────────────────────────────────────────────────┘ │ │ POST /acp/orders ▼ ┌─────────────────────────────────────────────────────────────────────┐ │ STATESET SYNC SERVER │ │ │ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │ │ Validate │───▶│ Route │───▶│ Track │ │ │ │ Order │ │ Order │ │ Job │ │ │ └─────────────┘ └─────────────┘ └─────────────┘ │ └─────────────────────────────────────────────────────────────────────┘ │ ┌─────────────────┼─────────────────┐ ▼ ▼ ▼ ┌───────────────────┐ ┌───────────────────┐ ┌───────────────────┐ │ STATESET │ │ FULFILLMENT │ │ ERP │ │ │ │ │ │ │ │ • Order Record │ │ • DCL Warehouse │ │ • NetSuite │ │ • Line Items │ │ • Cart.com │ │ • Sales Order │ │ • Status Track │ │ • Ship to Cust. │ │ • Revenue Rec. │ └───────────────────┘ └───────────────────┘ └───────────────────┘ ``` ## Why ACP? **Traditional e-commerce** requires customers to navigate websites, add items to carts, and go through checkout flows. **Agent Commerce** allows AI assistants to handle the entire purchase flow conversationally: * "I'd like to order 2 blue widgets shipped to my office" * The AI collects details, confirms the order, and places it via ACP * The order is fulfilled and tracked automatically This enables: * **Conversational Commerce** - Order via ChatGPT, Claude, Alexa, etc. * **Automated Purchasing** - Agents that reorder supplies automatically * **B2B Order Entry** - Sales reps using AI to place orders * **Multi-channel Unification** - Single API for all agent-based ordering *** ## Quick Start ### 1. Basic Order Submission ```bash theme={null} curl -X POST https://your-server.com/v1/tenants/{tenant_id}/acp/orders \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "orderId": "agent-order-001", "orderNumber": "ORD-1001", "customer": { "email": "customer@example.com", "firstName": "John", "lastName": "Doe" }, "shippingAddress": { "name": "John Doe", "address1": "123 Main Street", "city": "San Francisco", "state": "CA", "postalCode": "94102", "countryCode": "US" }, "lineItems": [ { "sku": "WIDGET-001", "title": "Blue Widget", "quantity": 2, "price": 29.99 } ] }' ``` ### 2. Response ```json theme={null} { "data": { "accepted": true, "statesetOrderId": "550e8400-e29b-41d4-a716-446655440000", "fulfillment": { "provider": "dcl", "success": true, "externalId": "ORD-1001" }, "erp": { "provider": "netsuite", "success": true, "netsuiteId": "12345", "tranId": "SO-12345" }, "warnings": [], "requestId": "7c9e6679-7425-40de-944b-e07fc1f90ae7" }, "meta": { "requestId": "7c9e6679-7425-40de-944b-e07fc1f90ae7", "timestamp": "2024-01-15T10:30:00Z" } } ``` *** ACP checkout is a multi-step conversation, not one call, and the session is the unit of retry. Hold the session id from the first response and resend against it — restarting the flow after a timeout creates a second session, and the buyer can end up confirming the one you abandoned. ## API Reference ### Endpoints | Method | Endpoint | Description | | ------ | ------------------------------------------ | --------------------------- | | POST | `/v1/tenants/{tenant_id}/acp/orders` | Process order synchronously | | POST | `/v1/tenants/{tenant_id}/acp/orders/async` | Process order in background | | POST | `/v1/tenants/{tenant_id}/acp/orders/batch` | Process multiple orders | ### Authentication Include your API key in the Authorization header: ``` Authorization: Bearer your-api-key-here ``` Or use the `X-API-Key` header: ``` X-API-Key: your-api-key-here ``` *** ## Request Schema ### AcpOrderRequest | Field | Type | Required | Description | | ----------------- | ------------- | -------- | ------------------------------------------ | | `orderId` | string | Yes | Unique order identifier from source system | | `orderNumber` | string | Yes | Display order number | | `source` | string | No | Source system (default: "agent") | | `customer` | Customer | Yes | Customer information | | `shippingAddress` | Address | Yes | Shipping destination | | `billingAddress` | Address | No | Billing address (defaults to shipping) | | `lineItems` | LineItem\[] | Yes | Order line items (min 1) | | `createdAt` | datetime | No | Order timestamp (default: now) | | `tags` | string\[] | No | Tags for categorization | | `shippingMethod` | string | No | Requested shipping method | | `notes` | string | No | Order notes/instructions | | `routing` | RoutingConfig | No | Control where order is sent | | `metadata` | object | No | Additional custom data | ### Customer | Field | Type | Required | Description | | ------------ | ------ | -------- | ----------------------------- | | `id` | string | No | Customer ID in source system | | `email` | string | Yes | Customer email | | `firstName` | string | Yes | First name | | `lastName` | string | Yes | Last name | | `phone` | string | No | Phone number | | `netsuiteId` | string | No | NetSuite customer internal ID | ### Address | Field | Type | Required | Description | | ------------- | ------ | -------- | ---------------------------------- | | `name` | string | Yes | Full name | | `company` | string | No | Company name | | `address1` | string | Yes | Street address line 1 | | `address2` | string | No | Street address line 2 | | `city` | string | Yes | City | | `state` | string | Yes | State/province code | | `postalCode` | string | Yes | ZIP/postal code | | `countryCode` | string | Yes | 2-letter country code (e.g., "US") | | `phone` | string | No | Phone number | ### LineItem | Field | Type | Required | Description | | ---------------- | ------ | -------- | ----------------------------- | | `sku` | string | Yes | Product SKU | | `title` | string | Yes | Product title | | `quantity` | number | Yes | Quantity (must be > 0) | | `price` | number | Yes | Unit price | | `variantId` | string | No | Variant ID from source system | | `netsuiteItemId` | string | No | NetSuite item internal ID | | `weight` | number | No | Weight in pounds | ### RoutingConfig Control where the order is sent: | Field | Type | Default | Description | | --------------------- | ------- | ------- | ------------------------------- | | `sendToFulfillment` | boolean | true | Send to DCL/Cart.com | | `sendToErp` | boolean | true | Send to NetSuite | | `createInStateset` | boolean | true | Create in StateSet | | `fulfillmentProvider` | string | "auto" | "dcl", "cart", or "auto" | | `dclLocation` | string | null | Override DCL warehouse location | *** ## Routing Options ### Full Integration (Default) Order flows to all configured systems: ```json theme={null} { "routing": { "sendToFulfillment": true, "sendToErp": true, "createInStateset": true } } ``` ### Fulfillment Only Skip ERP, just fulfill the order: ```json theme={null} { "routing": { "sendToFulfillment": true, "sendToErp": false, "createInStateset": true } } ``` ### Record Only Just create the record, don't fulfill yet: ```json theme={null} { "routing": { "sendToFulfillment": false, "sendToErp": false, "createInStateset": true } } ``` ### Specific Fulfillment Provider Force a specific fulfillment provider: ```json theme={null} { "routing": { "fulfillmentProvider": "dcl", "dclLocation": "LA" } } ``` *** ## Async Processing For high-volume or time-sensitive responses, use async processing: ### Submit Order ```bash theme={null} curl -X POST https://your-server.com/v1/tenants/{tenant_id}/acp/orders/async \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ ... order data ... }' ``` ### Response ```json theme={null} { "data": { "jobId": "550e8400-e29b-41d4-a716-446655440000", "status": "queued", "message": "Order ORD-1001 accepted for processing" } } ``` ### Check Status ```bash theme={null} curl https://your-server.com/v1/tenants/{tenant_id}/jobs/{jobId} \ -H "Authorization: Bearer YOUR_API_KEY" ``` ### Job Status Response ```json theme={null} { "data": { "jobId": "550e8400-e29b-41d4-a716-446655440000", "jobType": "acp_order_process", "state": "succeeded", "createdAt": "2024-01-15T10:30:00Z", "startedAt": "2024-01-15T10:30:01Z", "finishedAt": "2024-01-15T10:30:03Z", "result": { "orderId": "agent-order-001", "orderNumber": "ORD-1001", "statesetOrderId": "550e8400-e29b-41d4-a716-446655440000", "fulfillment": { "provider": "dcl", "success": true }, "erp": { "provider": "netsuite", "success": true, "netsuiteId": "12345" } } } } ``` *** ## Batch Processing Process up to 100 orders in a single request: ```bash theme={null} curl -X POST https://your-server.com/v1/tenants/{tenant_id}/acp/orders/batch \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "orders": [ { "orderId": "order-001", "orderNumber": "ORD-001", ... }, { "orderId": "order-002", "orderNumber": "ORD-002", ... }, { "orderId": "order-003", "orderNumber": "ORD-003", ... } ] }' ``` ### Response ```json theme={null} { "data": { "jobId": "550e8400-e29b-41d4-a716-446655440000", "status": "queued", "message": "Batch of 3 orders accepted for processing" } } ``` ### Batch Result ```json theme={null} { "total": 3, "processed": 3, "succeeded": 2, "failed": 1, "results": [ { "orderId": "order-001", "success": true, "statesetOrderId": "..." }, { "orderId": "order-002", "success": true, "statesetOrderId": "..." }, { "orderId": "order-003", "success": false, "fulfillment": { "error": "..." } } ] } ``` *** ## Integration Examples ### ChatGPT Function Calling Define this function for ChatGPT to place orders: ```json theme={null} { "name": "place_order", "description": "Place an order for products", "parameters": { "type": "object", "properties": { "customerEmail": { "type": "string" }, "customerFirstName": { "type": "string" }, "customerLastName": { "type": "string" }, "shippingAddress": { "type": "object", "properties": { "name": { "type": "string" }, "address1": { "type": "string" }, "city": { "type": "string" }, "state": { "type": "string" }, "postalCode": { "type": "string" }, "countryCode": { "type": "string" } } }, "items": { "type": "array", "items": { "type": "object", "properties": { "sku": { "type": "string" }, "title": { "type": "string" }, "quantity": { "type": "number" }, "price": { "type": "number" } } } } }, "required": ["customerEmail", "customerFirstName", "customerLastName", "shippingAddress", "items"] } } ``` ### Python Integration ```python theme={null} import requests import uuid def place_acp_order(tenant_id: str, api_key: str, order_data: dict) -> dict: """Place an order via the ACP endpoint.""" url = f"https://your-server.com/v1/tenants/{tenant_id}/acp/orders" headers = { "Authorization": f"Bearer {api_key}", "Content-Type": "application/json" } # Ensure order has an ID if "orderId" not in order_data: order_data["orderId"] = f"agent-{uuid.uuid4()}" response = requests.post(url, json=order_data, headers=headers) response.raise_for_status() return response.json() # Example usage order = { "orderNumber": "ORD-1001", "customer": { "email": "customer@example.com", "firstName": "John", "lastName": "Doe" }, "shippingAddress": { "name": "John Doe", "address1": "123 Main St", "city": "San Francisco", "state": "CA", "postalCode": "94102", "countryCode": "US" }, "lineItems": [ {"sku": "WIDGET-001", "title": "Blue Widget", "quantity": 2, "price": 29.99} ] } result = place_acp_order("my-tenant", "my-api-key", order) print(f"Order accepted: {result['data']['accepted']}") print(f"NetSuite ID: {result['data']['erp']['netsuiteId']}") ``` ### Node.js Integration ```javascript theme={null} const axios = require('axios'); const { v4: uuidv4 } = require('uuid'); async function placeAcpOrder(tenantId, apiKey, orderData) { const url = `https://your-server.com/v1/tenants/${tenantId}/acp/orders`; // Ensure order has an ID if (!orderData.orderId) { orderData.orderId = `agent-${uuidv4()}`; } const response = await axios.post(url, orderData, { headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' } }); return response.data; } // Example usage const order = { orderNumber: 'ORD-1001', customer: { email: 'customer@example.com', firstName: 'John', lastName: 'Doe' }, shippingAddress: { name: 'John Doe', address1: '123 Main St', city: 'San Francisco', state: 'CA', postalCode: '94102', countryCode: 'US' }, lineItems: [ { sku: 'WIDGET-001', title: 'Blue Widget', quantity: 2, price: 29.99 } ] }; placeAcpOrder('my-tenant', 'my-api-key', order) .then(result => { console.log('Order accepted:', result.data.accepted); console.log('NetSuite ID:', result.data.erp.netsuiteId); }); ``` *** ## Error Handling ### Validation Errors (422) ```json theme={null} { "error": { "code": "validation_error", "message": "lineItems[0].sku is required; shippingAddress missing required fields: postalCode", "requestId": "7c9e6679-7425-40de-944b-e07fc1f90ae7" } } ``` ### Integration Not Configured (424) ```json theme={null} { "error": { "code": "integration_not_configured", "message": "DCL integration not configured for this tenant", "requestId": "7c9e6679-7425-40de-944b-e07fc1f90ae7" } } ``` ### Partial Success (207) When some integrations succeed but others fail: ```json theme={null} { "data": { "accepted": false, "statesetOrderId": "550e8400-e29b-41d4-a716-446655440000", "fulfillment": { "provider": "dcl", "success": true }, "erp": { "provider": "netsuite", "success": false, "error": "NetSuite API timeout" }, "warnings": ["ERP submission failed but order was fulfilled"] } } ``` *** ## Configuration ### Tenant Configuration Ensure your tenant has the required integrations configured in `app_config.json`: ```json theme={null} { "tenants": [ { "id": "my-tenant", "displayName": "My Store", "auth": { "apiKeys": ["your-api-key"] }, "stateset": { "graphqlUrl": "https://your-hasura.com/v1/graphql", "adminSecret": "your-admin-secret" }, "dcl": { "apiUrl": "https://api.dclcorp.com", "username": "dcl-user", "password": "dcl-pass", "accountNumber": "12345", "defaultLocation": "LA" }, "netsuite": { "domain": "123456.suitetalk.api.netsuite.com", "consumerKey": "...", "consumerSecret": "...", "tokenKey": "...", "tokenSecret": "...", "defaultCustomerId": "14", "defaultSubsidiaryId": "1" } } ] } ``` ### Environment Variables Sensitive values can be stored as environment variables: ```bash theme={null} MY_TENANT_DCL_PASSWORD=dcl-password MY_TENANT_NETSUITE_CONSUMER_SECRET=netsuite-secret MY_TENANT_NETSUITE_TOKEN_SECRET=token-secret ``` *** ## Best Practices ### 1. Use Idempotent Order IDs Always provide a unique, deterministic `orderId` to prevent duplicate orders: ```json theme={null} { "orderId": "chatgpt-session-abc123-order-1" } ``` ### 2. Validate Before Submitting Have your AI agent confirm order details with the customer before calling ACP. ### 3. Handle Partial Failures Check both `fulfillment.success` and `erp.success` in responses. An order might be fulfilled even if ERP entry fails. ### 4. Use Async for High Volume For bulk imports or high-traffic scenarios, use `/acp/orders/async` or `/acp/orders/batch`. ### 5. Include Customer NetSuite ID If you know the customer's NetSuite ID, include it to ensure proper customer linking: ```json theme={null} { "customer": { "email": "customer@example.com", "firstName": "John", "lastName": "Doe", "netsuiteId": "12345" } } ``` ### 6. Map SKUs to NetSuite Items For fastest NetSuite processing, include NetSuite item IDs: ```json theme={null} { "lineItems": [ { "sku": "WIDGET-001", "title": "Blue Widget", "quantity": 2, "price": 29.99, "netsuiteItemId": "5678" } ] } ``` *** ## Glossary | Term | Description | | ------------ | -------------------------------------------------------------------- | | **ACP** | Agent Commerce Protocol - API for AI agents to place orders | | **DCL** | Distributed Commerce Logistics - 3PL fulfillment provider | | **ERP** | Enterprise Resource Planning - Business management system (NetSuite) | | **StateSet** | Order management and tracking system | | **Tenant** | A configured merchant/store in the multi-tenant system | *** ## Support For issues or questions: * GitHub Issues: [https://github.com/stateset/stateset-sync-server/issues](https://github.com/stateset/stateset-sync-server/issues) * Documentation: [https://docs.stateset.com](https://docs.stateset.com) ## Next steps Envelopes, error codes and correlation ids. Getting orders into the Sync Server in the first place. The protocol this implements. Authentication and tenancy. # Sync Server API Basics Source: https://docs.stateset.com/stateset-sync-server-api-basics Base URLs, envelopes, error codes, and idempotency. This guide summarizes the core HTTP and gRPC contract for the Sync Server. ## Base URLs & Versioning * HTTP base path: `/v1` * OpenAPI: `/api-docs/openapi.json` * Docs UI: `/docs/` or `/swagger-ui/` * Responses include `x-api-version` and `x-api-supported-versions` ## Response Envelopes Success: ```json theme={null} { "meta": { "requestId": "..." }, "data": { "statesetOrderId": "..." } } ``` Error: ```json theme={null} { "meta": { "requestId": "..." }, "error": { "code": "validation_failed", "message": "..." } } ``` ## Error Codes (Common) * `validation_failed` * `rate_limit_exceeded` * `integration_missing` * `upstream_error` * `internal_error` ## Idempotency * Order creation is idempotent on `shopify_order_id` * Write endpoints accept `idempotency-key` (cached for 24h) * Retries return `idempotency-replayed: true` ## Related Documentation * [API Contract](/stateset-sync-server-api-contract) * Sync Server README Keep the `requestId` from `meta` on every response, success or failure. It is the only handle that ties a call here to the downstream integration attempt it produced, and support cannot trace a failed sync without it. Log it before you branch on the result, not inside the error path — the calls that are hardest to explain later are the ones that returned `200`. ## Next steps The complete HTTP and gRPC contract this summarises. Getting orders into the Sync Server durably. What happens to an order after it is accepted. Protocol support built on this contract. # Sync Server API Contract Source: https://docs.stateset.com/stateset-sync-server-api-contract The complete HTTP and gRPC contract for the StateSet Sync Server. # API Contract This document captures the runtime contract for the HTTP and gRPC surfaces: versioning, envelopes, error codes, rate limits, pagination, and idempotency expectations. Every tenant endpoint has a generated reference page with a live playground under the [Sync Server API tab](/api-reference/sync/overview), built from the OpenAPI document `api.sync.stateset.com` publishes. This page is the contract those endpoints share. ## Base URLs & Versioning * HTTP base path: `/v1` (example: `http://localhost:8080/v1/tenants/{tenant_id}/orders`). * OpenAPI: `/api-docs/openapi.json` and interactive docs at `/docs/` (alias `/swagger-ui/`). * Version headers: responses include `x-api-version` (current `1.0.0`) and `x-api-supported-versions` (minimum supported version). Requests may optionally send `x-api-version`; unsupported versions return `400`. * Deprecation: `deprecation`/`sunset` headers are wired in the middleware for future sunset notices (none active now). ## Authentication * HTTP (tenant): `x-stateset-api-key: ` per request. * HTTP (operator / dashboard): `x-stateset-admin-key: ` is restricted to platform-admin `/v1/admin/*` calls and org-bound operator-safe tenant reads inside the WorkOS-authenticated dashboard proxy. * gRPC: same key via metadata header `x-stateset-api-key` on port `50051`. ### Webhook Authentication * Shopify webhook ingress (`/v1/tenants/{tenant_id}/webhooks/shopify`) uses `x-shopify-hmac-sha256`. * Amazon SNS webhook ingress (`/v1/tenants/{tenant_id}/webhooks/amazon`) uses SNS RSA signature verification. * TikTok webhook ingress (`/v1/tenants/{tenant_id}/webhooks/tiktok`) uses HMAC SHA-256 signature verification from `x-tts-signature`/`x-tiktok-signature`. * Walmart webhook ingress (`/v1/tenants/{tenant_id}/webhooks/walmart`) uses HMAC SHA-256 signature verification from `x-walmart-signature`. * eBay webhook ingress (`/v1/tenants/{tenant_id}/webhooks/ebay`) uses HMAC SHA-256 signature verification from `x-ebay-signature`. * Etsy webhook ingress (`/v1/tenants/{tenant_id}/webhooks/etsy`) uses HMAC SHA-256 signature verification from `x-etsy-signature`. * WooCommerce webhook ingress (`/v1/tenants/{tenant_id}/webhooks/woocommerce`) uses HMAC SHA-256 signature verification from `x-wc-webhook-signature`. * ACP webhook ingress (`/v1/tenants/{tenant_id}/webhooks/acp/orders`) uses HMAC SHA-256 with: * `x-acp-timestamp`: Unix seconds/milliseconds or RFC3339 timestamp * `x-acp-signature`: HMAC of `"{timestamp}.{raw_body}"` (hex or base64; `v1=`/`sha256=` prefixes supported) * Shopify secret is read from `{TENANT_ID}_SHOPIFY_WEBHOOK_SECRET` (for tenant `acme`: `ACME_SHOPIFY_WEBHOOK_SECRET`). * TikTok secret is read from `{TENANT_ID}_TIKTOK_WEBHOOK_SECRET` (for tenant `acme`: `ACME_TIKTOK_WEBHOOK_SECRET`). * Walmart secret is read from `{TENANT_ID}_WALMART_WEBHOOK_SECRET`. * eBay secret is read from `{TENANT_ID}_EBAY_WEBHOOK_SECRET`. * Etsy secret is read from `{TENANT_ID}_ETSY_WEBHOOK_SECRET`. * WooCommerce secret is read from `{TENANT_ID}_WOOCOMMERCE_WEBHOOK_SECRET`. * ACP secret is read from `{TENANT_ID}_ACP_WEBHOOK_SECRET` (for tenant `acme`: `ACME_ACP_WEBHOOK_SECRET`). * Unsigned ACP webhooks are rejected unless `STATESET_ALLOW_UNSIGNED_ACP_WEBHOOKS=true`. ## Response Envelopes & Correlation * Success shape: ```json theme={null} { "meta": { "requestId": "da02c39f-2d9b-4e3c-8aa2-6c4f3bda9d54" }, "data": { "statesetOrderId": "c2c2d8e1-9c05-4c0a-83b7-bc3a66dc7b2f" } } ``` * Error shape: ```json theme={null} { "meta": { "requestId": "da02c39f-2d9b-4e3c-8aa2-6c4f3bda9d54" }, "error": { "code": "validation_failed", "message": "shopifyOrderId is required", "details": { "cause": "timeout" } } } ``` * Every response also carries `x-request-id`; it matches `meta.requestId` and can be used for support tickets and trace correlation. A `200` with `success: false` is the normal shape for a partial failure here: the request was accepted and some integration target rejected it. Branch on the envelope, not the status code, and log the correlation id — it is the only thing that ties a failure back to the downstream call that produced it. ## Error Codes | HTTP | code | When it happens | | ---- | ------------------------------- | ------------------------------------------------------------------------ | | 400 | `integration_not_configured` | Optional integrations (Amazon/TikTok/etc.) are disabled for the tenant. | | 404 | `tenant_not_found`, `not_found` | Tenant or resource not present. | | 422 | `validation_failed` | Payload validation failed (field-level messages in `message`/`details`). | | 424 | `integration_missing` | Tenant lacks the requested integration. | | 429 | `rate_limit_exceeded` | Global/IP rate limit hit (see headers and `details.retryAfterSeconds`). | | 502 | `upstream_error` | Downstream (NetSuite/Shopify/DCL/Cart.com/etc.) returned an error. | | 500 | `internal_error` | Unhandled or unexpected error. | ## Rate Limiting * Keyed by client IP via `tower_governor`. Bucket: burst `500`, refill one token every `100ms` (\~10 requests/second sustained). * On `429` responses: * Headers: `x-ratelimit-limit`, `x-ratelimit-remaining`, `x-ratelimit-after`, `Retry-After` (seconds), `x-request-id`. * Body: standard error envelope with `code=rate_limit_exceeded` and `details.retryAfterSeconds`. * Per-tenant concurrency is additionally shaped by semaphores for each integration to keep within vendor quotas. ## Idempotency & Retries * `POST /v1/tenants/{tenant_id}/orders` is idempotent on `shopify_order_id`; if the order already exists in StateSet the existing `stateset_order_id` is returned without creating a duplicate. * `POST /v1/tenants/{tenant_id}/orders/batch` accepts up to `200` orders and returns per-item success/failure details; set `stopOnError=true` to fail-fast (otherwise it continues through all entries). * Batch imports support preview mode with `dryRun=true` (request body) or `?dry_run=true` (query param). In dry-run mode the API validates each item, reports `action` (`would_create` / `would_reuse_existing` / `failed`), and performs zero writes. * `POST /v1/tenants/{tenant_id}/orders/backfill/shopify/{shopify_order_id}` is an operator repair endpoint: it fetches the order snapshot from Shopify and persists it locally when missing. It returns `201` when it inserted a new row and `200` when the order already existed locally. * All write endpoints (`POST`/`PUT`/`PATCH`/`DELETE`) also accept an `idempotency-key` header. Successful responses are cached for 24h and replayed on retry with `idempotency-replayed: true`; if a matching request is still running, the server returns `409 operation_in_progress`. When PostgreSQL is enabled, idempotency state is shared across replicas; otherwise it falls back to process-local memory. * Sync triggers (`/sync/*`) enqueue background jobs; use `/v1/tenants/{tenant_id}/jobs/{job_id}` to poll instead of holding the original request open. Triggering twice will create two jobs, so prefer client-side de-duplication when retrying. * `GET /v1/tenants/{tenant_id}/jobs/throughput` returns time-bucketed job analytics (created/finished/succeeded/failed) for dashboard charts. Query params: `hours` (default `24`) and `bucket_minutes` (default `60`). * Downstream calls use exponential backoff (see `src/resilience.rs`) and classify retryable upstream errors (timeouts, 5xx, 429) automatically. * Webhook operator tooling supports both single replay (`POST /v1/tenants/{tenant_id}/webhook-events/{webhook_id}/replay`) and bulk replay (`POST /v1/tenants/{tenant_id}/webhook-events/replay?state=failed&limit=50`). * Bulk replay supports additional safety controls: * exact topic filtering via `topic` * preview mode via `dry_run=true` (returns selected ids without queueing) * source-level throttling via `max_per_source=` * Targeted replay is available via `POST /v1/tenants/{tenant_id}/webhook-events/replay-by-id` with JSON body (`webhookIds`, optional `dryRun`). ## Auditability * Bulk and operator repair actions emit `data_access` audit events (resource types: `orders_batch_import`, `orders_batch_import_preview`, `orders_backfill_shopify`, `webhook_events_bulk_replay`, `webhook_event_replay`). * Audit entries are best-effort persisted to the database (when configured) and always logged to structured application logs. * `x-stateset-user-email` and `user-agent` headers are captured when present for operator attribution. * Admin audit queries support `resource_type` filtering: `GET /v1/admin/audit-logs?event_type=data_access&resource_type=orders_batch_import`. ## Admin Onboarding Readiness * `GET /v1/admin/tenants/{tenant_id}/readiness` returns a scored onboarding report for customer go-live checks. * `GET /v1/admin/tenants/{tenant_id}/onboarding-plan` returns prioritized action items derived from the readiness report. * `PATCH /v1/admin/tenants/{tenant_id}/onboarding-plan/actions/{action_id}` persists operator workflow state for each action (`pending`, `in_progress`, `done`, `blocked`) with optional notes. * `PATCH /v1/admin/tenants/{tenant_id}/onboarding-plan/actions` supports bulk workflow updates for multiple actions in a single request. * `GET /v1/admin/tenants/{tenant_id}/onboarding-plan/actions/history` returns append-only action history with actor/timestamp context. * `PATCH /v1/admin/tenants/{tenant_id}/onboarding-profile` upserts tenant onboarding metadata (`ownerEmail`, `ownerName`, `goLiveTargetAt`, `notes`). * `GET /v1/admin/onboarding/portfolio` returns a cross-tenant onboarding pipeline summary (scores, risk, blockers, workflow counts, next action, owner/go-live metadata). Supports optional filters via `readinessStatus`, `integration`, `hasBlockers`, `minRiskScore`, `minBlockedActionAgeHours`, `ownerAssigned`, and `overdueGoLive`; sorting via `sortBy` (`risk|score|tenant_id`) + `sortOrder` (`asc|desc`); includes summary counters/thresholds for `highRiskTenants`, `staleBlockerTenants`, `unassignedOwnerTenants`, and `overdueGoLiveTenants`; use `includeIntegrationTests=true` to run live health checks in the aggregation response. * The response includes: * `score` (0-100) and `grade` (`A+`..`F`) * `overallStatus` (`ready`, `attention_required`, `not_ready`) * required check results (tenant runtime, integration config, channel/destination coverage, auth readiness, Shopify webhook secret, connectivity) * per-integration connectivity test results * guide readiness for Shopify -> NetSuite and Shopify -> DCL with missing requirements ## Pagination & Filtering * `GET /v1/tenants/{tenant_id}/orders` supports `page` (default `1`), `page_size` (default `50`, max `100`), `status`, `created_after`, `created_before`, `order_number`, `shopify_order_id`, `tags`. * Response uses `PaginatedResponse`: `items` plus `pagination` metadata (`page`, `pageSize`, `totalItems`, `totalPages`, `hasNext`, `hasPrevious`). Example: ```bash theme={null} curl -H "x-stateset-api-key: " \ "http://localhost:8080/v1/tenants/acme/orders?page=1&page_size=50&status=fulfilled" ``` ## Examples * Create order (idempotent on Shopify order id): ```bash theme={null} curl -X POST "http://localhost:8080/v1/tenants/acme/orders" \ -H "content-type: application/json" \ -H "x-stateset-api-key: " \ -d '{ "shopifyOrderId": "1234567890", "orderNumber": "M-1001", "createdAt": "2024-01-01T00:00:00Z", "tags": ["vip"], "lineItems": [{"title": "Widget", "quantity": 1, "price": 42.0}] }' ``` * Create orders in batch (up to 200 per request): ```bash theme={null} curl -X POST "http://localhost:8080/v1/tenants/acme/orders/batch" \ -H "content-type: application/json" \ -H "x-stateset-api-key: " \ -d '{ "stopOnError": false, "orders": [ { "shopifyOrderId": "1234567890", "orderNumber": "M-1001", "createdAt": "2024-01-01T00:00:00Z", "tags": ["vip"], "lineItems": [{"title": "Widget", "quantity": 1, "price": 42.0}] }, { "shopifyOrderId": "1234567891", "orderNumber": "M-1002", "createdAt": "2024-01-01T00:00:00Z", "lineItems": [{"title": "Widget", "quantity": 2, "price": 42.0}] } ] }' ``` * Error response (validation): `422` with body shown above and `x-request-id` for correlation. ## gRPC quick start ```bash theme={null} grpcurl -plaintext \ -H "x-stateset-api-key: " \ -d '{"tenant_id": "acme", "stateset_order": {"shopify_order_id": "123", "order_number": "M-1001", "line_items": [{"title": "Widget", "quantity": 1, "price": 42.0}]}}' \ localhost:50051 stateset.sync.v1.OrderIntegrationService/SubmitOrder ``` ## Next steps Keys, tenancy and getting a first call through. The gRPC half of this contract in use. What happens after a request is accepted. Protocol support built on this contract. # Sync Server connectors Source: https://docs.stateset.com/stateset-sync-server-connectors A connector is a JSON definition, not code — and one field in it decides whether an ambiguous failure becomes a duplicate shipment. Adding a warehouse or 3PL to the Sync Server is a **catalog entry**, not a code change. A connector is a JSON definition — auth, endpoints, pagination, cursors — plus one or more mapping rule sets, run by a generic runtime through the same resilience, ingestion, checkpoint and crash-safe submission machinery every hand-written integration uses. Hand-written integrations are reserved for the strategic platforms where depth justifies compiled code — Shopify, NetSuite, Amazon, DCL. Everything else is a definition, and existing per-partner code migrates by attrition: when one needs a fix, it is ported to a catalog entry rather than patched. ## What a definition carries | Section | Describes | | ----------------- | ---------------------------------------------------------------- | | `auth` | How to authenticate — the scheme and where credentials come from | | `endpoints` | The partner's URLs, per operation | | `pagination` | How to walk a result set | | `cursors` | Where to resume after a restart | | `dedupPolicy` | What happens when a submission fails ambiguously | | Mapping rule sets | How partner fields map to commerce records, per flow | A definition in outline: ```json theme={null} { "name": "acme-wms", "auth": { "type": "bearer", "credential": "ACME_WMS_TOKEN" }, "endpoints": { "order_push": { "method": "POST", "path": "/v2/orders" }, "shipment_pull": { "method": "GET", "path": "/v2/shipments" } }, "pagination": { "style": "cursor", "param": "after", "limit": 100 }, "cursors": { "shipment_pull": "updated_at" }, "dedupPolicy": "no_retry_after_ambiguous", "idempotencyHeader": "Idempotency-Key" } ``` ## dedupPolicy — read this before shipping a push flow An **ambiguous failure** is a timeout or dropped connection where the remote may already have accepted the order. You cannot tell from your side whether it did. One field decides what happens next, and it drives all three layers — transport retries, the submission guard, and the stale-in-flight watchdog — so they can never disagree with each other. | Value | On an ambiguous failure | Transport retries | | ---------------------------------------- | ------------------------------------------- | ----------------------------------------------------- | | `no_retry_after_ambiguous` **(default)** | The order parks in-flight for manual review | Never | | `safe_to_retry` | Released for normal backoff retry | Permitted, and only with `idempotencyHeader` declared | **The wrong answer here is how duplicate shipments happen.** Choose `safe_to_retry` only when you have *proof* the remote deduplicates — a documented idempotency key it honours, not an assumption that a second identical order would be rejected. A dedup claim without `idempotencyHeader` still gets exactly one send, because the platform will not take your word for it. The default parks the order rather than retrying, which means a real outage produces a queue of orders awaiting review instead of a queue of possible duplicates. That is the intended trade: a human looking at ten parked orders is cheaper than a customer receiving two shipments. ## Shipping one ```bash theme={null} # validate a definition against its golden fixtures before it goes in the catalog stateset-sync connectors validate ./connectors/acme-wms.json # dry-run a flow against the fixtures, with no partner traffic stateset-sync connectors test ./connectors/acme-wms.json --flow order_push ``` A catalog entry ships with golden fixtures, so a mapping change that silently drops a field fails review rather than production. Several long-tail ports have found long-standing silent mapping failures in the code they replaced — fixtures are the reason. ## Next steps Envelopes, error codes and correlation ids. Circuit breakers, stuck orders and rate limits. Getting orders into the Sync Server durably. What happens to an order once it is accepted. # Sync Server gRPC Dispatch Source: https://docs.stateset.com/stateset-sync-server-grpc-dispatch How StateSet API dispatches orders to the Sync Server over gRPC. # StateSet API → StateSet Sync Server (gRPC) Order Dispatch Flow This document describes a practical architecture for sending orders from the StateSet Rust API (`stateset-api`) to the StateSet Sync Server (`stateset-sync-server`) using the Sync Server’s existing gRPC API. ## Goal When an order is created (or updated) in `stateset-api`, reliably dispatch it to `stateset-sync-server` so the Sync Server can orchestrate downstream integrations (NetSuite, DCL, Cart.com, etc.) via its gRPC surface. ## Key Interfaces (as implemented today) * **Sync Server proto**: `proto/order_integration.proto` (`stateset.sync.v1.OrderIntegrationService`) * Primary RPC: `SubmitOrder(SubmitOrderRequest) returns (SubmitOrderResponse)` * **Sync Server gRPC server**: `src/grpc/mod.rs` (implements `submit_order`) * **Sync Server listen port**: `src/server.rs` (`server.grpc_port`, default `50051`) * **Sync Server auth**: `x-stateset-api-key` metadata header (`src/http/auth.rs`) * **StateSet API durable trigger**: outbox enqueue on order create (`stateset-api/src/services/orders.rs`) * Outbox worker: `stateset-api/src/events/outbox.rs` * Event loop hook point: `stateset-api/src/events/mod.rs` (`handle_order_created`) ## High-Level Flow 1. A client creates an order in `stateset-api` (HTTP or gRPC). 2. `stateset-api` commits the order + items to its DB and enqueues an `OrderCreated` outbox row in the same transaction. 3. An outbox worker polls for pending events and triggers a **Sync dispatch** step. 4. The dispatcher loads the order + items, builds a `SubmitOrderRequest`, and calls `stateset.sync.v1.OrderIntegrationService/SubmitOrder` on the Sync Server. 5. The Sync Server validates tenant + API key, then invokes its per-tenant orchestrators to call downstream integrations. 6. On success, the outbox row is marked delivered; on retryable failure, it is rescheduled with backoff. ## Sequence Diagram ```mermaid theme={null} sequenceDiagram autonumber participant Client participant API as stateset-api participant DB as Orders DB participant Outbox as outbox_events participant Worker as Outbox worker participant Dispatch as Sync gRPC dispatcher participant Sync as stateset-sync-server (gRPC) participant Orch as Tenant orchestrators participant Vendors as NetSuite/DCL/Cart.com participant Stateset as Stateset integration Client->>API: Create order (HTTP/gRPC) API->>DB: INSERT orders + order_items (txn) API->>Outbox: INSERT OrderCreated event (txn) DB-->>API: COMMIT Worker->>Outbox: Claim pending events (SKIP LOCKED) Worker->>Dispatch: Process OrderCreated(order_id) Dispatch->>DB: Load order + items Dispatch->>Sync: SubmitOrder(tenant_id, stateset_order?, netsuite?/dcl?/cart?) + x-stateset-api-key Sync->>Sync: Validate tenant + API key Sync->>Orch: create_stateset_order (optional) Sync->>Orch: create_netsuite_sales_order (optional) Sync->>Orch: create_dcl_batch (optional) Sync->>Orch: create_cart_order (optional) Orch->>Stateset: Create/update Stateset order (optional) Orch->>Vendors: Push downstream payloads (optional) Sync-->>Dispatch: SubmitOrderResponse(success, errors) Dispatch-->>Outbox: Mark delivered / schedule retry ``` ## Request Construction (what `stateset-api` sends) ### Minimal viable request (using existing proto) `SubmitOrderRequest` requires `tenant_id` and accepts optional payloads: * `stateset_order` (`CreateOrderPayload`) for “StateSet order create” inside the Sync Server * `netsuite`, `dcl`, `cart` (all `google.protobuf.Struct`) for direct downstream submission Sync Server proto: `proto/order_integration.proto`. ### Mapping from `stateset-api` order model to `CreateOrderPayload` `stateset-api` canonical entities: * Order: `stateset-api/src/entities/order.rs` * Order items: `stateset-api/src/entities/order_item.rs` Sync Server expects (subset shown): * `shopify_order_id` (string) * `order_number` (string) * `created_at` (`google.protobuf.Timestamp`) * `line_items[]` with `title`, `quantity`, `price`, `sku`, etc. Pragmatic mapping: * `CreateOrderPayload.order_number` ← `orders.order_number` * `CreateOrderPayload.created_at` ← `orders.created_at` (or `order_date`, choose one consistently) * `OrderLineItem.title` ← `order_items.name` * `OrderLineItem.quantity` ← `order_items.quantity` (convert `i32` → `f64`) * `OrderLineItem.price` ← `order_items.unit_price` (convert `Decimal` → `f64`, mind rounding) * `OrderLineItem.sku` ← `order_items.sku` * `CreateOrderPayload.shopify_order_id`: * If you have a true external/channel order id, use it. * If not, use the `stateset-api` order UUID string as a stable idempotency key. Important: the Sync Server’s existing idempotency logic for `stateset_order` keys off `shopify_order_id` (`src/orchestrators/order_sync.rs`), so this field must be stable for retries. ### Calling it by hand Before wiring the dispatcher, confirm the Sync Server accepts a request from where `stateset-api` runs. `grpcurl` needs the proto because the server does not serve reflection by default: ```bash theme={null} grpcurl -plaintext \ -proto proto/order_integration.proto \ -H 'x-stateset-api-key: ss_sync_...' \ -d '{ "tenant_id": "tnt_4471", "order": { "shopify_order_id": "0b7f1c62-1f3a-4a5e-9a0c-2f9a1d7b0e11", "order_number": "SO-10428", "created_at": "2026-08-29T12:00:00Z", "line_items": [ { "sku": "TS-CREW-NAVY-M", "title": "Crew Neck — Navy, M", "quantity": 1, "price": 68.00 } ] } }' \ sync.internal:50051 stateset.sync.v1.OrderIntegrationService/SubmitOrder ``` A healthy response carries per-target results rather than a bare acknowledgement: ```json theme={null} { "success": true, "results": [ { "target": "netsuite", "success": true, "external_id": "SO-99120" }, { "target": "dcl", "success": true, "external_id": "DCL-77341" } ], "errors": [] } ``` ## Auth & Tenancy * Set gRPC metadata header `x-stateset-api-key` (same key used for the Sync Server’s HTTP API). * Provide `tenant_id` in every request; Sync Server resolves tenant configuration and validates the key before executing orchestrators (`src/grpc/mod.rs`). ## Delivery Semantics (reliability) ### Recommended: outbox-driven gRPC dispatch To get “at-least-once” delivery to the Sync Server, perform the gRPC call as part of the outbox processing step and only mark the outbox row `delivered` after the RPC is successful (or after a “non-retryable” error policy decision). This matches the existing outbox pattern in `stateset-api` (`stateset-api/src/events/outbox.rs`) but changes the “delivery target” from “in-process channel send” to “cross-service gRPC”. ### Retry policy * Retry on transport/transient errors (`UNAVAILABLE`, `DEADLINE_EXCEEDED`, timeouts). * Do not retry on permanent validation errors without operator action (`INVALID_ARGUMENT`, `UNAUTHENTICATED`, `PERMISSION_DENIED`). * Treat application-level partial failures carefully: * `SubmitOrder` can return `OK` but `success=false` with `errors[]` per integration target (`src/grpc/mod.rs`). * Decide which targets are “required” vs “best-effort” and only consider the outbox event delivered when your required targets succeeded. A gRPC status of `OK` does not mean the order was integrated. `SubmitOrder` returns `OK` whenever the request was well-formed and the tenant authenticated — including when every downstream target failed. A dispatcher that checks only the gRPC status will mark the outbox row delivered and drop the order silently. Check `SubmitOrderResponse.success`, and inspect `errors[]` per target, before acknowledging. ## Observability Suggested minimum telemetry on the `stateset-api` side for each dispatch attempt: * `order_id`, `tenant_id`, target `grpc_addr` * attempt count + backoff delay * gRPC status code on failure * `SubmitOrderResponse.success` and `errors[].target/message` on application-level failures Sync Server already logs a “gRPC server listening” line and uses tracing around orchestrator operations; correlate logs via request ids where possible. ## Implementation Notes (code placement) In `stateset-api`, the simplest integration point is the existing event processing hook: * Add a Sync dispatcher call in `stateset-api/src/events/mod.rs` inside `handle_order_created`. * For true durability, move the gRPC call into the outbox worker path (`stateset-api/src/events/outbox.rs`) so outbox “delivered” reflects remote delivery rather than local enqueue. ## Open Questions / Future Improvements * The current Sync Server gRPC proto is integration-oriented and uses `google.protobuf.Struct` for downstream payloads. If you want Sync Server to derive NetSuite/DCL/Cart payloads from the canonical `stateset-api` order model, add a new RPC/message that accepts a first-class “StateSet order” envelope rather than expecting pre-mapped JSON structs. * TLS: the Sync Server gRPC listener is plaintext by default; deploy behind a TLS-terminating proxy or add tonic TLS configuration if needed. ## Next steps Authentication, tenancy and the HTTP surface this shares a key with. What the Sync Server does with the order once SubmitOrder returns. The request and response shapes the mapping above targets. The alternative to an outbox worker when the dispatch needs to be a durable workflow in its own right. # Sync Server gRPC Dispatch Flow Source: https://docs.stateset.com/stateset-sync-server-grpc-flow How an order moves from stateset-api to the Sync Server over gRPC — the service contract, the fan-out, partial failure, and what is safe to retry. The Sync Server is the fan-out point. `stateset-api` hands it one order; it submits that order to every downstream system the tenant has configured — StateSet, NetSuite, DCL, Cart.com — and reports per-target results. The important consequence, which shapes everything below: **one call can partially succeed.** ## The flow ``` order created in stateset-api │ ├─▶ outbox row written in the same transaction as the order ▼ outbox worker claims the row │ ├─▶ gRPC SubmitOrder ──▶ Sync Server │ ├─ authenticate tenant │ ├─ validate payload │ └─ fan out ──┬─▶ Stateset │ ├─▶ NetSuite │ ├─▶ DCL │ └─▶ Cart.com ▼ mark delivered, or schedule a retry ``` The outbox row is written **in the same transaction** as the order. That's what makes dispatch at-least-once rather than best-effort: if the process dies after committing the order, the row is still there to be claimed. ## Service contract `stateset.sync.v1.OrderIntegrationService`, default port `50051`: | RPC | Purpose | | -------------------------- | -------------------------------------------------------------- | | `SubmitOrder` | The fan-out entry point — one order to every configured target | | `CreateNetSuiteSalesOrder` | NetSuite only | | `CreateDclBatch` | DCL only, batched | | `CreateCartOrder` | Cart.com only | ### `SubmitOrderRequest` | Field | Type | Notes | | ---------------- | ---------------------- | -------------------------------------------------- | | `tenant_id` | `string` | Required. Resolves the tenant's integration config | | `stateset_order` | `CreateOrderPayload` | The canonical order | | `netsuite` | `NetSuiteOrderPayload` | Optional per-target payload | | `dcl` | `DclOrderPayload` | Optional | | `cart` | `CartComOrderPayload` | Optional | `CreateOrderPayload` carries `shopify_order_id`, `order_number`, `line_items[]`, `created_at`, `tags[]`, `location_id`, `workflow_id`, and `netsuite_id`. Each line item carries `sku`, `title`, `quantity`, `price`, `variant_id`, and `shopify_line_item_id`. ### `SubmitOrderResponse` | Field | Type | Notes | | --------- | -------------------------- | ----------------------------------------------------- | | `success` | `bool` | Whether the submission succeeded overall | | `result` | `OrderSubmissionResult` | Per-target results | | `errors` | `repeated ProcessingError` | Per-target failures, each with `target` and `message` | **`success` and a non-empty `errors` list are not mutually exclusive.** `OrderSubmissionResult` carries a separate result per target — `stateset_order_id`, `netsuite`, `dcl`, `cart` — precisely because NetSuite can accept an order in the same call where DCL rejects it. Always inspect `errors[]` and the per-target results. Branching on `success` alone will silently lose a failed downstream submission. ## Authentication Credentials go in gRPC **metadata**, not the payload. Two schemes, tried in order: | Metadata key | Scheme | | ------------------------------------ | ---------------- | | `authorization` | `Bearer ` | | `x-stateset-api-key`, or `x-api-key` | API key | An unparseable bearer token fails `UNAUTHENTICATED`. Keys are matched against the tenant's active key set, and a successful match updates that key's last-used timestamp asynchronously. ```bash theme={null} grpcurl -plaintext \ -proto proto/order_integration.proto \ -H 'x-stateset-api-key: ss_sync_…' \ -d '{ "tenant_id": "tnt_4471", "order": { "shopify_order_id": "5512034" } }' \ sync.internal:50051 stateset.sync.v1.OrderIntegrationService/SubmitOrder ``` ## Status codes and what they mean The server maps its internal errors onto gRPC statuses deliberately. This mapping *is* the retry policy: | Status | Cause | Retry? | | --------------------- | -------------------------------------------------------------------------------------- | ------------------------------------------------------- | | `INVALID_ARGUMENT` | Validation failure — missing `tenant_id`, absent `created_at`, unparseable DCL payload | **No.** Deterministic; a retry produces the same error | | `NOT_FOUND` | Unknown tenant or record | **No** | | `FAILED_PRECONDITION` | The integration is not configured for this tenant | **No.** Needs a config change, not another attempt | | `UNAVAILABLE` | Upstream integration error | **Yes**, with backoff — the upstream is down, not wrong | | `DEADLINE_EXCEEDED` | Timeout | **Yes**, with care — see below | | `INTERNAL` | Unexpected server-side failure | Retry once, then escalate | `FAILED_PRECONDITION` is the one worth special handling. It means the tenant has no configuration for that integration — retrying forever will never fix it, and it usually indicates onboarding was left incomplete rather than a transient fault. Alert on it instead of burying it in a retry queue. ## Retrying safely At-least-once delivery means **duplicates are expected**, and the fan-out makes them asymmetric. A `DEADLINE_EXCEEDED` does not tell you whether downstream systems accepted the order. The call may have timed out *after* NetSuite created the sales order. A naive retry of the whole `SubmitOrder` re-submits to every target, including the ones that already succeeded. Two things make this tractable: 1. **Retry per target, not per call.** When you have partial results, use the single-target RPCs — `CreateNetSuiteSalesOrder`, `CreateDclBatch`, `CreateCartOrder` — to complete only what failed. 2. **Key downstream deduplication on `shopify_order_id`.** It is stable across retries in a way that an internal ID generated per attempt is not. ```bash theme={null} # After a DEADLINE_EXCEEDED where NetSuite succeeded and DCL did not: # complete only what failed, rather than re-submitting to everything. grpcurl -plaintext \ -proto proto/order_integration.proto \ -H 'x-stateset-api-key: ss_sync_…' \ -d '{ "tenant_id": "tnt_4471", "shopify_order_id": "5512034" }' \ sync.internal:50051 stateset.sync.v1.OrderIntegrationService/CreateDclBatch ``` Only mark the outbox row delivered when every configured target has a result. A row marked delivered on a partially-successful response is an order that is permanently half-integrated with nothing left to reconcile it. ## Related * [StateSet API → Sync Server dispatch](/stateset-sync-server-grpc-dispatch) * [API contract](/stateset-sync-server-api-contract) * [API basics](/stateset-sync-server-api-basics) # Sync Server Skill Source: https://docs.stateset.com/stateset-sync-server-skill Skill file for agents operating the StateSet Sync Server — tenant-scoped order, inventory and product sync across 180+ integrations, over REST, CLI or MCP. Operate and use the StateSet Sync Server as a commerce integration platform. ## How It Works 1. Confirm configuration source: `STATESET_APP_CONFIG_JSON`, `APP_CONFIG_PATH`, or `config/app_config.json`. 2. Start the server or verify an existing instance. 3. Check health and API docs before invoking workflows. 4. Use tenant-scoped REST, CLI, or MCP calls with tenant auth. 5. For writes, use idempotency keys and explicit confirmation where the surface supports it. 6. After syncs, inspect jobs, webhook events, audit logs, and reconciliation/commerce ops views. ## Status Flows * **Server:** configured -> starting -> healthy | degraded | failed * **Sync Job:** queued -> running -> succeeded | retrying | failed | dead\_lettered * **Webhook:** received -> validated -> persisted -> processed | retrying | failed * **MCP Tool:** discovered -> validated -> confirmed -> executed -> audited ## Usage * Local server: `cargo run` or `stateset-sync-server` * Healthcheck: `stateset-sync-server --healthcheck` * API docs: `GET /docs/` or `GET /api-docs/openapi.json` on your server * REST base path: `/v1/tenants/{tenant_id}/...` * MCP smoke test: `stateset-sync-server mcp-test --url --tenant --api-key ` * MCP stdio: `stateset-sync-server mcp-stdio --tenant ` * CLI: `stateset-sync --help` ## Permissions * **Read:** health, docs, list/get orders, list jobs, inventory reads, plan tiers, MCP `tools/list`, MCP resources. * **Write:** create/update orders, trigger syncs, replay webhooks, inventory reservations, custom MCP tools. * **Admin:** platform diagnostics, tenant policy, audit logs, operator-safe dashboard routes. Writes should include an `Idempotency-Key` header when available and should not be retried blindly after a timeout without checking job or idempotency state. ## Examples ```bash theme={null} export APP_CONFIG_PATH=./config/app_config.json cargo run curl -fsS http://127.0.0.1:8080/healthz curl -fsS -H "x-stateset-api-key: $STATESET_API_KEY" \ http://127.0.0.1:8080/v1/tenants/acme/jobs stateset-sync-server mcp-test --url http://127.0.0.1:8080 --tenant acme --api-key "$STATESET_API_KEY" ``` ## Output ```json theme={null} {"health":"ok","tenant":"acme","jobs":{"queued":0,"running":0,"failed":0},"mcp":{"tools":42}} ``` ## Present Results to User * Server URL, tenant ID, and config source used. * Health/readiness result and any degraded dependency. * Auth method used: tenant API key, JWT, CLI token, or admin key. * For syncs: job ID, state, retry count, failures, and next inspection endpoint. * For MCP: tool name, params, confirmation/dry-run status, and response `_meta`. * Exact command or endpoint for the next step. ## Troubleshooting * Config missing: set `APP_CONFIG_PATH` or `STATESET_APP_CONFIG_JSON`. * Auth rejected: verify tenant ID, `x-stateset-api-key`, JWT claims, or CLI token. * Metrics protected: set `STATESET_METRICS_AUTH_TOKEN` and pass `Authorization: Bearer`. * Sync stuck: inspect `/v1/tenants/{tenant_id}/jobs`, DLQ, and webhook events. * Shopify inventory mismatch: verify the fulfillment integration's `shopify_inventory_location_id` and the Shopify locations it maps to. * MCP mismatch: run `mcp-test`, then compare tool schemas from `tools/list`. ## Error Codes * `tenant_not_found`: Tenant ID is not configured. * `unauthorized`: Missing or invalid tenant authentication. * `missing_integration`: Tenant lacks the requested connector. * `rate_limit_exceeded`: Retry after the provided `Retry-After` value. * `idempotency_conflict`: Same key was reused with a different request fingerprint. ## Further reading * [Sync Server MCP](/stateset-sync-mcp) — the 186-tool registry and four transports * [Sync Server connectors](/stateset-sync-server-connectors) * [Sync Server troubleshooting](/stateset-sync-server-troubleshooting) # Sync Server troubleshooting Source: https://docs.stateset.com/stateset-sync-server-troubleshooting Why an order is stuck, why a circuit breaker is open, and which failures you should not retry. Most Sync Server incidents are one of four shapes. Work them in this order — each rules out the layer below it. ## 1. Is the order actually stuck? An order sitting in-flight is not necessarily failing. If its connector uses the default `no_retry_after_ambiguous` ([dedupPolicy](/stateset-sync-server-connectors#deduppolicy--read-this-before-shipping-a-push-flow)), an ambiguous submission **parks** for review rather than retrying — by design. ```bash theme={null} # what is in flight, and for how long curl "$SYNC_API/v1/orders?status=in_flight" \ --header "Authorization: Bearer $SYNC_API_KEY" # everything that happened to one order curl "$SYNC_API/v1/lifecycle/$ORDER_REF" \ --header "Authorization: Bearer $SYNC_API_KEY" ``` Do not resubmit a parked order until you have checked the partner for it. The policy parked it precisely because the platform could not tell whether the remote accepted it — resubmitting on the assumption that it did not is how a parked order becomes a duplicate shipment. ## 2. Is a circuit breaker open? The breaker opens after **10 consecutive failures** to an upstream and stays open for 30 seconds, then admits a test request. A request failing with "circuit breaker is open" is the platform protecting the upstream, not a bug in your call. ```bash theme={null} curl -s http://localhost:8080/metrics | grep circuit_breaker ``` An open breaker means the upstream failed ten times in a row — so the fix is almost never at your end. Check the partner's status before changing anything on yours. Breakers reset on restart, but restarting to clear one hides the outage that opened it. ## 3. Is it the upstream's limits? | Symptom | Cause | What to do | | ----------------------------------------- | --------------------------------- | ----------------------------------------------------------- | | Bursts succeed, sustained load 429s | Shopify's leaky-bucket rate limit | Reduce concurrency; the platform already backs off | | Intermittent failures under parallel load | NetSuite concurrency limits | Lower parallelism for that tenant | | Slow then timing out | 3PL connection timeouts | Raise the timeout for that connector before raising retries | ## 4. Is it local — the database or the process? | Symptom | Usually | | --------------------------- | -------------------------------------------------------------------------- | | "connection pool exhausted" | Long-running queries holding connections, not pool size | | Memory growing steadily | An unbounded ingest without checkpointing | | Server will not start | A migration that failed halfway, or a `GLIBC` mismatch in the built binary | ## Errors you should not retry The [status-code mapping](/stateset-sync-server-grpc-flow#status-codes-and-what-they-mean) is the retry policy. Two are worth repeating here: * **`FAILED_PRECONDITION`** — the tenant has no configuration for that integration. Retrying forever will never fix it, and it almost always means onboarding was left incomplete. Alert on it rather than burying it in a queue. * **`DEADLINE_EXCEEDED`** — does not tell you whether downstream systems accepted the order. Retry per target with the single-target RPCs, never by resubmitting the whole call. ## Next steps dedupPolicy, and why a parked order is the safe outcome. The full status-code mapping and per-target retry. Envelopes and correlation ids — keep the requestId. Authentication and tenancy. # Sync Server: What's New Source: https://docs.stateset.com/stateset-sync-server-whats-new 3PL integration expansion, agent tool planning, reconciliation and catalog APIs, and organization-tenant mapping. Changes to the StateSet Sync Server since January 2026. ## 3PL and WMS integration expansion The integration catalog grew from **162 to 182 integration ids**, with 20 long-tail 3PL integrations added: `accion`, `aplin`, `coghlan`, `evobox`, `expeditors`, `fillogic`, `fulfillment247`, `haul_and_store`, `interfulfillment`, `invenco`, `its_logistics`, `metro_supply_chain`, `omnipack`, `outerspace`, `pro_fs`, `quiver`, `red_stag`, `valencia`, `verde`, and `visible`. Each is wired through config, concurrency limits, job kinds, routes, and webhook handlers. Counting platform-backed and long-standing modules, the sync server carries roughly **155 3PL/WMS integrations** in total. ### Verification status Integrations are not all equal, and the difference matters when you're planning an onboarding. A verification matrix (researched 2026-07-20 against vendor sites, developer portals, and middleware integration guides) classifies each: | Status | Meaning | | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **Verified** | The client implements the vendor's documented API — auth and endpoints. | | **Platform-backed** | The vendor runs a WMS we already integrate. Use that service with vendor-issued credentials for full-fidelity flows. | | **Partner-only** | No public API docs. The spec and credentials are provided during vendor onboarding; the generic REST client is a scaffold whose `api_url` must be configured per tenant. | Verified against public docs: | Vendor | Module | Auth | Notes | | -------------------- | ------------ | -------------------------- | ---------------------------------------------------------------------------------------------------- | | Quiver | `quiver` | `x-api-key` header | Deliveries, products, returns, inventory transfers; webhooks | | Visible SCM (Maersk) | `visible` | username/password → token | Orders, inventory, shipments, RMA; `CurrentPage`/`PageSize` paging; webhooks | | Red Stag Fulfillment | `red_stag` | HTTP Basic plus store code | ShipStream WMS protocol; webhooks for inventory and tracking | | Expeditors | `expeditors` | OAuth2 client-credentials | **Tracking/visibility only** — no order submission. 2000 req/hr; the sync job is a shipment snapshot | | Amazon MCF | `amazon` | LWA token | SP-API Fulfillment Outbound v2020-07-01; `nextToken` paging | Platform-backed — prefer the underlying service: | Vendor | Module | Underlying platform | Prefer | | --------------------- | ------------------ | --------------------------------------- | --------------------- | | Haul & Store (UK) | `haul_and_store` | Mintsoft | `mintsoft` | | InterFulfillment (CA) | `interfulfillment` | Extensiv / 3PL Central | `extensiv` | | Verde Fulfillment USA | `verde` | VeraCore | `veracore` | | Outerspace | `outerspace` | Deposco Bright Suite | `deposco` | | Evobox | `evobox` | Likely Extensiv (inferred, unconfirmed) | Confirm at onboarding | Partner-only vendors — `fulfillment247`, `accion`, `aplin`, `bdi`, `coghlan`, `fillogic`, `invenco`, `its_logistics`, `metro_supply_chain`, `omnipack`, `pro_fs`, `valencia` — have no public API documentation. Some do not offer a REST API at all: Coghlan is **CSV-over-FTP only**, and ITS Logistics uses private push APIs plus EDI X12 over AS2/sFTP/VAN. Plan onboarding with the vendor's integration team rather than assuming a REST path exists. The `bdi` module's endpoints are explicitly **unverified** against Barrett Distribution. ## Agent tool planning ```http theme={null} POST /v1/tenants/{tenant_id}/agent/plan ``` Turns a free-text operator request into a **reviewable single-tool execution plan** against the existing tool registry. The plan is returned for review rather than executed, so an operator approves before anything runs. This complements the existing `/agent/execute` endpoint: `plan` decides *what* to call, `execute` runs it. ## Reconciliation and catalog APIs | Addition | Purpose | | -------------------------------------- | --------------------------------------------------------- | | Inventory reconciliation listing API | List reconciliation discrepancies programmatically | | Order intelligence reconciliation view | Order-level reconciliation | | Product catalog listing filters | Filter by search term, sync status, source, and readiness | ## Order submission reliability * **Failed submission attempts are counted per backend**, so a backend that is failing consistently is visible rather than hidden behind aggregate retry noise. * **Exponential backoff with a max-attempt cutoff** on order submission retries, replacing unbounded retrying. * **The full upstream error chain is logged** on submission failure — the root cause is no longer collapsed into a generic message. ## Multi-tenancy and channels * **Organization-tenant mapping** — an organization can now map to its tenants. * **TikTok retailer code support.** * **TryNow partial-payment order handling.** ## Dashboard Tenant branding, an agent console, a product catalog view, and a general UI refresh. ## Related * [Sync Server API Basics](/stateset-sync-server-api-basics) * [Sync Server gRPC Flow](/stateset-sync-server-grpc-flow) # Universal Commerce Protocol Source: https://docs.stateset.com/stateset-ucp Platform-neutral discovery and checkout — one contract so commerce apps interoperate without custom integrations. UCP standardises **discovery and checkout** so commerce apps interoperate across platforms instead of each pair needing a bespoke integration. A client fetches one well-known document, learns what the merchant supports, and drives a checkout session through a lifecycle that's identical everywhere. ## How it works ``` Agent / client │ GET /.well-known/ucp discover capabilities ▼ UCP Handler ──▶ create session ──▶ update ──▶ complete │ │ │ └──▶ order webhook └──▶ optional: OAuth identity linking, AP2 mandate, tokenization ``` 1. **Discover** — `GET /.well-known/ucp` advertises which capabilities the merchant supports: checkout, fulfillment, discounts, and orders. 2. **Transact** — a standard checkout session lifecycle: create, get, update, complete, cancel. 3. **Confirm** — completion emits an order webhook. ## Run the handler The reference implementation is a standalone Rust server. ```bash theme={null} cargo run # listens on http://0.0.0.0:8081 ./demo_test.sh # exercise the full flow ``` By default the handler **requires** a `UCP-Agent` header on every request and a `Request-Signature` on `POST`/`PUT`. For local testing you can relax both with `UCP_REQUIRE_UCP_AGENT=false` and `UCP_REQUIRE_REQUEST_SIGNATURE=false` — never in production, since the signature is what authenticates the caller. ## Endpoints **Discovery** | Method | Path | | ------ | ------------------ | | `GET` | `/.well-known/ucp` | **Checkout sessions** | Method | Path | | ------ | ------------------------------------- | | `GET` | `/api/checkout-sessions` | | `POST` | `/api/checkout-sessions` | | `GET` | `/api/checkout-sessions/:id` | | `PUT` | `/api/checkout-sessions/:id` | | `POST` | `/api/checkout-sessions/:id/complete` | | `POST` | `/api/checkout-sessions/:id/cancel` | **Orders** | Method | Path | | ------ | ------------------------------------ | | `GET` | `/api/orders` · `/api/orders/:id` | | `POST` | `/api/orders/:id/fulfillment-events` | | `POST` | `/api/orders/:id/adjustments` | **Credentials and audit** | Method | Path | Purpose | | ------ | --------------------------- | ----------------------- | | `POST` | `/tokenize` · `/detokenize` | Credential tokenization | | `GET` | `/api/tokenizations` | List tokenizations | | `GET` | `/api/audit-events` | Audit trail | | `GET` | `/api/webhook-deliveries` | Webhook delivery log | **Optional OAuth identity linking** (when enabled): `/.well-known/oauth-authorization-server`, `/oauth2/authorize`, `/oauth2/token`, `/oauth2/revoke`. **Operations:** `/metrics` (Prometheus), `/health`, `/ready`. A gRPC server with JSON payloads, health, and reflection listens on `GRPC_HOST:GRPC_PORT` (default `0.0.0.0:50051`). ## Extensions | Extension | What it adds | | --------------------- | ---------------------------------------------------------- | | **Fulfillment** | Shipping options | | **Discount** | Promo codes | | **AP2 mandate** | Optional embedded authorization | | **OAuth 2.0** | Identity linking via authorization code flow | | **iCommerce backend** | Real execution with SQLite persistence, rather than a stub | Enabling the [iCommerce](/stateset-icommerce/stateset-icommerce-quickstart) backend turns the handler from a protocol endpoint into a working commerce system — the session actually reserves inventory and creates an order. ## Bindings Node.js, Python, and Go bindings are available in addition to the Rust crate, which is published on crates.io. ## When to use UCP | Situation | Why UCP fits | | --------------------------- | ---------------------------------------------------------------- | | **Cross-platform commerce** | One checkout contract instead of per-platform integrations | | **Marketplace apps** | Standard discovery, so capabilities are machine-readable | | **Agent workflows** | A predictable lifecycle an agent can drive without bespoke logic | ## How it relates to ACP and ICP | Protocol | Scope | | ------------------------------------- | -------------------------------------------------------------------------------------------- | | [**ACP**](/stateset-acp/stateset-acp) | ChatGPT-style conversational checkout and delegated payment | | **UCP** | Platform-neutral checkout interop — discovery, tokenization, OAuth, AP2 mandates | | [**ICP**](/stateset-icp) | The superset: agent identity, mandates, negotiation, returns, subscriptions, global commerce | ICP subsumes both. UCP is the right choice when you want checkout interoperability without adopting the full intent model. ## Related * [UCP Quickstart](/stateset-ucp-quickstart) * [UCP Integration Guide](/guides/universal-commerce-protocol-handler) * [ICP](/stateset-icp) — the superset protocol # UCP Quickstart Source: https://docs.stateset.com/stateset-ucp-quickstart Run the UCP handler locally and drive one checkout from discovery to completion. Run the [Universal Commerce Protocol](/stateset-ucp) handler locally and walk one full checkout: discover, create a session, complete it. ## 1 — Run the server ```bash theme={null} git clone https://github.com/stateset/stateset-ucp-handler.git cd stateset-ucp-handler UCP_REQUIRE_UCP_AGENT=false UCP_REQUIRE_REQUEST_SIGNATURE=false cargo run # listens on http://0.0.0.0:8081 ``` Those two flags disable caller authentication and are for **local testing only**. In production the `Request-Signature` header is what authenticates writes — see the [integration guide](/guides/universal-commerce-protocol-handler) for the signed setup. ## 2 — Discover One well-known document advertises what the merchant supports: ```bash theme={null} curl http://localhost:8081/.well-known/ucp ``` You get back the capability list — checkout, fulfillment, discounts, orders. A client should branch on this rather than assuming; that is the point of the discovery step. ## 3 — Create a checkout session ```bash theme={null} curl -X POST http://localhost:8081/api/checkout-sessions \ -H 'content-type: application/json' \ -d '{ "line_items": [{ "sku": "SHOE-RED-10", "quantity": 1 }], "currency": "USD" }' ``` The response carries the session `id` and its state. Update it (address, shipping option, discount code) with `PUT /api/checkout-sessions/{id}` as many times as needed. ## 4 — Complete it ```bash theme={null} curl -X POST http://localhost:8081/api/checkout-sessions/{id}/complete ``` Completion emits an **order webhook** — that, not the HTTP response, is the durable record of the purchase. Check `/api/webhook-deliveries` to see it. Enable the **iCommerce backend** to make this real: the session then reserves actual inventory and creates an order in the embedded engine, instead of a protocol-level stub. See [UCP](/stateset-ucp#extensions). ## Verify the whole loop ```bash theme={null} ./demo_test.sh # exercises discovery → session → complete → order ``` The handler requires `UCP-Agent` on every request and `Request-Signature` on writes. The local-testing flags that relax them turn off the only thing authenticating the caller, so a handler started with them accepts a checkout from anyone who can reach the port. Never set them on anything reachable beyond your own machine. ## Next * [UCP](/stateset-ucp) — endpoints, extensions, and how it relates to ACP and ICP * [Integration guide](/guides/universal-commerce-protocol-handler) — signatures, headers, production setup * [ICP](/stateset-icp) — the superset protocol, if you need mandates and negotiation ## Next steps Signatures, headers and the full endpoint set. The protocol this handler implements. The sibling protocol, and when to use which. Running one on managed Kubernetes. # Voice Skill Source: https://docs.stateset.com/stateset-voice-skill Skill file for agents managing StateSet Voice — voice agents, number routing, calls, transcripts, latency and the MCP server — over the tenant REST API. ## Overview **stateset-voice** is StateSet's AI phone platform: a Rust/axum server that answers and places Twilio calls with realtime-model voice agents, plus browser web calls over the identical media pipeline. Tenants drive it through a versioned REST API (`/api/v1/...`) with a single tenant API key, or through the bundled MCP server that wraps that API 1:1. * **Prod (default target):** `https://api.voice.stateset.com` (API docs also reference `https://voice.stateset.com`) * **Local dev:** `http://localhost:5050` * **Auth:** `Authorization: Bearer stsk___` (tenant API key) ## Two ways in ### 1. MCP server (`mcp-server/`, package `@stateset/voice-mcp-server`) Thin, stateless stdio server; every tool is one REST call with the configured key passed through as Bearer. ```bash theme={null} cd mcp-server && npm install && npm run build # -> dist/index.js ``` Env (both required): | Var | Value | | ------------------------- | ------------------------------------------------------------- | | `STATESET_VOICE_BASE_URL` | `https://api.voice.stateset.com` (or `http://localhost:5050`) | | `STATESET_VOICE_API_KEY` | tenant API key | Wire into a client (`claude mcp add`, or MCP config JSON). The package is not yet on npm; build it from the repository's `mcp-server/` directory: ```json theme={null} { "mcpServers": { "stateset-voice": { "command": "node", "args": ["/abs/path/rust-phone-server/mcp-server/dist/index.js"], "env": { "STATESET_VOICE_BASE_URL": "https://api.voice.stateset.com", "STATESET_VOICE_API_KEY": "YOUR_TENANT_API_KEY" } } } } ``` Smoke test: pipe `initialize` / `notifications/initialized` / `tools/list` JSON-RPC lines into `node dist/index.js` (exact lines in `mcp-server/README.md`). A `tools/call` with a bad key returns a clean `HTTP 401: Invalid tenant credentials` — path works, key doesn't. ### 2. Raw REST ```bash theme={null} curl -s https://api.voice.stateset.com/api/v1/voice/agents \ -H "Authorization: Bearer $STATESET_VOICE_API_KEY" ``` ## Tool ↔ endpoint map | MCP tool | REST | | -------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `get_account` | `GET /voice/account` (`PATCH` to update profile/greeting/handoff number) | | `list_agents` / `get_agent` | `GET /voice/agents`, `GET /voice/agents/{uuid}` | | `create_agent` | `POST /voice/agents` — `agent_key`, `name`, `config`, `publish` | | `update_agent` | `PATCH /voice/agents/{uuid}` — mints a NEW version from `config` | | `list_phone_numbers` | `GET /voice/phone-numbers` | | `upsert_phone_number` | `POST /voice/phone-numbers` — `phone_number`, `direction` (`inbound`/`outbound`/`sip_inbound`/`sip_outbound`), `agent_id` | | `delete_phone_number` | `DELETE /voice/phone-numbers/{uuid}` | | `search_knowledge` | `POST /voice/knowledge/search` — `query`, `limit` | | `list_knowledge_sources` / `upsert_knowledge_source` / `delete_knowledge_source` | `GET`/`POST /voice/knowledge/sources`, `DELETE /voice/knowledge/sources/{id}` — fast-answer sources (`id`, `answer`, `match_phrases`, `deterministic`) | | `list_webhook_endpoints` / `upsert_webhook_endpoint` | `GET`/`POST /webhooks/endpoints` — `url`, `secret`, `events[]` (empty list = all events); `POST /webhooks/endpoints/{id}/test` | | `list_api_keys` / `create_api_key` / `revoke_api_key` | `GET`/`POST /api-keys`, `DELETE /api-keys/{uuid}` | | `create_realtime_token` | `POST /voice/realtime/token` — short-lived observer WS token | | `create_web_call` | `POST /voice/web-calls` — signed WS URL + start message, no phone number | | `make_call` | `POST /make-call` — REAL outbound PSTN call; `POST /voice/calls` is the same hardened flow with extras | | `list_call_logs` / `get_call_log` | `GET /call-logs` (filter by `call_sid`, `status`, `outcome`, `direction`, number, date, transcript text), `GET /call-logs/{uuid}` — full transcript + summary | | `get_call_stats` / `get_call_latency` | `GET /call-logs/stats`, `GET /call-logs/latency` — p50/p95 voice latency | Not wrapped by MCP but in the REST API: voice sessions (`/voice/sessions...`), live-session supervisor actions (`monitor`/`whisper`/`barge`/`escalate`/`end` on `/voice/sessions/{stream_sid}/actions/*`), and translation calls (`POST /voice/translation-calls`). ## Agent `config` schema Accepted keys: `instructions`, `assistant_profile`, `realtime_model`, `chat_model`, `voice`, `greeting`, `agent_name`, `tools[]`. **Unknown keys are rejected with a 400 — never silently dropped.** ```json theme={null} { "agent_key": "front-desk", "name": "Front Desk", "publish": true, "config": { "agent_name": "Front Desk", "greeting": "Thanks for calling Acme Dental. How can I help?", "instructions": "Answer questions, book appointments, escalate urgent concerns.", "realtime_model": "gpt-4o-realtime-preview", "chat_model": "gpt-4o-mini", "voice": "alloy", "tools": [ { "name": "get_order_status", "description": "Look up an order.", "parameters": { "type": "object", "required": ["order_id"], "properties": { "order_id": { "type": "string" } } }, "endpoint": { "method": "POST", "url": "https://api.acme.example/voice/get-order-status", "auth_header": "Bearer $TOKEN", "timeout_ms": 3000 } } ] } } ``` `tools[]` are customer functions fulfilled by YOUR backend: mid-call the server POSTs `{tool, arguments, call}` to `endpoint.url` (HMAC `X-Webhook-Signature`); the JSON reply is the tool result. Validation is eager: POST-only, public-https URL, 250–10000 ms timeout, max 16 tools; names colliding with built-in platform functions are ignored in favor of the built-in. ## Canonical flows **Agent → number → call → transcript** 1. `create_agent` with `publish: true` (response includes agent + version UUIDs). 2. `upsert_phone_number` `{phone_number, direction: "inbound"|"outbound", agent_id}`. 3. `make_call` (or `POST /voice/calls`) → returns `call_sid`. 4. `list_call_logs {call_sid}` → `get_call_log {uuid}` for full transcript + summary. **Outbound extras on `POST /voice/calls`:** * `Idempotency-Key` header — same key within 24 h replays the stored outcome instead of dialing again; a concurrent duplicate gets `409` (retry shortly). On a deployment without a database the header itself returns `503` — the server never silently places a duplicate call. * `config` body — transient inline agent config for this call only (mutually exclusive with `agent_version_id`). * `metadata` (≤ 4 KB JSON) — persisted and inherited by retry children. * `machine_detection`: `off` | `enable` | `detect_message_end`; retries via `max_attempts`, `retry_on: ["no_answer","busy"]`, `retry_backoff_seconds`. **Web call (browser test, no phone number):** `create_web_call` → short-lived signed `websocket_url` + ready-to-send `start_message` (embedded `stream_token` is the whole auth story), audio `ulaw_8000` 20 ms base64 frames, `expires_at` \~5 min — mint one per call. Requires server `STREAM_AUTH_SECRET`. Full browser client + wire protocol: `docs/sdk-web.md` in the repo. **Knowledge:** `upsert_knowledge_source` → `search_knowledge {query}`; sources are deterministic low-latency fast answers used before broader search. ## Gotchas * **`make_call` dials a real phone.** DNC and TCPA quiet-hours gates apply — quiet hours evaluated in the *called party's* timezone (NANP); blocked calls return `403` (policy) or `503`. * **`create_api_key` returns the secret token exactly once**; store it immediately. `revoke_api_key` is irreversible. * **Agents are versioned.** `PATCH /voice/agents/{id}` creates a new version; it's not live unless `publish: true` (or published later). * **Unknown/invalid agent config → 400**, not silently ignored. * **Voice lag?** Start at `get_call_latency` (p50/p95), then per-call `get_call_log`. * Errors are `{ "ok": false, "error": "..." }`; common: 400 validation, 401 auth, 403 policy block (DNC/quiet hours), 409 duplicate, 503 missing backend (no DB, no `STREAM_AUTH_SECRET`, no Twilio creds). ## Further reading * [Voice API reference](/api-reference/voice/overview) — the 65 tenant endpoints * [Voice MCP server](/stateset-voice/mcp-server) — the 24 tools * [Voice webhooks](/stateset-voice/webhooks) # Voice API Source: https://docs.stateset.com/stateset-voice/api Call control, media streaming, session supervision, and the admin surface. Interactive documentation is served by the running instance at `/swagger-ui`, with the OpenAPI 3.0 document at `/api-docs/openapi.json`. Every tenant endpoint below also has a generated reference page with a live playground under the [Voice API tab](/api-reference/voice/overview) — 65 operations, built from the OpenAPI document the server publishes. This page is the narrative map; use the tab to make calls. ## Service | Endpoint | Purpose | | ---------------------------- | ------------------------------------------- | | `GET /` | Service metadata, timestamp, org id | | `GET /health` | Health information, active sessions, uptime | | `GET /health/live` | Kubernetes liveness probe | | `GET /health/ready` | Kubernetes readiness probe | | `GET /metrics` | Prometheus metrics | | `GET /swagger-ui` | Interactive OpenAPI documentation | | `GET /api-docs/openapi.json` | OpenAPI 3.0 specification | ## Calls | Endpoint | Purpose | | --------------------------------- | -------------------------------------------- | | `POST /make-call` | Place an outbound call via Twilio REST | | `POST /voice/web-calls` | Mint a signed browser voice session | | `POST /call-status` | Twilio status callbacks, signature-validated | | `GET /media-stream` *(WebSocket)* | Twilio media bridge | `POST /make-call` supports an **`Idempotency-Key`** with 24-hour replay protection, a transient per-call `config` (custom tools included) for prototyping, and persisted `metadata`. Browser calls run the same media protocol and pipeline as phone calls, so anything configured for telephony — agents, tools, transfers, evals — applies unchanged. ### Media stream authentication The WebSocket handshake authenticates a tenant by `Authorization: Bearer …` or a short-lived signed `?stream_token=…`. Raw tenant-token query transport is **opt-in** via `LEGACY_QUERY_AUTH_ENABLED=true` and exists for migration only. It defaults to off. Configure `STREAM_AUTH_SECRET` to issue signed `stream_token` values instead. In `NODE_ENV=production`, startup **fails fast** if legacy auth is off and `STREAM_AUTH_SECRET` is not configured — so you cannot accidentally deploy with neither. ```bash theme={null} # Place a call. POST /voice/calls is the same flow with idempotency and retries. curl --request POST "https://api.voice.stateset.com/api/v1/make-call" \ --header "Authorization: Bearer $STATESET_VOICE_API_KEY" \ --header "Content-Type: application/json" \ --data '{ "agent_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "to": "+14155550123" }' ``` ## Translation calls A translation call bridges two people who do not share a language: the platform dials both, and each hears the other translated in near real time. | Endpoint | Purpose | | --------------------------------------------- | --------------------------------------------- | | `POST /voice/translation-calls` | Dial both participants into a translated room | | `GET`/`POST` `/translation-call-intake/start` | Start one from an inbound intake flow | | `GET`/`POST` `/translation-call-leg` | Twilio leg callback | | `GET /translation-media-stream` *(WebSocket)* | The bridged media path | ```bash theme={null} curl --request POST "https://api.voice.stateset.com/api/v1/voice/translation-calls" \ --header "Authorization: Bearer $STATESET_VOICE_API_KEY" \ --header "Content-Type: application/json" \ --data '{ "participant_a": "+15551230001", "participant_b": "+15551230002", "language_a": "en", "language_b": "es" }' ``` ```json Response theme={null} { "ok": true, "room_id": "8f1f7d28-0ea9-4531-95d7-8d8f0fd72516", "participant_a_call_sid": "CA123", "participant_b_call_sid": "CA456" } ``` `language_a` and `language_b` default to `en` and `es`. Omitting them does not mean "detect the language" — it means English and Spanish. A call between two other languages that leaves them out will be translated into a pair neither participant is speaking. `ok: true` means both legs were placed, not that both were answered — the `call_sid` values are how you follow each leg's actual outcome. When Twilio is not configured the response carries `error` instead of the sids. ## Sessions and supervision | Endpoint | Purpose | | ------------------------------------ | ------------------------- | | `GET /sessions` | Enumerate active sessions | | `POST /sessions/:id/actions/:action` | Supervisor action | | `GET /logs` *(WebSocket)* | Real-time broadcast logs | Supervisor actions: **`monitor`**, **`whisper`**, **`barge`**, **`escalate`**, **`end`**. ```bash theme={null} # Whisper to the agent mid-call without the caller hearing it curl --request POST "%(b)s/voice/sessions/$STREAM_SID/actions/whisper" \ --header "Authorization: Bearer $STATESET_VOICE_API_KEY" \ --header "Content-Type: application/json" \ --data '{ "message": "Offer the replacement, not the refund." }' ``` Supervisor actions address a session by its **`stream_sid`**, not the call record id — a live session and its call log are different handles for the same call. Take the `stream_sid` from `GET /voice/sessions`; a supervisor action sent with a call id silently addresses nothing. ## Functions and automations | Endpoint | Purpose | | ----------------------- | ------------------------------------------------------ | | `GET /tools` | Function registry definitions, for OpenAI tool priming | | `POST /test-function` | Execute a function manually with a JSON payload | | `POST /automations/...` | Automation endpoints backed by the function registry | ## Human handoff `/transfer-call` dials a human with a bounded ring timeout (`twilio.human_agent_timeout_seconds`, default 25s). If nobody answers, `/transfer-complete` **reconnects the caller to the AI** with a message-taking greeting rather than Twilio's default hangup. A failed background escalation likewise speaks a recovery prompt — counted by `escalation_bridge_failed_total` — instead of leaving the caller on a silent line. ## Admin Admin routes require `ADMIN_API_KEY`, passed as `x-admin-key`. | Endpoint | Purpose | | ------------------------------------------------------- | -------------------------------------------------------------------- | | `GET /admin/sessions` | Active sessions across cached tenants | | `GET /admin/auth/transport` | Auth transport diagnostics | | `GET /admin/alerts/auth-transport` | Rollout alert evaluation — `ok`/`warn`/`error` with alert codes | | `GET /admin/tenants/cached` | Cached-tenant runtime summary | | `GET /admin/tenants/audio-providers` | Configured vs effective audio provider, fallback state and reason | | `GET /admin/tenants/cache-metrics` | Entries, hits, misses, reloads, evictions | | `GET /admin/tenants/cache-metrics/by-token` | Top token-level cache counters | | `POST /admin/tenants/reload-cached` | Reload cached tenants, preserving live session and log state | | `POST /admin/tenants/evict-idle` | Evict idle cache entries (`max_idle_secs`) | | `GET /admin/twilio/webhook-diagnostics` | Tenant-resolution diagnostics — hint-cache size, counters, hit rates | | `POST /admin/twilio/webhook-diagnostics/cache/clear` | Clear hint-cache entries for incident recovery | | `POST /admin/twilio/webhook-diagnostics/counters/reset` | Reset diagnostics counters | `GET /metrics` on the main port requires `x-admin-key`. The dedicated `METRICS_PORT` is **unauthenticated** and must be network-restricted. Admin tenant responses **redact secrets**, and updates can preserve existing values by sending the `` placeholder. Tenant reads expose deterministic config ETags, so writes can use `If-Match` to avoid lost updates. ## Next steps Tenant settings, audio providers and turn behaviour. Signature verification, shared with custom tool calls. TypeScript, Python and the browser Web SDK. First agent, number and call, end to end. # Voice Configuration Source: https://docs.stateset.com/stateset-voice/configuration Tenants, auth transport, audio providers, turn response mode, VAD, and readiness gating. ## Multi-tenancy Tenant configuration is database-backed via PostgreSQL, with an in-memory fallback for development. Cached tenant state is reused for performance and evicted when idle per `TENANT_CACHE_IDLE_TTL_SECS` (default 3600s). Reload cached tenants without dropping live sessions using `POST /admin/tenants/reload-cached`. ## Auth transport This is the setting most worth understanding before deploying. | Variable | Default | Purpose | | ----------------------------- | ------- | --------------------------------------------------------------------------------------------- | | `STREAM_AUTH_SECRET` | — | Signs short-lived `stream_token` values for Twilio media streams | | `STREAM_AUTH_SECRET_PREVIOUS` | — | Temporarily accepts in-flight tokens while rotating the secret, with no media-stream downtime | | `LEGACY_QUERY_AUTH_ENABLED` | `false` | Opt-in raw `auth_token`/`tenant_token` query transport, for migration only | `LEGACY_QUERY_AUTH_ENABLED=true` re-enables raw tenant tokens in query strings and Twilio custom-parameter bootstrap. It exists for migration scenarios and defaults to **off**. In `NODE_ENV=production`, startup **fails fast** when legacy auth is off but `STREAM_AUTH_SECRET` is unset — so a deployment cannot end up with neither transport working. Rotate `STREAM_AUTH_SECRET` by setting the old value in `STREAM_AUTH_SECRET_PREVIOUS`, deploying the new secret, then removing the previous once in-flight tokens have expired. ## Audio output Per-tenant `audio_output_provider`: **`elevenlabs`** (default) or **`openai`** (explicit override). ElevenLabs transport and 5xx/429 failures retry with backoff, and a **circuit breaker** prevents a provider outage from cascading. Provider outcome metrics are emitted, and `GET /admin/tenants/audio-providers` shows configured vs *effective* provider along with fallback state and reason. ### Provider base URLs `OPENAI_BASE_URL`, `ELEVENLABS_BASE_URL`, and `ELEVENLABS_WS_BASE_URL` redirect Chat Completions, TTS, and STT traffic to an LLM gateway, regional proxy, or test harness. Defaults are the public provider hosts. These are read **once at startup**. Changing them requires a restart. ```bash theme={null} # Read the tenant config, then write back with If-Match so a concurrent # update cannot be lost ETAG=$(curl -sD- -o /tmp/acct.json "https://api.voice.stateset.com/api/v1/voice/account" \ --header "Authorization: Bearer $STATESET_VOICE_API_KEY" \ | tr -d '\r' | awk -F': ' '/^ETag/{print $2}') curl --request PATCH "https://api.voice.stateset.com/api/v1/voice/account" \ --header "Authorization: Bearer $STATESET_VOICE_API_KEY" \ --header "Content-Type: application/json" \ --header "If-Match: $ETAG" \ --data '{ "audio_provider": "elevenlabs", "greeting": "Thanks for calling Acme. How can I help?" }' ``` Reading a tenant back **redacts its secrets** — provider keys come out as ``. Writing that value back preserves the stored secret, which is the intended behaviour; writing the literal string `` as a *new* key is not something the API can distinguish from meaning it. Send only the fields you are actually changing. ## Turn response mode Controls how assistant replies reach TTS on the Chat Completions path: | Mode | Behaviour | | ----------------------------- | -------------------------------------------------------------------------------------------------------- | | `single_sentence` *(default)* | Waits for the full completion, then speaks one sanitised sentence | | `streaming` | Flushes sentences to TTS as LLM deltas arrive — first audio ≈ first sentence, the lowest-latency posture | Resolution order: per-tenant `turn_response_mode` → process-wide `TURN_RESPONSE_MODE` → `single_sentence`. The per-tenant field is settable live via the admin tenant-config `PATCH`, so you can roll `streaming` out tenant by tenant rather than flipping the whole fleet. ## Voice activity detection VAD tuning — threshold, prefix padding, silence — is set **per tenant** in tenant config. Values are clamped to safe defaults with a warning at load, so a bad value degrades rather than breaks. ## Handoff workflows `receptionist.handoff_workflow` drives number-specific bridge targets and post-call notes from tenant config rather than compiled code. `twilio.human_agent_timeout_seconds` (default 25s) bounds the ring on a human handoff. See [Voice API](/stateset-voice/api#human-handoff) for the no-answer recovery path. ## Custom tools Agent versions accept `tools[]` — a JSON-schema function plus your HTTPS endpoint. Mid-call invocations POST to your endpoint with `Authorization` passthrough and the webhook-style `X-Webhook-Signature` HMAC; the JSON reply becomes the tool result. `POST /make-call` also accepts a transient per-call `config` including tools, for prototyping without publishing an agent version. ## Rate limiting Per-IP and per-tenant token buckets with configurable limits and automatic cleanup. Tenant key extraction supports bearer auth, signed `stream_token`, and — only when explicitly enabled — legacy raw token transport. ## Readiness gating Kafka is probed and reported in `/health/ready`, but only fails readiness when `HEALTH_READY_REQUIRE_KAFKA=true`. That default is deliberate: a broker outage cannot pull call-serving pods out of rotation. Enable it only if you genuinely want calls to stop when Kafka is down. ## Admin safety * Admin tenant responses **redact secrets**; updates preserve existing values when you send the `` placeholder. * Tenant reads expose deterministic config **ETags**, so writes can use `If-Match` to prevent lost updates. * `ADMIN_API_KEY` is required on admin routes via `x-admin-key`. ## Related * [Voice API](/stateset-voice/api) * [Operations](/stateset-voice/operations) # Voice MCP Server Source: https://docs.stateset.com/stateset-voice/mcp-server Drive the voice platform from Claude — agents, phone routing, knowledge, calls, and call logs as MCP tools. A thin **stdio** MCP server wrapping the versioned voice REST API (`/api/v1/...`) with a single tenant **API key** as a Bearer token. It holds no state and adds no surface to the voice server itself, so it stays in lockstep with the REST API. **24 tools.** ## Package | | | | --------- | ---------------------------- | | Package | `@stateset/voice-mcp-server` | | Binary | `stateset-voice-mcp` | | Transport | stdio | The package is not yet published to npm. Until it is, build it in-repo (`cd mcp-server && npm install && npm run build`) and point your host at `dist/index.js` as below. Once published, `npx -y @stateset/voice-mcp-server` replaces the `command`/`args` pair. ## Connect ```json theme={null} { "mcpServers": { "stateset-voice": { "command": "node", "args": ["/absolute/path/to/rust-phone-server/mcp-server/dist/index.js"], "env": { "STATESET_VOICE_BASE_URL": "https://api.voice.stateset.com", "STATESET_VOICE_API_KEY": "YOUR_TENANT_API_KEY" } } } } ``` With Claude Code: ```bash theme={null} claude mcp add stateset-voice \ --env STATESET_VOICE_BASE_URL=https://api.voice.stateset.com \ --env STATESET_VOICE_API_KEY=YOUR_TENANT_API_KEY \ -- node /absolute/path/to/rust-phone-server/mcp-server/dist/index.js ``` ## Tools ### Account and agents | Tool | Maps to | | -------------- | ------------------------------------------------------------------ | | `get_account` | `GET /voice/account` | | `list_agents` | `GET /voice/agents` | | `get_agent` | `GET /voice/agents/{uuid}` | | `create_agent` | `POST /voice/agents` — `agent_key`, `name`, `config`, `publish` | | `update_agent` | `PATCH /voice/agents/{uuid}` — creates a new version from `config` | ### Phone-number routing | Tool | Maps to | | --------------------- | --------------------------------------------------------------------- | | `list_phone_numbers` | `GET /voice/phone-numbers` | | `upsert_phone_number` | `POST /voice/phone-numbers` — `phone_number`, `direction`, `agent_id` | | `delete_phone_number` | `DELETE /voice/phone-numbers/{uuid}` | ### Knowledge base | Tool | Maps to | | ------------------------- | ----------------------------------------------------------------- | | `search_knowledge` | `POST /voice/knowledge/search` — `query`, `limit` | | `list_knowledge_sources` | `GET /voice/knowledge/sources` | | `upsert_knowledge_source` | `POST /voice/knowledge/sources` — `id`, `answer`, `match_phrases` | | `delete_knowledge_source` | `DELETE /voice/knowledge/sources/{id}` | ### Webhooks and API keys | Tool | Maps to | | ------------------------- | ------------------------------------------------------ | | `list_webhook_endpoints` | `GET /webhooks/endpoints` | | `upsert_webhook_endpoint` | `POST /webhooks/endpoints` — `url`, `secret`, `events` | | `list_api_keys` | `GET /api-keys` | | `create_api_key` | `POST /api-keys` | | `revoke_api_key` | `DELETE /api-keys/{uuid}` | ### Calls | Tool | Maps to | | ----------------------- | --------------------------------------------------------------------------- | | `create_realtime_token` | `POST /voice/realtime/token` — short-lived observer token | | `create_web_call` | `POST /voice/web-calls` — signed WS URL plus start message, no phone number | | `make_call` | `POST /make-call` | ### Call logs and monitoring | Tool | Maps to | | ------------------ | ------------------------------------------------------------------------------------------------------------ | | `list_call_logs` | `GET /call-logs` — inbound and outbound, filterable by `call_sid`, status, outcome, direction, number, date | | `get_call_log` | `GET /call-logs/{uuid}` — one call in full: status, duration, transcript, summary, outcome | | `get_call_stats` | `GET /call-logs/stats` — total, answered and failed counts, answer rate, average duration, outcome breakdown | | `get_call_latency` | `GET /call-logs/latency` — p50/p95 voice response latency, the first place to look when a call feels slow | Two tools have real-world consequences, so scope the key you give an agent accordingly: * **`make_call` places an actual outbound PSTN call.** DNC and TCPA gates apply, but the call is real and billable. * **`create_api_key` mints a tenant credential**, returned once. An agent holding this can issue further credentials. ## Related * [Voice API](/stateset-voice/api) * [Webhooks](/stateset-voice/webhooks) * [Configuration](/stateset-voice/configuration) * [All MCP servers](/mcp-servers) # Voice Operations Source: https://docs.stateset.com/stateset-voice/operations Deployment, health probes, metrics, alerting, and scaling for the voice platform. ## Deployment The repo ships Docker Compose for local and single-host use, and Kubernetes manifests for production — namespace, deployment, configmap, ingress, a separate **admin ingress**, and an HPA. Admin routes have their own ingress. Keep it internal — it exposes tenant cache controls, diagnostics, and session data, gated only by `ADMIN_API_KEY`. ## Health probes | Endpoint | Use | | ------------------- | ------------------------------------ | | `GET /health` | Health info, active sessions, uptime | | `GET /health/live` | Kubernetes liveness | | `GET /health/ready` | Kubernetes readiness | Kafka is probed and reported in readiness but only **fails** it when `HEALTH_READY_REQUIRE_KAFKA=true` — so by default a broker outage cannot pull call-serving pods out of rotation. ## Metrics Prometheus, with pre-configured Grafana dashboards, plus Alertmanager in the Compose stack. | Port | Auth | | ------------------------ | ---------------------- | | Main port `/metrics` | Requires `x-admin-key` | | Dedicated `METRICS_PORT` | **Unauthenticated** | The dedicated metrics port is unauthenticated by design and must be network-restricted. Do not expose it through a public ingress. Metrics worth alerting on: * `escalation_bridge_failed_total` — a background escalation failed and the caller got a recovery prompt instead of a human. * Provider outcome metrics for ElevenLabs — these reveal circuit-breaker trips before callers report bad audio. ```bash theme={null} # Where latency actually sits, before guessing curl "https://api.voice.stateset.com/api/v1/call-logs/latency" \ --header "Authorization: Bearer $STATESET_VOICE_API_KEY" # Then the individual call that looked wrong curl "https://api.voice.stateset.com/api/v1/call-logs?call_sid=CA9f2c" \ --header "Authorization: Bearer $STATESET_VOICE_API_KEY" ``` Start every "the agent feels slow" investigation at `/call-logs/latency` for p50/p95 rather than at a single call. One slow call is usually a provider hiccup; a moved p95 is a configuration change, and the two have completely different fixes. ### Whose latency is it "The agent is slow" is two different problems, and the metrics separate them. `tts_time_to_first_frame_ms` — the gap between deciding to speak and the first audio frame reaching the caller — is labelled by `provider` and by response `mode`, so the same query answers whether the speech vendor got slower or whether your own pipeline did. | Metric | Measures | | ---------------------------- | -------------------------------------------------------------- | | `latency_llm_first_token_ms` | Time to the model's first token — your prompt and model choice | | `latency_sentence_ready_ms` | Time until a sentence is ready to speak | | `tts_time_to_first_frame_ms` | Time until the caller hears audio, split by `provider` | | `latency_barge_in_ms` | How fast the agent yields when the caller interrupts | Split before you optimise. A `tts_time_to_first_frame_ms` that rose while `latency_llm_first_token_ms` held flat is a speech-vendor problem, and no amount of prompt tuning will move it — switching the audio provider will. The reverse is equally true, and the two are routinely confused because the caller experiences both as the same silence. ## Auth transport rollout `GET /admin/alerts/auth-transport` evaluates rollout state as `ok` / `warn` / `error` with actionable alert codes, and `GET /admin/auth/transport` shows the legacy fallback toggle and stream-secret readiness. Use these while migrating off legacy query auth — they tell you whether any tenant still depends on it before you turn it off. ## Tenant cache | Route | Use | | ------------------------------------------- | -------------------------------------------------- | | `GET /admin/tenants/cache-metrics` | Entries, hits, misses, reloads, evictions | | `GET /admin/tenants/cache-metrics/by-token` | Top token-level counters | | `POST /admin/tenants/reload-cached` | Reload while preserving live session and log state | | `POST /admin/tenants/evict-idle` | Evict idle entries (`max_idle_secs`) | Idle entries also expire automatically per `TENANT_CACHE_IDLE_TTL_SECS` (default 3600s). ## Twilio webhook diagnostics When inbound calls fail to resolve to the right tenant: | Route | Use | | ------------------------------------------------------- | ---------------------------------------------------------------------------- | | `GET /admin/twilio/webhook-diagnostics` | Hint-cache size and limits, resolution counters, success and cache-hit rates | | `POST /admin/twilio/webhook-diagnostics/cache/clear` | Clear hint-cache entries during incident recovery | | `POST /admin/twilio/webhook-diagnostics/counters/reset` | Reset counters | ## Resilience * **Retries** with exponential backoff on external API calls. * **Circuit breaker** on ElevenLabs, so transport and 5xx/429 failures don't cascade. * **Graceful shutdown** with resource cleanup. * **Rate limiting** per IP and per tenant, with automatic bucket cleanup. ## Load testing A `load-test/` harness ships with the repo for validating capacity before a launch. ## Related * [Configuration](/stateset-voice/configuration) * [Voice API](/stateset-voice/api) — the full admin surface # StateSet iCommerce Voice Engine Source: https://docs.stateset.com/stateset-voice/overview Production AI voice agents over Twilio and OpenAI Realtime — multi-tenant, streaming, observable. Production-grade AI voice agents over **Twilio** and the **OpenAI Realtime API**, written in Rust. Multi-tenant, streaming, and observable. ``` Caller ──PSTN / SIP──▶ Twilio │ signed webhook ▼ Stateset Voice ◀──Realtime WS──▶ OpenAI Realtime │ │ │ tool call ──┘ │ └── signed webhook ──▶ your receiver (your backend) │ └── persisted ──▶ VoiceSession + transcripts ``` A browser call uses the same media protocol and pipeline as a phone call, so [web calls](/stateset-voice/sdks#web-sdk-browser-calls) get the whole platform — agents, custom tools, transfers, evals, billing — with nothing extra to configure. ## Capabilities Twilio REST plus webhooks with signature validation, inbound and outbound PSTN, and a WebSocket media bridge. OpenAI Realtime bridge with WebSocket media streaming, plus optional ElevenLabs TTS as the audio output provider. Monitor, whisper, barge, escalate, and end an in-flight session. Bring your own backend — a JSON-schema function plus an HTTPS endpoint, invoked mid-call. Also included: a function registry for business logic (auth, subscriptions, orders, returns, exchanges, logistics), multi-tenant configuration, Kafka event streaming, and voice activity detection with per-tenant tuning. ## Production surface | Area | What ships | | ----------------- | ------------------------------------------------------------------ | | **Rate limiting** | Token bucket, per-IP and per-tenant, with automatic bucket cleanup | | **Metrics** | Prometheus with pre-configured Grafana dashboards | | **Health** | Kubernetes liveness and readiness probes | | **API docs** | OpenAPI 3.0 with interactive Swagger UI at `/swagger-ui` | | **Resilience** | Exponential-backoff retries, plus a circuit breaker on ElevenLabs | | **Debugging** | Real-time log broadcast over WebSocket | | **Shutdown** | Graceful, with resource cleanup | ## Stack Axum 0.7 on Tokio, SQLx against PostgreSQL for tenant configuration (with an in-memory fallback for development), DashMap for rate-limit and session state, Tokio-Tungstenite for WebSockets, RDKafka for event streaming, and Utoipa for the OpenAPI spec. Twilio signature validation uses HMAC with constant-time comparison, so it is resistant to timing attacks. ## Getting started ```bash theme={null} cp .env.example .env # add your credentials docker-compose up -d curl http://localhost:5050/health ``` The Compose stack brings up the server, PostgreSQL, Kafka, Prometheus, Alertmanager, and Grafana. | Service | URL | | ------------ | ---------------------------------- | | Phone server | `http://localhost:5050` | | Swagger UI | `http://localhost:5050/swagger-ui` | | Prometheus | `http://localhost:9090` | | Alertmanager | `http://localhost:9093` | Rust 1.75+ is recommended; Docker 24+ for the containerised path. ```bash theme={null} # The smallest thing that proves the platform is reachable and your key works curl "https://api.voice.stateset.com/api/v1/voice/agents" \ --header "Authorization: Bearer $STATESET_VOICE_API_KEY" ``` Voice is a separate service from the commerce and agent APIs: it is served from `api.voice.stateset.com`, and it takes a tenant key (`stsk_…`) rather than a platform key. A key that works against `api.stateset.com` returns `401` here. ## Next End to end, from a phone number to a working agent. Call control, sessions, supervisor actions, admin. TypeScript, Python, and browser web calls. The event catalog and signature verification. Tenants, auth transport, audio providers, VAD. Drive the platform from Claude. # Build Your First Voice Agent Source: https://docs.stateset.com/stateset-voice/quickstart From a tenant API key to a live call with a custom tool, end to end. The end-to-end path: mint a key, configure a tenant, create an agent with a custom tool, attach a phone number, register a webhook, place a test call, and watch it live. ## Prerequisites * A running voice server — see [Overview](/stateset-voice/overview#getting-started) for the Docker Compose stack * Twilio credentials and a phone number * An OpenAI key (and an ElevenLabs key if you want its TTS) Everything below runs against `https://api.voice.stateset.com/api/v1`, or `http://localhost:5050/api/v1` against a local server. ## 1. Mint a tenant API key Tenant credentials are the unit of access. Every subsequent call uses one as a Bearer token. Create your first key in the console; after that, a key can mint further keys for itself: ```bash theme={null} curl --request POST "https://api.voice.stateset.com/api/v1/api-keys" \ --header "Authorization: Bearer $STATESET_VOICE_API_KEY" \ --header "Content-Type: application/json" \ --data '{ "name": "front-desk-prod" }' ``` ```json Response theme={null} { "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "name": "front-desk-prod", "token": "stsk_acme_01H8_9f2c…" } ``` ```bash theme={null} export STATESET_VOICE_API_KEY="stsk_acme_01H8_9f2c…" ``` The key is returned **once**. Store it before moving on — it can be revoked and reissued, not retrieved. ## 2. Configure the tenant account Set the tenant's provider credentials and defaults — audio output provider, VAD tuning, and handoff workflow. See [Configuration](/stateset-voice/configuration). ```bash theme={null} curl --request PATCH "https://api.voice.stateset.com/api/v1/voice/account" \ --header "Authorization: Bearer $STATESET_VOICE_API_KEY" \ --header "Content-Type: application/json" \ --data '{ "greeting": "Thanks for calling Acme Dental. How can I help?", "handoff_number": "+14155550100" }' ``` ## 3. Create the agent An agent version carries its prompt, model settings, and `tools[]`. Each tool is a JSON-schema function plus **your** HTTPS endpoint. ```bash theme={null} curl --request POST "https://api.voice.stateset.com/api/v1/voice/agents" \ --header "Authorization: Bearer $STATESET_VOICE_API_KEY" \ --header "Content-Type: application/json" \ --data '{ "agent_key": "front-desk", "name": "Front Desk", "publish": true, "config": { "agent_name": "Front Desk", "greeting": "Thanks for calling Acme Dental. How can I help?", "instructions": "Answer questions, book appointments, escalate urgent concerns.", "realtime_model": "gpt-4o-realtime-preview", "voice": "alloy", "tools": [ { "name": "get_order_status", "description": "Look up an order.", "parameters": { "type": "object", "required": ["order_id"], "properties": { "order_id": { "type": "string" } } }, "endpoint": { "method": "POST", "url": "https://api.acme.example/voice/get-order-status", "timeout_ms": 3000 } } ] } }' ``` Unknown keys in `config` are rejected with a `400` rather than ignored, and a `PATCH` mints a new version that is not live unless you publish it. If an edit appears to have no effect, check which version is published before looking at the prompt. ### How your tool endpoint is called Mid-call, the server POSTs to your endpoint with: * `Authorization` passthrough * An `X-Webhook-Signature` HMAC over the raw body — the **same** scheme as webhooks Your JSON reply becomes the tool result the model sees. Verify that signature with the same helper you use for webhooks. One implementation covers both — see [Verifying signatures](/stateset-voice/webhooks#verifying-signatures). ## 4. Attach a phone number Route a number to the agent with `direction` and `agent_id`. Inbound calls to that number now reach this agent. ```bash theme={null} curl --request POST "https://api.voice.stateset.com/api/v1/voice/phone-numbers" \ --header "Authorization: Bearer $STATESET_VOICE_API_KEY" \ --header "Content-Type: application/json" \ --data '{ "phone_number": "+14155550123", "direction": "inbound", "agent_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6" }' ``` ## 5. Register a webhook for call results Point an endpoint at the events you care about. For a first agent, `voice.call.started`, `voice.call.ended`, and `voice.session.completed` are enough — the last one carries persisted transcripts. ```bash theme={null} curl --request POST "https://api.voice.stateset.com/api/v1/webhooks/endpoints" \ --header "Authorization: Bearer $STATESET_VOICE_API_KEY" \ --header "Content-Type: application/json" \ --data '{ "url": "https://example.com/hooks/voice", "secret": "whsec_…", "events": ["voice.call.started", "voice.call.ended", "voice.session.completed"] }' ``` ## 6. Place a test call Test in the browser first — a web call runs the same pipeline with no number and no charges: ```bash theme={null} curl --request POST "https://api.voice.stateset.com/api/v1/voice/web-calls" \ --header "Authorization: Bearer $STATESET_VOICE_API_KEY" \ --header "Content-Type: application/json" \ --data '{ "agent_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6" }' ``` When that sounds right, dial for real: ```bash theme={null} curl --request POST "https://api.voice.stateset.com/api/v1/voice/calls" \ --header "Authorization: Bearer $STATESET_VOICE_API_KEY" \ --header "Content-Type: application/json" \ --header "Idempotency-Key: appointment-reminder-88213" \ --data '{ "to": "+14155550123", "agent_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6" }' ``` This dials a real phone. DNC and TCPA quiet-hours checks apply, evaluated in the **called party's** timezone, and a blocked call returns `403` rather than ringing. Send an `Idempotency-Key` derived from the thing you are calling about: the same key inside 24 hours replays the stored outcome instead of dialling again. ## 7. Read the transcript ```bash theme={null} # Find the call curl "https://api.voice.stateset.com/api/v1/call-logs?call_sid=CA9f2c…" \ --header "Authorization: Bearer $STATESET_VOICE_API_KEY" # Full transcript and summary curl "https://api.voice.stateset.com/api/v1/call-logs/3fa85f64-5717-4562-b3fc-2c963f66afa6" \ --header "Authorization: Bearer $STATESET_VOICE_API_KEY" ``` If replies feel laggy, start at `GET /call-logs/latency` for p50/p95 rather than guessing — then drill into the individual call log. See the full [event catalog](/stateset-voice/webhooks#event-catalog). ## 6. Place a test call Either call the number, or place an outbound call with `POST /make-call`. Faster still: `POST /voice/web-calls` gives you a browser call with **no phone number needed**, on the same pipeline. It's the tightest loop while iterating on a prompt. Send an `Idempotency-Key` on `make-call` — 24-hour replay protection means a retried request won't place a second real call. ## 7. Inspect the session afterwards Session records and transcripts are persisted. `voice.session.completed` delivers them to your webhook, and the session APIs let you fetch them directly. ## 8. Watch a live call | Route | Use | | ------------------------------------ | ------------------------------------------------ | | `GET /sessions` | Enumerate active sessions | | `GET /logs` *(WebSocket)* | Real-time log broadcast | | `POST /sessions/:id/actions/:action` | `monitor`, `whisper`, `barge`, `escalate`, `end` | `monitor` listens; `whisper` speaks to the agent only; `barge` joins the caller; `escalate` hands to a human; `end` terminates. ## Troubleshooting | Symptom | Check | | ---------------------------------- | -------------------------------------------------------------------------------------------------------------- | | Twilio webhooks rejected | Signature validation — credentials must match the account sending them | | Media stream fails to authenticate | `STREAM_AUTH_SECRET` is set, or legacy transport is explicitly enabled. Production **fails fast** with neither | | No audio from the agent | `GET /admin/tenants/audio-providers` — compare configured vs effective provider and the fallback reason | | Agent talks over the caller | Per-tenant VAD threshold and silence padding | | First audio feels slow | Try `turn_response_mode: streaming` for that tenant | ## Next TypeScript, Python, browser calls. Auth transport, providers, VAD, readiness. Events and signature verification. Build and tune agents from Claude. # Voice SDKs Source: https://docs.stateset.com/stateset-voice/sdks TypeScript, Python, and the browser Web SDK for in-page voice calls. Three client paths, all against the same REST API and media pipeline. ## TypeScript Generated from the OpenAPI spec, or hand-rolled against the REST API. Configure a base URL and an auth header, and an outbound call is a handful of lines. The SDK guide covers: * Placing an outbound call, minimally * Outbound calls **with idempotency and retries** * Inbound webhook handlers for **Express** and **Next.js App Router** ```typescript theme={null} const res = await fetch("https://api.voice.stateset.com/api/v1/make-call", { method: "POST", headers: { Authorization: `Bearer ${process.env.STATESET_VOICE_API_KEY}`, "Content-Type": "application/json", // Derived from the thing being called, not generated per attempt "Idempotency-Key": `reminder:${appointment.id}`, }, body: JSON.stringify({ agent_id: "agent_8Kx2mN4pQr", to: "+14155550123", from: "+14155550100", }), }); ``` Send an `Idempotency-Key` on `POST /make-call`. It gives you 24-hour replay protection, so a retry after a network timeout will not place a second real call. ## Python The same surface for Python services, including webhook signature verification. ## Web SDK (browser calls) `POST /voice/web-calls` mints a **signed browser voice session** — a WebSocket URL plus a start message. ``` Browser mic + speaker ◀──WS: start / media / stop──▶ Stateset Voice ``` ```javascript theme={null} // Mint a browser session server-side; never expose the API key to the page. const session = await fetch("https://api.voice.stateset.com/api/v1/voice/web-calls", { method: "POST", headers: { Authorization: `Bearer ${process.env.STATESET_VOICE_API_KEY}`, "Content-Type": "application/json", }, body: JSON.stringify({ agent_id: "agent_8Kx2mN4pQr" }), }).then((r) => r.json()); // The browser connects with the signed URL only. const ws = new WebSocket(session.websocket_url); ws.onopen = () => ws.send(JSON.stringify(session.start_message)); ``` Because a web call uses the same media protocol and pipeline as a phone call, it inherits the entire platform — agents, custom tools, transfers with no-answer recovery, evals, billing — with nothing extra to configure. No phone number is involved, so a browser call needs no Twilio number provisioning. It is the fastest way to exercise an agent end to end while building. ## Verifying webhooks All three SDK paths need the same verification: HMAC-SHA256 over the **raw** request body, compared in constant time. See [Webhooks](/stateset-voice/webhooks#verifying-signatures). The same helper also covers **custom tool call** invocations, which are signed identically. ## Next steps Full endpoint reference for calls, agents and numbers. Signature verification, shared by all three SDK paths. A first call, end to end. Configuring what the agent says. # Voice Webhooks Source: https://docs.stateset.com/stateset-voice/webhooks The event catalog, delivery headers, signature verification, and retry behaviour. The voice server posts signed events to your endpoint for call lifecycle, session and supervisor activity, and callback-task workflow. ## Delivery headers Every outbound POST includes: | Header | Value | | --------------------- | ----------------------------------------------------- | | `Content-Type` | `application/json` | | `User-Agent` | `stateset-phone-server/` | | `X-Webhook-Event` | Event name, e.g. `voice.call.started` | | `X-Webhook-Tenant` | Tenant that owns the event | | `X-Idempotency-Key` | Stable hash for safe retry | | `X-Webhook-Signature` | HMAC-SHA256 hex of the raw body, when a secret is set | | `traceparent` | W3C trace context for correlation | ## Verifying signatures The signature is `hex(HMAC_SHA256(secret, raw_request_body))`. Verify against the **raw bytes** of the body, before any JSON parsing, and use a **constant-time** comparison. Re-serialising the parsed body will not reproduce the same bytes, so the signature will not match. ```js theme={null} import crypto from 'node:crypto'; function verify(rawBody, signature, secret) { const expected = crypto.createHmac('sha256', secret).update(rawBody).digest(); const provided = Buffer.from(signature, 'hex'); // Check length first: timingSafeEqual throws if the buffers differ in size, // so a short or malformed signature would raise instead of returning false. return provided.length === expected.length && crypto.timingSafeEqual(provided, expected); } ``` **Custom tool calls are signed the same way.** An agent's `tools[]` endpoints receive the same `X-Webhook-Signature` header, the same secret, and the same scheme — so one verification helper covers both webhooks and tool invocations. Reference implementations for Node.js, Python, and Rust are in the repo's webhook guide. ## Event catalog ### Call lifecycle | Event | Fires when | | ------------------------------------- | ------------------------------------------- | | `voice.call.started` / `call.started` | A call begins | | `call.status` | Twilio reports a status change | | `call.recording` | A recording becomes available | | `voice.call.ended` / `call.ended` | The call ends | | `call.completed` | The call reaches a completed terminal state | ### Sessions and supervision | Event | Fires when | | ------------------------- | ---------------------------------------------------------- | | `session.started` | A voice session opens | | `session.updated` | Session state changes | | `session.action` | A supervisor acts — monitor, whisper, barge, escalate, end | | `session.escalated` | The session escalates to a human | | `session.ended` | The session closes | | `voice.session.completed` | The session completes with transcripts persisted | ### Callback tasks The follow-up workflow emits a fuller set, useful for building an SLA-aware queue: `callback_task.updated`, `.dispatch_initiated`, `.dispatch_status`, `.dispatch_status_updated`, `.completed`, `.reopened`, `.snooze_expired`, `.follow_up_notification_sent`, `.follow_up_escalated`, `.human_takeover_required`, `.sla_breached`, `.external_reconciled`. `callback_task.sla_breached` and `.human_takeover_required` are the two worth alerting on — they signal work that will not complete without a person. ## Retry and durability Deliveries are retried, and every POST carries a stable `X-Idempotency-Key`. Key your handler on it so a retried delivery is a no-op rather than a duplicate. ## Related * [Voice API](/stateset-voice/api) * [SDKs](/stateset-voice/sdks) — inbound webhook handlers for Express and Next.js * [Configuration](/stateset-voice/configuration) # StateSet Worker Source: https://docs.stateset.com/stateset-worker Run the StateSet iCommerce engine in a Cloudflare Sandbox Run the [StateSet](https://stateset.io/) iCommerce engine in a [Cloudflare Sandbox](https://developers.cloudflare.com/sandbox/) — `wrangler deploy` and you have a full commerce engine with messaging channels, MCP tools, and an admin UI. ## Architecture ``` Browser / AI Agent / Messaging Platform │ ┌───────┴──────────────────────────────────┐ │ Cloudflare Worker (Hono.js) │ │ - CF Access auth │ │ - Admin UI (React SPA) │ │ - HTTP/WebSocket proxy to container │ │ - Cron: SQLite backup to R2 │ └───────┬──────────────────────────────────┘ │ ┌───────┴──────────────────────────────────┐ │ CF Container (Durable Object) │ │ - @stateset/cli (stateset-channels.js) │ │ - @stateset/embedded (Rust/SQLite) │ │ - 6 messaging channels │ │ - MCP server (923 tools) │ │ - HTTP gateway on port 8080 │ │ - R2 mounted at /data/stateset │ └──────────────────────────────────────────┘ ``` A Worker deployment is reachable from the internet the moment it is published. Configure Cloudflare Access before you deploy, not after — the admin UI below ships with the Worker, and an unprotected deployment exposes commerce data to anyone who finds the URL. ## Requirements * [Workers Paid plan](https://www.cloudflare.com/plans/developer-platform/) (\$5 USD/month) — required for Cloudflare Sandbox containers * [Anthropic API key](https://console.anthropic.com/) — or use [AI Gateway](https://developers.cloudflare.com/ai-gateway/) for routing and analytics Free-tier Cloudflare features used: * Cloudflare Access (authentication) * AI Gateway (optional, for API routing/analytics) * R2 Storage (optional, for persistence) ## Quick Start ```bash theme={null} # Install dependencies npm install # Set your API key npx wrangler secret put ANTHROPIC_API_KEY # Deploy npm run deploy ``` After deploying, your commerce engine is live at `https://stateset-sandbox.your-subdomain.workers.dev/`. The first request takes 1-2 minutes while the container boots. A loading page is shown during startup. To use the admin UI and protected routes, you'll need to: 1. [Set up Cloudflare Access](#setting-up-cloudflare-access) for authentication 2. [Enable R2 storage](#persistent-storage-r2) so commerce data persists (recommended) ## Setting Up Cloudflare Access The admin UI at `/_admin/` and all `/api/admin/*` routes require Cloudflare Access authentication. ### 1. Enable Access on workers.dev 1. Go to the [Workers & Pages dashboard](https://dash.cloudflare.com/?to=/:account/workers-and-pages) 2. Select your Worker (`stateset-sandbox`) 3. In **Settings** > **Domains & Routes**, click the `...` menu on the workers.dev row 4. Click **Enable Cloudflare Access** 5. Configure who can access (email allow list, Google, GitHub, etc.) 6. Copy the **Application Audience (AUD)** tag ### 2. Set Access Secrets ```bash theme={null} npx wrangler secret put CF_ACCESS_TEAM_DOMAIN # Enter: myteam.cloudflareaccess.com npx wrangler secret put CF_ACCESS_AUD # Enter: your-application-audience-tag ``` Find your team domain in the [Zero Trust Dashboard](https://one.dash.cloudflare.com/) under **Settings** > **Custom Pages**. ### 3. Redeploy ```bash theme={null} npm run deploy ``` Now `/_admin/` requires Cloudflare Access authentication. ## Persistent Storage (R2) By default, commerce data (SQLite database, config) is lost when the container restarts. R2 storage enables persistence. ### Setup 1. Go to **R2** > **Overview** in the [Cloudflare Dashboard](https://dash.cloudflare.com/) 2. Click **Manage R2 API Tokens** 3. Create a token with **Object Read & Write** permissions for the `stateset-data` bucket ```bash theme={null} npx wrangler secret put R2_ACCESS_KEY_ID npx wrangler secret put R2_SECRET_ACCESS_KEY npx wrangler secret put CF_ACCOUNT_ID ``` ### How It Works **SQLite backup** — The commerce database is backed up safely using SQLite's built-in `.backup` API, which handles concurrent writes correctly. A cron job runs every 5 minutes. **Config sync** — Gateway configuration is synced to R2 via rsync. **Restore on boot** — On container startup, data is restored from R2 if the backup is newer than local data. An integrity check verifies the database before restoring. **Manual backup** — Trigger an immediate backup from the admin UI Storage tab or via `POST /api/admin/storage/sync`. ## Admin UI Access the admin UI at `/_admin/` with three tabs: * **Overview** — Gateway status, commerce stats (customers, orders, returns, products), channel summary, restart controls * **Channels** — Per-channel status cards for all 6 messaging channels (Telegram, Discord, Slack, WhatsApp, Email, SMS) * **Storage** — R2 configuration status, last backup time, manual backup button, backup strategy info ## Messaging Channels Configure channels via environment secrets. Each channel is enabled automatically when its token is set. ### Telegram ```bash theme={null} npx wrangler secret put TELEGRAM_BOT_TOKEN ``` ### Discord ```bash theme={null} npx wrangler secret put DISCORD_BOT_TOKEN ``` ### Slack ```bash theme={null} npx wrangler secret put SLACK_BOT_TOKEN npx wrangler secret put SLACK_APP_TOKEN ``` ### WhatsApp ```bash theme={null} npx wrangler secret put WHATSAPP_TOKEN ``` ### Email ```bash theme={null} npx wrangler secret put EMAIL_SMTP_HOST ``` ### SMS (Twilio) ```bash theme={null} npx wrangler secret put TWILIO_ACCOUNT_SID ``` ## Commerce Settings ```bash theme={null} # Enable apply mode npx wrangler secret put ALLOW_APPLY # Enter: true # Override default AI model npx wrangler secret put DEFAULT_MODEL # Enter: claude-opus-4-5 ``` ## AI Gateway (Optional) Route API requests through [Cloudflare AI Gateway](https://developers.cloudflare.com/ai-gateway/) for caching, rate limiting, and analytics: ```bash theme={null} npx wrangler secret put AI_GATEWAY_API_KEY npx wrangler secret put AI_GATEWAY_BASE_URL # Enter: https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/anthropic ``` AI Gateway variables take precedence over direct `ANTHROPIC_API_KEY` if both are set. ## Container Lifecycle By default, the container stays alive indefinitely. To sleep after inactivity (reduces cost, adds cold start latency): ```bash theme={null} npx wrangler secret put SANDBOX_SLEEP_AFTER # Enter: 10m (or 1h, 30m, etc.) ``` With R2 configured, data persists across restarts. ## Debug Endpoints Enable with `DEBUG_ROUTES=true` (requires Cloudflare Access): | Endpoint | Description | | ----------------------------- | ----------------------------------------------------- | | `GET /debug/version` | Container and stateset version info | | `GET /debug/processes` | All container processes (add `?logs=true` for output) | | `GET /debug/cli?cmd=...` | Run a CLI command in the container | | `GET /debug/logs?id=...` | Logs for a specific process | | `GET /debug/env` | Sanitized environment configuration | | `GET /debug/container-config` | Gateway config from inside the container | ## Local Development ```bash theme={null} cp .dev.vars.example .dev.vars # Edit .dev.vars with your ANTHROPIC_API_KEY npm install npm run dev ``` Set `DEV_MODE=true` in `.dev.vars` to skip Cloudflare Access auth. ## All Secrets Reference | Secret | Required | Description | | ------------------------ | -------- | -------------------------------------------------------- | | `ANTHROPIC_API_KEY` | Yes\* | Anthropic API key (or use AI Gateway) | | `AI_GATEWAY_API_KEY` | Yes\* | AI Gateway provider key (requires `AI_GATEWAY_BASE_URL`) | | `AI_GATEWAY_BASE_URL` | No | AI Gateway endpoint URL | | `OPENAI_API_KEY` | No | OpenAI API key (alternative provider) | | `CF_ACCESS_TEAM_DOMAIN` | Yes | Cloudflare Access team domain | | `CF_ACCESS_AUD` | Yes | Cloudflare Access application audience | | `STATESET_GATEWAY_TOKEN` | No | Gateway auth token | | `R2_ACCESS_KEY_ID` | No | R2 access key for persistent storage | | `R2_SECRET_ACCESS_KEY` | No | R2 secret key for persistent storage | | `CF_ACCOUNT_ID` | No | Cloudflare account ID (required for R2) | | `TELEGRAM_BOT_TOKEN` | No | Telegram channel | | `DISCORD_BOT_TOKEN` | No | Discord channel | | `SLACK_BOT_TOKEN` | No | Slack channel | | `SLACK_APP_TOKEN` | No | Slack channel (required with SLACK\_BOT\_TOKEN) | | `WHATSAPP_TOKEN` | No | WhatsApp channel | | `EMAIL_SMTP_HOST` | No | Email channel | | `TWILIO_ACCOUNT_SID` | No | SMS channel (Twilio) | | `ALLOW_APPLY` | No | Set to `true` to enable apply mode | | `DEFAULT_MODEL` | No | Override default AI model | | `DEV_MODE` | No | Set to `true` for local dev (skips auth) | | `DEBUG_ROUTES` | No | Set to `true` to enable `/debug/*` routes | | `SANDBOX_SLEEP_AFTER` | No | Container sleep timeout: `never` (default), `10m`, `1h` | \* One of `ANTHROPIC_API_KEY` or `AI_GATEWAY_API_KEY` is required. ## Troubleshooting **Container fails to start:** Check `npx wrangler tail` for logs. Verify `ANTHROPIC_API_KEY` is set. **R2 not mounting:** Ensure all three secrets are set (`R2_ACCESS_KEY_ID`, `R2_SECRET_ACCESS_KEY`, `CF_ACCOUNT_ID`). R2 mounting only works in production. **Access denied on admin routes:** Verify `CF_ACCESS_TEAM_DOMAIN` and `CF_ACCESS_AUD` are set correctly. **Slow first request:** Cold starts take 1-2 minutes. A loading page is shown automatically. **Config changes not working:** Update the Dockerfile cache bust comment and redeploy. ## Links * [StateSet](https://stateset.io/) * [Cloudflare Sandbox Docs](https://developers.cloudflare.com/sandbox/) * [Cloudflare Access Docs](https://developers.cloudflare.com/cloudflare-one/policies/access/) * [Cloudflare AI Gateway](https://developers.cloudflare.com/ai-gateway/) ## Next steps The section below, and the first thing to do. The same engine, run locally. Getting the engine working before deploying it. Other places the engine runs. # Complete an x402 payment Source: https://docs.stateset.com/stateset-x402-walkthrough A full x402 payment from HTTP 402 to a verified Merkle receipt — the exact wire format, signing preimage, and replay semantics. x402 turns HTTP 402 into a working payment protocol. You request a resource, the server tells you what it costs, you present a signed payment intent, and you get the resource plus a receipt you can verify against an on-chain batch commitment. This walks the whole loop against a real sequencer. ## Prerequisites * A payment-gated endpoint (the sequencer ships a demonstration premium route) * An Ed25519 signing key registered in the agent key registry * The sequencer base URL ## Step 1 — Request the resource, get a 402 ```bash theme={null} curl -i https://api.sequencer.stateset.com/api/v1/premium/resource ``` The requirements arrive in the **response body**, as a structured error whose `details` carry the payment terms: ```json theme={null} { "error": { "code": "PAYMENT_REQUIRED", "details": { "x402_version": 1, "asset": "usdc", "network": "set_chain", "chain_id": 84532001, "amount": 10000, "pay_to": "0x0000000000000000000000000000000000000402", "resource": "/api/v1/premium/resource", "max_validity_secs": 86400 } } } ``` Read the terms from the body, not from a header. `amount` is in the asset's **smallest unit** — `10000` is 0.01 USDC at 6 decimals, not 10,000 USDC. Getting this wrong by six orders of magnitude is the most common first mistake. ## Step 2 — Build and sign the intent The signing hash is SHA-256 over a **domain-separated, order-dependent** preimage. Every field is concatenated in exactly this order, integers as **big-endian u64**: ``` signing_hash = SHA256( "X402_PAYMENT_V1" // domain separator, ASCII || payer // ASCII address || payee // ASCII address || U64_BE(amount) || lowercase(asset) // "usdc" || network // "set_chain" || U64_BE(chain_id) || U64_BE(valid_until) // unix seconds || U64_BE(nonce) ) ``` Then sign those 32 bytes with Ed25519. ```js theme={null} import { createHash } from 'node:crypto'; const u64be = (n) => { const b = Buffer.alloc(8); b.writeBigUInt64BE(BigInt(n)); return b; }; function signingHash({ payer, payee, amount, asset, network, chainId, validUntil, nonce }) { return createHash('sha256') .update('X402_PAYMENT_V1') .update(payer) .update(payee) .update(u64be(amount)) .update(asset.toLowerCase()) .update(network) .update(u64be(chainId)) .update(u64be(validUntil)) .update(u64be(nonce)) .digest(); } ``` The order is part of the hash. Reordering any two fields, encoding an integer little-endian, or sending `USDC` where the preimage expects `usdc` all produce a different hash and the signature will be rejected. `valid_until` must be within `max_validity_secs` (86,400 — 24 hours) of now. ## Step 3 — Retry with `X-Payment` The `X-Payment` header carries **standard base64 of the JSON document** that `POST /api/v1/x402/payments` accepts — base64 because raw JSON is not safe in a header value. ```bash theme={null} INTENT='{ "tenant_id": "...", "store_id": "...", "agent_id": "...", "payer_address": "0x1234...", "payee_address": "0x0987...", "amount": 10000, "asset": "usdc", "network": "set_chain", "valid_until": 1705320000, "nonce": 42, "signing_hash": "0x...", "payer_signature": "0x..." }' curl -i https://api.sequencer.stateset.com/api/v1/premium/resource \ -H "X-Payment: $(printf '%s' "$INTENT" | base64 -w0)" ``` A valid intent returns **200** with the resource, plus a receipt header: ``` X-Payment-Receipt: base64({ "intent_id": "...", "status": "sequenced", "sequence_number": 8814, "sequenced_at": "2026-01-15T10:30:00Z", "receipt_url": "/api/v1/x402/payments//receipt" }) ``` A gated request and a direct submission to `POST /api/v1/x402/payments` run through the **same** verification and sequencing path. The header route is not a shortcut — the intent gets a sequence number, burns its nonce, and enters the normal batching pipeline either way. ## Replay protection, and one trap The intent is **consumed by the request it pays for**. * The nonce is reserved in a nonce-tracking table keyed by payer. Presenting the same signed intent twice fails the second time with `Nonce already used for this payer`. * Any `idempotency_key` in the decoded intent is **discarded** on the header path, deliberately: it is not part of the signed hash, so honouring it would let a replay short-circuit through the idempotency lookup and get the resource twice for one payment. Do not build retry logic that re-presents the same `X-Payment` value. A network timeout after the server sequenced your intent has already spent the nonce — retrying returns a nonce error, not the resource. Sign a fresh intent with a new nonce, and reconcile the first one via its `receipt_url`. ## Step 4 — Fetch and verify the receipt `receipt_url` becomes fully populated once the intent is **batched** — sequencing is immediate, batching is not. ```bash theme={null} curl https://api.sequencer.stateset.com/api/v1/x402/payments/$INTENT_ID/receipt ``` The receipt carries a Merkle inclusion proof. Verify it against the anchored batch commitment independently of the sequencer — that verification, including the domain-separated leaf and node hashing, is documented in [Set L2 verification](/set/stateset-set-l2-verification-example). Verifying the proof is what makes the receipt worth having. An unverified receipt is just the sequencer asserting it did its job; a verified one is arithmetic anybody can check against the chain. ## The full loop ``` GET /premium/resource └─▶ 402 body.details = payment requirements │ ├─ sign X402_PAYMENT_V1 preimage (Ed25519) ▼ GET /premium/resource X-Payment: base64(intent) └─▶ 200 resource + X-Payment-Receipt (sequenced, seq #) │ ├─ batch worker commits the batch, anchors the root ▼ GET /x402/payments/:id/receipt └─▶ Merkle inclusion proof ──▶ verify against on-chain commitment ``` ## Related * [Sequencer x402 reference](/stateset-sequencer/stateset-sequencer-x402) — every field and endpoint * [Set L2 verification example](/set/stateset-set-l2-verification-example) — proving inclusion * [Sequencer architecture](/stateset-sequencer-architecture) — batching and anchoring # Getting Help Source: https://docs.stateset.com/support Where to ask, what to check first, and how problems get resolved — Discord, a call with the team, host status, and the outcome dispute process. ## Check these first Diagnose MCP access, agent behavior, workflow trials, and missing replies step by step. Timestamped probe results. Compare the checked date with your request; this report is context rather than a live diagnosis. Every error envelope side by side, and the five families with what to do about each. The most common cause of a mystery 401 or 404 is the right key on the wrong engine. If a call that used to work stopped, the tab's history page says what changed in the surface. ## Ask a person * **[Discord](https://discord.gg/VfcaqgZywq)** — the fastest path for build questions; the team and other builders are in the channel. * **[Book time with the team](https://calendly.com/stateset)** — onboarding, architecture review, or anything commercial. * **Tell us when a page is wrong** — say so in Discord with the URL; we treat "this page is wrong" as a bug report. Include the failing step and observed result so we can check the guide or underlying service contract. ## Report an API problem Include the details below so the team can reproduce the failure. Remove API keys, authorization headers and customer data before sharing a request or response. ```text theme={null} Service: ResponseCX / Sync / Voice / other Documentation page: URL of the example or endpoint Request: HTTP method and URL path Time: timestamp with timezone HTTP status: response status code Error code and request ID: if supplied by the service Expected result: what you expected to happen Actual result: redacted response body Reproduction: smallest request that still fails ``` If a documentation example fails, include the step number and any changes you made to it. For a timeout, say whether a later read showed that the operation completed. The [error guide](/api-reference/errors-across-engines) shows where each service puts its error code and request identifier. ## When money is involved Billing follows the outcome model: outcomes can be **disputed within 14 days**, a dispute lands as a credit on your next invoice, and the ledger is visible in the billing portal — the process and the High-Value Action gate are documented in [Billing](/stateset-billing). When you report an API problem, include the request id from the response (`request_id`, `meta.requestId`, or the correlation header, depending on the engine — the [errors page](/api-reference/errors-across-engines) says which) and the timestamp. That turns a mystery into a lookup. # Why StateSet Source: https://docs.stateset.com/why-stateset What we built, how it differs from payment APIs, iPaaS, helpdesk AI and building it yourself, what it costs, and honest answers to the hard questions. ## The shift AI reasoning, agent protocols and buyer readiness have all advanced. Models are already good enough to decide what *should* happen. The remaining bottleneck is **trusted execution across real commerce systems**: an agent that concludes "this customer deserves a refund" still has to find the order, check the policy, issue the payment, update inventory, and close the ticket — through interfaces designed for a human with a mouse, in systems that must stay consistent with each other afterwards. StateSet is the iCommerce company, and the iCommerce Engine is the layer that closes that gap: an agent-native commerce execution runtime and control layer that sits above and across the systems you already run. ## How we're different StateSet is not only an AI interaction layer or a workflow connector. It combines commerce semantics, deterministic execution, policies, integrations, auditability and outcome measurement — and each comparison below is really about one of those. ### Against payment and storefront APIs: embedded, not remote They give you endpoints over the network. **iCommerce is an engine you compile into your runtime.** | | API-first (Stripe, Shopify) | iCommerce | | ---------------- | ------------------------------ | ---------------------------- | | Where it runs | Their servers | **Your process** | | Latency per call | Network round-trip | In-memory | | Works offline | No | **Yes** | | Type safety | Whatever your client generates | Native, across 10 bindings | | Scope | Payments, or a storefront | Full lifecycle state machine | Stripe handles *payments*. It does not handle orders, inventory, tax, shipping, returns, or the state transitions between them. iCommerce holds that entire state machine locally — including a double-entry general ledger, AP with three-way match, AR with dunning, fixed-asset depreciation, and ASC 606 revenue recognition. The analogy we use is **SQLite for commerce**. Not a service you integrate with. A library you embed. ### Against iPaaS: commerce semantics, not only orchestration Zapier, Workato and their peers are excellent integration and orchestration tools, and the Engine is happy to sit beside them. What they do not carry is the commerce model: what an order *is*, which transitions are legal, what a refund does to the ledger, when an action needs a human. StateSet adds that — commerce semantics, policy, autonomous decisioning and outcome accountability — on top of the plumbing. ### Against your ERP, OMS and WMS: above and across, not instead of StateSet is not another ERP. Your systems of record stay where they are; the Engine makes them safely operable by agents — the [Sync Server](/stateset-sync-server-api-basics) and [EDI](/stateset-edi/overview) keep them agreeing with each other, and every write is validated against the commerce model before it lands. ### Against RPA: adaptive, not brittle RPA scripts encode pixel positions and DOM paths. Move a button and the bot breaks, silently, at 3am. Our [Computer Use agents](/computer-use-agent) perceive the screen and reason about it, so a redesigned page is something they adapt to rather than fail on. They also run on three interchangeable engines — Claude, OpenAI, or a local zero-API engine — so you're not locked to one vendor's availability or pricing. ### Against helpdesk AI: close the operation, not just the conversation Gorgias, Intercom and Sierra-style agents automate the interaction and a selected set of actions, and they do it well. A support ticket, though, is usually a symptom of an order, subscription or fulfillment problem underneath it. StateSet is built to execute — and maintain correctness across — that underlying operation: it looks up the order, applies your policy, issues the refund, updates every record that depends on it, and replies. The interaction is resolved *and* the operational fix behind it is done, so closing the ticket does not create downstream work. This is also the only honest basis for outcome pricing. You cannot bill per resolution if a person still has to finish every one. ## How it adds value ### You pay for work completed, not capacity provisioned Traditional software bills for seats or tokens whether it works or not. StateSet bills per **verified outcome**, priced against the BPO labor it replaces. | Live today | Price | | ------------------------------------------------------ | ------ | | Triage outcome — enquiry triaged and handed to a human | \$0.50 | | Resolved customer contact — resolved end to end | \$2.00 | Plans pair an optional platform fee with a per-outcome rate; a larger fee buys a lower rate. See [Billing](/stateset-billing) for the current catalog and plans. Outcomes can be **disputed within 14 days**. A dispute becomes a Stripe customer balance transaction, so a credit lands on your next invoice, and the Outcome Accounting Dashboard shows the full ledger. If the work wasn't done, you don't pay for it. ### Consequential actions are provable This is the part we'd most want a skeptical buyer to test. When an agent proposes something that moves money or changes an order, the action is routed through [NSR](/stateset-nsr-decisions), which returns `approved`, `denied`, or `refused` **with the cited rules that produced the verdict** and a SHA-256 pin to the exact policy inputs it was evaluated against. Two properties are enforced in code, not policy: * **Grounded-only approvals.** An `approved` verdict only executes when its proof is grounded. A confident-but-ungrounded approval escalates to a human instead. * **Fail-closed.** A `refused` verdict and an unreachable policy engine both escalate. An outage degrades into human review, never into unauthorized automation. **High-Value Actions** — by default anything at or above \$100 — require human approval before execution, configured per brand. ### The audit trail is the execution Automated work runs as [Temporal workflows](/next-temporal/overview). The verdict and proof chain that authorized an action live permanently in the workflow's event history, and replays reuse them. There is no separate audit log that can drift from what actually happened. For cross-party verification, [Sequencer](/stateset-sequencer/stateset-sequencer) events are Ed25519-signed and [ICP](/stateset-icp) receipts are JWS over a canonicalized body — verifiable offline by any party, with no callback to us. ### It reaches the systems you actually run An agent is only as useful as what it can touch: the commerce platforms and 3PL/WMS systems a brand runs on, through [Sync](/stateset-sync-server-api-basics), X12 and EDIFACT via [EDI](/stateset-edi/overview), phone through [Voice](/stateset-voice/overview), email through [Mail](/stateset-mail/overview) — and for systems with no API at all, Computer Use drives the UI. ## The hard questions **"Isn't this just automation?"** Automation executes predefined steps. The Engine combines an agent's reasoning with deterministic commerce execution, policy controls and operational state — it decides what to do within the bounds you set, and the bounds are enforced in code. **"AI writes are too risky."** That is the right instinct, and the Engine is designed around it: simulation, permissions, invariants, approval paths, auditability and fail-closed execution. The decision gate fails closed, ungrounded approvals escalate, and anything above your value threshold needs a human. You set where the line is. And when an outcome is wrong, you dispute it and don't pay. **"AI makes mistakes. I trust my BPO team."** Humans make mistakes too, usually from fatigue and turnover. The difference is that agents follow your written policy every time. And StateSet is not a replacement for a BPO: it removes the repetitive labor while giving a BPO a higher-leverage delivery model for QA, exceptions and managed operations. **"Why not just use the Stripe API?"** Stripe handles payments, not commerce. You still have to manage orders, inventory, tax, shipping, and returns state yourself. iCommerce handles that full lifecycle locally — and the finance layer underneath it. **"Is this proprietary? What's the lock-in?"** The core engine is open source, dual-licensed **MIT OR Apache-2.0**. Inspect it, fork it, run it anywhere. We charge for enterprise features and hosted infrastructure. Because the engine is embedded and the data is a SQLite or Postgres database you control, you can [export or back it up](/stateset-icommerce/stateset-icommerce-backup-restore) at any time. **"We already have an ERP."** Good. StateSet is not another ERP; it gives agents a governed way to operate your existing systems of record. **"Our helpdesk already has AI."** Keep it. StateSet can become the execution layer that safely performs the underlying commerce work across systems — it runs behind Gorgias or Intercom as the thing that actually does the work. **"Why not build this ourselves?"** The hard part is not an LLM call. It is years of commerce semantics, integrations, safe state transitions, policy and operational edge cases — and that is what the Engine is. **"Does this require a proprietary protocol?"** No. Every service speaks REST, most expose an MCP server and a skill, and the Engine interoperates with the protocols and agents you already use. **"How do I know the agent did what it says?"** Every automated action carries a cited proof chain in the workflow's permanent event history. That's a stronger claim than most vendors make, and it's the one we'd invite you to audit first. ## Where to start Embed the engine. Open source, 10 bindings, 938 agent-callable tools. Deploy outcome-priced agents against your existing stack. The verified-decision contract, in detail. MCP servers for every service on the platform.