> ## Documentation Index
> Fetch the complete documentation index at: https://docs.stateset.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Run an agent's eval suite (regression check)

> Each eval costs a real generation plus a judge call, so runs are sequential and capped per call; has_more reports when the suite was longer than the cap.

Each eval costs a real generation plus a judge call, so runs are sequential and capped per call; has\_more reports when the suite was longer than the cap.

### Path parameters

**id** `string (uuid)` *required*

The agent id.

### Request body

`object`

**limit** `integer`

How many of the agent's evals to run (newest first). Capped at 20 per call. Default: `10`. Minimum: `1`. Maximum: `20`.

**status** `string`

Only run evals with this `status`. Maximum length: `50`.

### Response

`object`

**object** `string` *required*

Allowed values: `'eval.run_summary'`.

**agent\_id** `string (uuid)` *required*

**total** `integer` *required*

Number of evals run (`runs.length`).

**passed** `integer` *required*

Runs with `passed: true`.

**failed** `integer` *required*

Runs with `passed: false`.

**ungraded** `integer` *required*

Runs with `passed: null`.

**has\_more** `boolean` *required*

True when more evals exist for the agent than were run — the suite stopped at `limit`.

**note** `string`

Present only when `has_more` is true: states that only the first N were run.

**runs** `object[]` *required*

**runs**

**object** `string` *required*

Allowed values: `'eval.run'`.

**eval\_id** `string (uuid)` *required*

**agent\_id** `string (uuid)` *required*

The agent the eval was replayed against (the request `agent_id`, else the eval's own).

**user\_message** `string,null`

The replayed customer message. Absent when the run itself failed (`reason: 'run_failed'`).

**expected** `string,null`

The eval's `preferred_output`. Absent when the run itself failed.

**actual** `string,null`

The agent's reply on this run; null when the agent returned no reply. Absent when the run itself failed.

**passed** `boolean,null` *required*

True when `verdict` is `pass`, false when it is `partial` or `fail`, and null when the reply was not graded — 'the judge was unavailable' and 'the agent got it wrong' are kept distinct.

**graded** `boolean` *required*

Whether the judge produced a verdict. When false, `reason` says why and the verdict fields are absent.

**reason** `string`

Present only when `graded` is false. Allowed values: `'judge_unavailable'`, `'judge_failed'`, `'judge_empty'`, `'judge_invalid_json'`, `'judge_invalid_verdict'`, `'agent_returned_no_reply'`, `'run_failed'`.

**error** `string`

Present only when `reason` is `run_failed` (a run inside a suite threw); the public error message.

**duration\_ms** `integer`

Generation plus judge time. Absent when the run itself failed.

**verdict** `string`

Present only when `graded` is true. Judged on substance, not wording. Allowed values: `'pass'`, `'partial'`, `'fail'`.

**equivalence** `number,null`

Present only when `graded` is true. 0–5, how closely the substance matches (5 = equivalent); one decimal. Minimum: `0`. Maximum: `5`.

**contradicts** `boolean,null`

Present only when `graded` is true.

**omits** `string[]`

Present only when `graded` is true. Essential points from the reference that the reply is missing. Maximum items: `10`.

**differences** `string,null`

Present only when `graded` is true. Maximum length: `2000`.

**explanation** `string,null`

Present only when `graded` is true. The judge's reasoning, so a caller can disagree with the grade. Maximum length: `2000`.

### Status codes

| Code | Meaning |
| - | - |
| `200` | The suite result. Evals run sequentially; a run that throws is recorded as an ungraded `eval.run` with `reason: 'run_failed'` rather than failing the whole suite. |
| `400` | Invalid request. `issues` names the offending field(s). |
| `401` | Missing, invalid, or expired API key. |
| `403` | The key lacks a required scope. |
| `404` | Not found, or not in the organization that owns the key. |
| `429` | Rate limit exceeded. |

### Access

| | |
| - | - |
| Required scope | `responses:write` |
| Rate limit | 30 requests per minute |

### Using this contract

Read the [source OpenAPI document](https://docs.stateset.com/spec/responsecx.json) for declared schemas and alternatives.
This page also includes documented corrections from the [spec overlays](https://github.com/stateset/docs/tree/main/spec/overlays).
Example IDs and values are illustrative; replace them with records from your workspace.

```bash cURL theme={null}
curl --silent --show-error --fail-with-body --request POST \
  --url 'https://response.stateset.com/api/v1/agents/{id}/evals/run' \
  --header "Authorization: Bearer $RESPONSECX_API_KEY" \
  --header 'Content-Type: application/json' \
  --data '{
  "limit": 10,
  "status": "pending"
}'
```

```json 200 theme={null}
{
  "object": "eval.run_summary",
  "agent_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "total": 102,
  "passed": 1,
  "failed": 1,
  "ungraded": 1,
  "has_more": true,
  "note": "Two-person tent, green — replacement for damaged pole set.",
  "runs": [
    {
      "object": "eval.run",
      "eval_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
      "agent_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
      "user_message": "string",
      "expected": "string",
      "actual": "string",
      "passed": true,
      "graded": true,
      "reason": "judge_unavailable",
      "error": "string",
      "duration_ms": 8,
      "verdict": "pass",
      "equivalence": 1.5,
      "contradicts": true,
      "omits": [
        "string"
      ],
      "differences": "string",
      "explanation": "string"
    }
  ]
}
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.