> ## Documentation Index > Fetch the complete documentation index at: https://docs.stateset.com/llms.txt > Use this file to discover all available pages before exploring further. # Your First Agent Evaluation > Capture an approved correction, replay it against a Response agent, and verify a behavior change with MCP tools.
WORK INResponse MCP
BRING

Agent and response IDs · Approved correction · Generation access

FINISH WITH

A saved case and reviewed before/after results

**Outcome:** one saved evaluation and an observed before/after result for an agent change. This walkthrough uses Response MCP. It does not publish an agent or send a customer message. An evaluation captures what a good answer should accomplish. Running it generates a new reply and grades its substance against that reference. Saving an evaluation alone does not change instructions, update knowledge, or train a model. ## Prerequisites * A [Response MCP connection](/guides/manage-platform-with-mcp) to the intended organization. * An existing agent and a real response you are authorized to review. * A corrected answer approved by someone who knows the applicable policy. * Read access to responses/evaluations and `responses:write` for creation and evaluation runs. Evaluation runs use the real generation path and consume usage. Choose one case first; do not start by replaying your entire history. ## 1. Inspect the original case Discover the current tool schemas. Use `get_response` to inspect the response and confirm its agent and organization before constructing a correction. Preserve the response UUID and agent UUID; do not use the conversation ID or a workflow run ID in their place. Ask the reviewer to explain which fact, policy, or action was wrong and provide the desired answer. The reference should match the evidence available to the agent. Do not invent an order status or a refund commitment merely to make a more helpful-looking expected answer.
**Success:** you have a specific response, a policy-backed correction, and an identified agent.
## 2. Check whether the case already exists Use `list_evals` filtered by the response ID. These blocks are **MCP tool arguments**, not REST request bodies. Replace each angle-bracket placeholder with your actual value before calling. For `list_evals`: ```json theme={null} { "response_id": "", "limit": 25, "offset": 0 } ``` Inspect the returned cases and follow pagination if needed. If an existing case already records the approved correction, retain its ID rather than creating another copy. ## 3. Save the approved correction For a new case, call `create_eval` with the actual response ID and the reviewer's correction: ```json theme={null} { "response_id": "", "agent_id": "", "name": "Shipping policy: avoid an unsupported arrival promise", "description": "The original answer promised a date not supported by our shipping policy.", "preferred_output": "", "reason_type": "wrong_policy", "idempotency_key": "" } ``` The server reads the original message and reply from the stored response. Do not retype them into `user_message` or `non_preferred_output` when using `response_id`. A hypothetical case can instead supply `user_message`, but keep it distinguishable from a real customer case. Retain the same idempotency key and input for a retry of this creation attempt. A different correction is not the same request. Read the new case using `get_eval` with its returned `id` and verify its reference answer and agent association.
**Success:** a saved evaluation ID with the intended original question and approved reference.
## 4. Run the baseline Call `run_eval`: ```json theme={null} { "eval_id": "", "agent_id": "" } ``` Inspect the returned `eval_id`, `agent_id`, `user_message`, `expected`, `actual`, and grading fields. The implementation returns the run result directly; save the result for comparison rather than assuming a separate persisted evaluation-run history exists. | Result | What it establishes | Next step | | ------------------------------- | ---------------------------------------------------------------------------- | -------------------------------------------------------------------------- | | `graded: true`, `passed: true` | The generated reply passed the grader's substantive comparison for this case | Inspect the reply and retain the baseline; this case may already pass | | `graded: true`, `passed: false` | The generated reply failed that comparison | Inspect the actual reply and grading explanation before changing the agent | | `graded: false`, `passed: null` | The case was not graded | Inspect `reason`; this is neither a pass nor a failure of agent quality | | Tool/request error | The requested run did not return a usable result | Resolve access, input, or service errors before another run | Ungraded reasons can include an unavailable judge, a failed judge request, invalid judge output, or an agent returning no reply. Do not rewrite the reference answer to hide these failures. ## 5. Change the behavior, then rerun Find the cause before editing: missing source material, incorrect instructions, an ineffective rule, or missing access to the information needed for the answer. Use the app or an authorized MCP mutation to correct that specific cause. Inspect the saved change and `list_changes`, then call `run_eval` for the same case and agent. Keep the before/after outputs and relevant change identifiers. If the agent or knowledge is already serving customers, understand that editing it can affect live behavior; this walkthrough does not create an isolated agent copy automatically. A new pass supports this case under the tested configuration. It does not establish that every intent works or that all future generations will match the same wording. Review neighboring cases, such as a missing order identifier and a request for an exception, before expanding use. ## 6. Check a small suite Once you have several relevant cases, call `run_evals` with an explicit limit: ```json theme={null} { "agent_id": "", "limit": 10 } ``` The current implementation runs sequentially, defaults to ten cases, and caps a call at twenty. Inspect `passed`, `failed`, `ungraded`, individual run results, and `has_more`. These counts cover the returned subset, not necessarily every evaluation belonging to the agent. If `has_more` is true, enumerate the remaining case IDs through `list_evals` using its pagination and call `run_eval` for those you need. Repeating `run_evals` with the same limit does not advance a cursor. Record how many cases were checked and which remain untested. ## Let your assistant perform the walkthrough ```text theme={null} Review response [response UUID] for agent [agent UUID] in my Response workspace. Use this approved correction: [corrected reply and policy reason]. Check for an existing evaluation first. If none matches, I authorize creating one with a stable idempotency key. Read it back and run it once as a baseline. Report the saved case ID, actual reply, expected reply, and grading result. If it fails, propose a targeted change for my review. If it is ungraded, diagnose that instead. Do not change the agent, publish it, send a customer message, or launch more evaluations without further authorization. ``` ## Keep evaluation systems distinct Response evaluations belong to the Response agent/response system. Workstream golden cases belong to workflow brands and are evaluated through the engine's configuration tools. Creating a Response case does not satisfy Workstream's `require_golden_pass` gate automatically. Use the [first-week guide](/guides/first-week-with-stateset) to combine evaluations, reviewed live outcomes, and measured improvement without treating them as the same evidence. ## Next steps Compare your evaluation evidence with real customer outcomes. Trace missing knowledge, incorrect behavior, or blocked tests. Related: [Response MCP reference](/stateset-response/response-mcp).