> ## Documentation Index
> Fetch the complete documentation index at: https://docs.stateset.com/llms.txt
> Use this file to discover all available pages before exploring further.
# StateSet Agents
> The RL improvement control plane: turn an agent's production conversations into a training set, and prove the retrained model got better.
Your agent already produces conversation logs. StateSet Agents treats them as a
training set: it grades every turn against a reward, curates the ones that
worked, trains on those, and evaluates the result against prompts the model
never saw.
The framework is `stateset-agents` on PyPI. Multi-turn generation exists in
several trainer libraries now; what this keeps coherent is the **whole loop** —
traces, rewards, training, and a deployed model with lineage back to the
conversations that produced it.
Earlier revisions of these docs called this framework `grpo-agent-framework`,
which is not a published package. The install is `pip install stateset-agents`.
## The loop
```text theme={null}
production logs ──▶ ingest ──▶ grade ──▶ curate ──▶ train ──▶ evaluate
▲ │
└──────────────── deploy with lineage ─────────────────────┘
```
Each step is a command, and each writes a file the next one reads — so you can
stop after any of them, inspect what happened, and resume.
| Step | Command | Produces |
| ---------------- | ----------------------------------------- | -------------------------------------------------------- |
| Ingest | `stateset-agents ingest` | Per-conversation transcript JSONL |
| Grade and curate | `stateset-agents improve run` | `curated.jsonl`, `improve_summary.json`, `next_steps.md` |
| Train | `stateset-agents train` or `train-remote` | An adapter, plus `eval_results.json` |
| Talk to it | `stateset-agents chat` / `chat-remote` | An interactive session against the result |
The whole loop, end to end:
```bash theme={null}
pip install stateset-agents
stateset-agents ingest --format openai --input logs.jsonl --output transcripts/
stateset-agents improve run --transcripts transcripts/ --reward customer_support --output improved/
stateset-agents train-remote --provider runpod --dataset improved/curated.jsonl \
--base-model meta-models/Muse-Glimmer-30B --eval-prompts held_out.txt --max-cost 5
```
`next_steps.md` is worth reading rather than skipping: the improve step prints
the exact training command for what it just curated, so step 3 is a copy rather
than a decision.
## Rewards
Grading is rule-based and offline. Four rewards ship:
| Reward | Grades |
| ------------------ | --------------------------------------------------------------------------------------- |
| `customer_support` | Support conversations — resolution, tone, escalation behaviour |
| `tool_calling` | Whether the right tool was called with the right arguments |
| `gsm8k` | Grade-school maths, as a correctness baseline |
| `nsr` | Decisions against a symbolic policy — see [Verified decisions](/stateset-nsr-decisions) |
There are deliberately no LLM-judge rewards in this list. A judge needs an API
key and a network call per turn, which makes grading non-reproducible and
turns a local loop into a billed one. Judge-based evaluation exists in the
framework for benchmarking; it is not part of the offline improve loop.
## Training backends
The framework does not implement its own trainer. It drives version-pinned
backends behind a fail-closed adapter — PPO, GRPO and GSPO through OpenRLHF,
PPO/GRPO through verl, and distributed maths GRPO through NeMo RL.
Every backend adapter is **fail-closed**: a version it has not been
conformance-tested against refuses to run rather than proceeding on the
assumption that the interface is unchanged. Live GPU conformance is an
explicit evidence gate, not something an upgrade inherits. If a backend
refuses after you bump a dependency, that is the gate doing its job.
## Next steps
Logs to a retrained model, with no GPU of your own.
Every command, and which ones cost money.
Drive the improve loop from Claude Code or your own agent.
Choosing what to reward before you train on it.