> ## Documentation Index
> Fetch the complete documentation index at: https://docs.stateset.com/llms.txt
> Use this file to discover all available pages before exploring further.
# StateSet Agents
> The RL improvement control plane: turn an agent's production conversations into a training set, and prove the retrained model got better.
Your agent already produces conversation logs. StateSet Agents treats them as a
training set: it grades every turn against a reward, curates the ones that
worked, trains on those, and evaluates the result against prompts the model
never saw.
The framework is `stateset-agents` on PyPI. Multi-turn generation exists in
several trainer libraries now; what this keeps coherent is the **whole loop** —
traces, rewards, training, and a deployed model with lineage back to the
conversations that produced it.
Earlier revisions of these docs called this framework `grpo-agent-framework`,
which is not a published package. The install is `pip install stateset-agents`.
## The loop
```text theme={null}
production logs ──▶ ingest ──▶ grade ──▶ curate ──▶ train ──▶ evaluate
▲ │
└──────────────── deploy with lineage ─────────────────────┘
```
Each step is a command, and each writes a file the next one reads — so you can
stop after any of them, inspect what happened, and resume.
| Step | Command | Produces |
| - | - | - |
| Ingest | `stateset-agents ingest` | Per-conversation transcript JSONL |
| Grade and curate | `stateset-agents improve run` | `curated.jsonl`, `improve_summary.json`, `next_steps.md` |
| Train | `stateset-agents train` or `train-remote` | An adapter, plus `eval_results.json` |
| Talk to it | `stateset-agents chat` / `chat-remote` | An interactive session against the result |
The whole loop, end to end:
```bash theme={null}
pip install stateset-agents
stateset-agents ingest --format openai --input logs.jsonl --output transcripts/
stateset-agents improve run --transcripts transcripts/ --reward customer_support --output improved/
stateset-agents train-remote --provider runpod --dataset improved/curated.jsonl \
--base-model meta-models/Muse-Glimmer-30B --eval-prompts held_out.txt --max-cost 5
```
`next_steps.md` is worth reading rather than skipping: the improve step prints
the exact training command for what it just curated, so step 3 is a copy rather
than a decision.
## Rewards
Grading is rule-based and offline. Four rewards ship:
| Reward | Grades |
| - | - |
| `customer_support` | Support conversations — resolution, tone, escalation behaviour |
| `tool_calling` | Whether the right tool was called with the right arguments |
| `gsm8k` | Grade-school maths, as a correctness baseline |
| `nsr` | Decisions against a symbolic policy — see [Verified decisions](/stateset-nsr-decisions) |
There are deliberately no LLM-judge rewards in this list. A judge needs an API
key and a network call per turn, which makes grading non-reproducible and
turns a local loop into a billed one. Judge-based evaluation exists in the
framework for benchmarking; it is not part of the offline improve loop.
## Algorithms
GRPO is the default and the one the [quickstart](/stateset-agents/quickstart) uses. Six other
objectives ship as trainers in `stateset_agents.training`, each with its own config and the same
reward and environment interfaces, so switching is a one-line change:
| Trainer | Import | Reach for it when |
| - | - | - |
| `GRPOTrainer`, `MultiTurnGRPOTrainer` | `stateset_agents` | The default — group-relative advantages, no critic |
| `GSPOTrainer` | `stateset_agents.training.gspo_trainer` | You want GRPO's stability at the sequence level; `GSPOConfig` and `train_with_gspo` come with it |
| `GEPOTrainer` | `stateset_agents.training.gepo_trainer` | Heterogeneous or distributed groups — expectation-based rather than sample-based |
| `DAPOTrainer` | `stateset_agents.training.dapo_trainer` | Reasoning-heavy tasks — decoupled clipping and dynamic sampling |
| `VAPOTrainer` | `stateset_agents.training.vapo_trainer` | Math and reasoning — value-augmented group optimization; fits a 16 GB card as of 0.42.5 |
| `PPOTrainer` | `stateset_agents.training.ppo_trainer` | A standard PPO baseline to compare against |
| `RLAIFTrainer` | `stateset_agents.training.rlaif_trainer` | Reward from a judge model rather than a hand-written function |
All seven run under one measured protocol: the 0.42.5 release reports a 15-run matrix across
five of them on matched hardware, and 0.42.4 adds framework-neutral comparison gates against TRL
and measured 1/2/4/8-GPU weak and strong scaling. The benchmark harness that produced those numbers
ships in the repository, so a claim about one algorithm beating another is reproducible rather
than quoted.
## Training backends
The framework does not implement its own trainer. It drives version-pinned
backends behind a fail-closed adapter — PPO, GRPO and GSPO through OpenRLHF,
PPO/GRPO through verl, and distributed maths GRPO through NeMo RL.
Every backend adapter is **fail-closed**: a version it has not been
conformance-tested against refuses to run rather than proceeding on the
assumption that the interface is unchanged. Live GPU conformance is an
explicit evidence gate, not something an upgrade inherits. If a backend
refuses after you bump a dependency, that is the gate doing its job.
## Next steps
Logs to a retrained model, with no GPU of your own.
Every command, and which ones cost money.
The REST gateway: agents, conversations, training jobs, and Anthropic/OpenAI-compatible inference.
Drive the improve loop from Claude Code or your own agent.
Choosing what to reward before you train on it.
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.