Skip to main content
Your agent already produces conversation logs. StateSet Agents treats them as a training set: it grades every turn against a reward, curates the ones that worked, trains on those, and evaluates the result against prompts the model never saw. The framework is stateset-agents on PyPI. Multi-turn generation exists in several trainer libraries now; what this keeps coherent is the whole loop — traces, rewards, training, and a deployed model with lineage back to the conversations that produced it.
Earlier revisions of these docs called this framework grpo-agent-framework, which is not a published package. The install is pip install stateset-agents.

​
The loop

Each step is a command, and each writes a file the next one reads — so you can stop after any of them, inspect what happened, and resume. The whole loop, end to end:
next_steps.md is worth reading rather than skipping: the improve step prints the exact training command for what it just curated, so step 3 is a copy rather than a decision.

​
Rewards

Grading is rule-based and offline. Four rewards ship:
There are deliberately no LLM-judge rewards in this list. A judge needs an API key and a network call per turn, which makes grading non-reproducible and turns a local loop into a billed one. Judge-based evaluation exists in the framework for benchmarking; it is not part of the offline improve loop.

​
Training backends

The framework does not implement its own trainer. It drives version-pinned backends behind a fail-closed adapter — PPO, GRPO and GSPO through OpenRLHF, PPO/GRPO through verl, and distributed maths GRPO through NeMo RL.
Every backend adapter is fail-closed: a version it has not been conformance-tested against refuses to run rather than proceeding on the assumption that the interface is unchanged. Live GPU conformance is an explicit evidence gate, not something an upgrade inherits. If a backend refuses after you bump a dependency, that is the gate doing its job.

​
Next steps

Quickstart

Logs to a retrained model, with no GPU of your own.

CLI reference

Every command, and which ones cost money.

MCP server

Drive the improve loop from Claude Code or your own agent.

Agent objectives

Choosing what to reward before you train on it.