Skip to main content
The framework package is stateset-agents, with canonical imports under stateset_agents. The earlier name grpo-agent-framework is not the installation path. For the current source scope and setup, start with StateSet Agents.

​
Trainer choices

These are different trainers and configs, not interchangeable one-line substitutions. The studio intentionally exposes only SFT, GRPO, GSPO, and DAPO locally. River’s native CISPO workflow uses its own rollout engine and trainer. The reviewed source imports include:
Importing training modules requires their optional dependencies. These imports identify surfaces; they do not start a training run.

​
Group-relative objectives

Sample several responses or trajectories for the same task. Their rewards define relative advantages; the selected objective controls normalization, ratios, clipping, and reference regularization. A simplified centered example is:
This is illustrative centering, not a complete trainer configuration. Standardization and zero-variance handling follow the configured estimator. GRPO evaluates token-level policy ratios; GSPO derives sequence-level ratios from generated-token log probabilities and response lengths. Neither objective guarantees stability or improvement on every task.

​
Rollout and update integrity

The current code validates sample shape, token IDs, masks, finite rewards/likelihoods, and complete group ownership before learning. Required sampled vLLM token log probabilities must be present exactly; missing values are not replaced with invented likelihoods. PPO’s online path retains actual sampled completions and critic values, rather than fabricated text. Retain rollout receipts and committed update counts separately from sampled trajectories. Partial learner updates or failed late checkpoint restores can leave state quarantined until a complete reload succeeds. An exception must not be presented as successful training.

​
Use the shared control plane

For local RL, select GRPO/GSPO/DAPO and a support-scenario dataset. The studio records the training seed, model revision when supplied, generations, positive KL coefficient, LoRA settings, and update budget. The plan is reviewed before launch; exact retries preserve its run ID and checksum. CUDA and the training dependencies are required for execution. See the native RL data format and structured launch controls.

​
External engines and remote providers

A remote provider allocates or operates compute; an external engine implements a training backend. These are separate choices. StateSet has its own native trainers and also has version-bound adapters for OpenRLHF, verl, and NeMo RL. External backend requests bind algorithm, model/data revisions, seed, environment, reward, capabilities, and configuration into a canonical experiment digest. Results must retain that identity and valid artifacts/finite metrics. Unsupported versions or capabilities are rejected. Live GPU conformance is still pending in the reviewed backend documentation; contract tests do not prove throughput or numerical equivalence.

​
Recovery and selection

Use worker-published full checkpoints for exact continuation. Explicit resume requests must not silently start fresh. Some dedicated convenience loops reject resume before setup; use the documented control-plane recovery path when supported. Native DDP recovery has its own versioned package/model/runtime identity requirements. Validation and fresh test are separate from training completion. A final checkpoint is not automatically the best checkpoint, and a recommended validation checkpoint is not production-approved.

​
Source and next step

Checked against the October 20, 2026 local source: training/gspo_trainer.py, the native trainer/runtime modules, docs/TRAINING_BACKENDS.md, and contracts/component_maturity_v1.json. This page replaces obsolete examples using helpers that the package does not export. Run the fine-tuning quickstart or read the wider RL training and evidence workflow.
Last modified on October 20, 2026