Keep four outcomes separate: training completed, learner state can resume, a checkpoint
performed well on validation, and a candidate passed an independent test protocol.
The studio and
agent CLI retain different reviews for each.
Recover exact training state
Discovery hashes/checks worker-published checkpoints without loading optimizer state.
Retain the chosen checkpoint, checkpoint_sha256, recovery_sha256, saved progress, copy
size, and remaining budget. Unavailable choices are explicit; model-only exports are not
full recovery checkpoints.
Preparation creates a new private child run, copying the retained dataset and complete
checkpoint. It does not alter the parent or launch. Review the returned child ID and its
new plan_sha256, then use the normal separate finetune launch protocol.
Recovery requires Linux, one local training process, a terminal parent with stopped worker
and supervisor, identical framework code, a pinned nonzero full model commit, complete
learner state, and a worker-published plan-bound receipt. SFT restores its exact trainer,
optimizer, scheduler, and RNG checkpoint; native RL restores its saved cursor/runtime state.
Changed bytes or settings fail instead of starting fresh.
The original total epochs/updates stay fixed; an exhausted checkpoint cannot resume.
Inspection is bounded to 100 receipts, 4,096 files per checkpoint, and 1 TiB of bytes.
Copying needs the checkpoint, dataset, and 64 MiB overhead in addition to launch resources.
Remote recovery is provider-specific. The studio’s c reuses settings for new training;
z performs this exact recovery flow.
Run a local paired evaluation
The final-artifact path requires a successful local run, a pinned Hub base model, and a
safetensors adapter. Baseline, candidate, and pinned independent judge must already be
cached. Loading is cache-only with native model code; missing weights fail rather than
download or substitute a stub.
Create an evaluation configuration using real paths and a cached judge commit:
Cases are schema-version-1 JSON with the declared test or validation split,
reviewed provenance, reviewer, matching freshness flag, and support cases containing
id, context, user_turns, policy_facts, expected_resolution, and must_avoid.
Do not mark synthetic documentation examples as independently reviewed evidence.
All commands accept --workspace. Planning reads and hashes inputs; preparation snapshots
them; launch separately submits queued inference. Retain IDs across disconnects and inspect
uncertain submission instead of automatically retrying work. Buildr bounds the job’s
CPU/RAM/scratch/time; those reservations do not reserve a GPU.
Baseline, candidate, and judge run sequentially in separate processes. Policies see fixed
user turns and supplied facts with greedy decoding. The judge scores each transcript twice
in reversed checklist order. Parsing failures or disagreement above 0.2 reject the evaluation.
Success requires score at least 0.8 plus every resolution, accuracy, policy, and
non-repetition check in both passes, with no token-limit truncation.
This is a one-run fixed-dialogue diagnostic, not adaptive-customer or tool-execution testing.
Exact overlap checks do not establish semantic independence. Raw cases, transcripts, and
judge recordings remain private under the run’s evaluation directory. --verify rehashes
weights and replays recordings without new model calls. Evaluation stops affect only that
evaluation and remain cooperative.
Validate and compare retained checkpoints
finetune evaluation-targets RUN_ID discovers checkpoint receipt metadata; discovery
explicitly reports bytes_verified: false. The evaluation review hashes the selected bytes.
Named published checkpoints use split: validation unless separately nominated for test.
Inference eligibility is broader than exact recovery: an exhausted budget or older trainer
does not prevent validation if the terminal local parent and artifact checks pass.
This queues checkpoint hashing and recorded judge replay without inference. Cohorts must
match cases/provenance, model/parent, training/evaluation seeds, judge/rubric, generation,
decoding, and code protocol. Consistent repeat evaluations count once, not as independent
training seeds. Invalid or conflicting evidence prevents a recommendation.
Eligible checkpoints improve over the base without any case-level success, policy,
truncation, or required-check regression. Ranking uses successful outcomes, judge mean,
then earliest saved step. If none qualifies, retain the base recommendation. In the studio,
open j, choose a completed checkpoint validation, and press b for comparison.
Nominate for a separate fresh test
Nomination retains a byte-bound selection receipt without copying weights or running a
model. Configure the next evaluation with the returned checkpoint and nomination_id,
split: test, and a separate reviewed test file. A fresh evaluation review and launch are
still required. Unnominated named checkpoints remain validation-only.
Test IDs and normalized user turns must not overlap retained validation history, including
prepared or failed jobs. Unreadable history or excessive entries fails closed. Exact checks
and declared reviewer freshness do not certify semantic or external independence.
In the studio, n on a completed checkpoint recommendation opens nomination review;
h reopens saved nominations. The fresh-test form leaves cases blank until separately
selected. Nomination does not replace the final adapter, alter completion/quality receipts,
or approve deployment.
Record paired quality evidence
Evidence import validates paired test reports and binds one candidate’s completed local
weights to this run. Inspection rehashes this artifact and recomputes recorded comparisons;
it runs no model or judge. Hosted pointers are not supported by this local evidence path.
recorded_study_passed requires at least six paired training seeds, distinct candidate
inventories, at least 32 distinct cases per seed, significance and safety gates, matching
protocols, pinned model/judge/rubric identity, reviewer provenance, and this run’s recorded
training seed. One-seed results remain descriptive. Other artifacts, independence,
evaluator identity, and actual train/test isolation still require external review.
The workflow does not set quality_gain_verified or automatically promote an artifact.
Source basis: the October 20, 2026 docs/FINETUNING_TUI.md, FINETUNING_AGENTS.md, and
current CLI schema. See River’s separate training and study contracts
for managed checkpoints and native RL evidence. Last modified on October 20, 2026
