The models
Three LoRA adapters for Qwen3-1.7B. Imitation teaches the rule; reinforcement learning teaches the reasoning. So far, no recipe teaches both.
conviction-1.7b-sft ↗ conviction-1.7b-rl ↗ conviction-1.7b-v2 ↗ Training code ↗
Three recipes
All three start from the same base model and see disjoint halves of the same training split. Bars show held-out problem types (the hardest split); the grey tick is the base model.
Where each recipe moves the model
Each arrow starts at the base model. RL moves it right (more accurate on new problems); SFT moves it up on the provenance rule. v2 gets furthest up on held-out problems but gives back some of RL’s accuracy.
Fingerprints
Every metric on one scale, higher is better. Whiskers are 95% cluster-bootstrap intervals; the hollow ring is the base model. Hover a row for the numbers.
Watching them learn
GRPO learns only from groups of samples that earn different rewards. After full SFT, the model already answers the training prompts the same way every time, so most groups carry no signal and RL has almost nothing to work with. A 1,000-example warm-start (v2) leaves room to learn.
One problem, four challenges
A held-out problem (modular arithmetic never appears in training) from one counterfactual group. Each reply is graded on two things: the final answer, and the decision (keep or revise). These are the models’ real, unedited outputs.
Use them
Greedy decoding, non-thinking mode. The adapters expect the benchmark system prompt (system_prompt.txt in each repository) and treat tool messages as the only authoritative source of facts. They are research artefacts trained on synthetic problems: do not deploy them as a defence against manipulation.