# Version ledger

> The ledger of every trained version: config, training stats and measured results, plus the test-set eras.

Canonical: https://marvinlabs.org/versions
Data: [versions.json](https://marvinlabs.org/data/versions.json)
HTML: https://marvinlabs.org/versions

LoRA fine-tunes of unsloth/gemma-3-4b-it as a closed-set command router (one JSON tool call per utterance), trained and served on 8 GB NVIDIA consumer cards, 13-25 Sep 2026.

## Version ledger {#ledger}

One row per trained model or variant. Test sets differ between eras, so compare accuracy only within one test set. Each version links to its full record below.

| Version | Headline result | Status | Rank | Corpus examples | Epochs | Alpha | rsLoRA | Dropout | Peak VRAM (MB) | Trained | Alias | Ollama tag |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| [v1](#v1) | 80.4% on 550 (2026-09-14) |  | 16 | 152 | 3 | 16 | no | 0 | 4926 | 2026-09-13 to 2026-09-14 |  | `gemma3-4b-router-finetuned` |
| [v2](#v2) | 89.6% on 550 (2026-09-14) |  | 16 | 490 | 3 | 16 | no | 0 | 5053 | ended 2026-09-14 |  | `gemma3-4b-router-v2` |
| [v3](#v3) | 89.8% on 790 (2026-09-15) |  | 16 | 696 | 3 | 16 | no | 0 | 5040 | ended 2026-09-14 |  | `gemma3-4b-router-v3` |
| [v4](#v4) | 82.9% on 865 (2026-09-15) |  | 16 | 59 | 8 | 16 | no | 0 | 4889 | ended 2026-09-15 |  | `gemma3-4b-router-v4` |
| [v5](#v5) | 89.2% on 865 (2026-09-15) | [deployed] [unverified] | 16 | 757 | 3 | 16 | no | 0 | 5117 | ended 2026-09-15 | marvin-v1.0 | `gemma3-4b-router-v5` |
| [v5-q4](#v5-q4) | 86.1% on 865 (2026-09-16) | [rejected] |  |  |  |  |  |  |  |  |  | `gemma3-4b-router-v5-q4` |
| [v6-wide](#v6-wide) | 93.1% on 865 (2026-09-17) | [unverified] | 64 | 757 | 3 | 64 | no | 0 | 5796 | ended 2026-09-17 |  | `gemma3-4b-router-v6-wide` |
| [v7](#v7) | 94.3% on 865 (2026-09-18) | [deployed] [unverified] | 112 | 757 | 3 | 112 | no | 0 | 6647 | ended 2026-09-17 | marvin-v1.1 | `gemma3-4b-router-v7-r112` |
| [v8](#v8) | 88.8% on 1,036 raw (2026-09-18) |  | 112 | 1050 | 3 | 112 | no | 0 | 6643 | ended 2026-09-18 |  | `gemma3-4b-router-v8-r112` |
| [v9-step175](#v9-step175) | 86.6% on 1,021 runnable (2026-09-19) | [rejected] | 112 | 1080 |  | 112 | no | 0 |  |  |  | `gemma3-4b-router-v9-r112-ckpt175` |
| [v9](#v9) | 93.0% on 1,021 runnable (2026-09-19) | [deployed] | 112 | 1080 | 3 | 112 | no | 0 | 6643 | ended 2026-09-19 | marvin-v1.2 | `gemma3-4b-router-v9-r112` |
| [v10-r64](#v10-r64) | 91.6% on 1,125, harness fixed (2026-09-20) |  | 64 | 1080 | 3 | 64 | no | 0 | 5767 | ended 2026-09-20 |  | `gemma3-4b-router-v10-r64` |
| [v11](#v11) | 96.4% on 1,125, harness fixed (2026-09-21) |  | 112 | 1167 | 3 | 112 | no | 0 | 6648 | 2026-09-20 to 2026-09-21 | marvin-v1.3 | `gemma3-4b-router-v11-r112` |
| [v12-e2](#v12-e2) | 95.6% on 1,131 (2026-09-21) |  | 112 | 1167 | 2 | 112 | no | 0 | 6648 | ended 2026-09-21 |  | `gemma3-4b-router-v12-r112-e2` |
| [v11+grammar](#v11-grammar) | 95.7% on 1,125 (2026-09-21) |  |  |  |  |  |  |  |  |  |  | `gemma3-4b-router-v11-r112` |
| [v11-avg](#v11-avg) | 96.9% on 1,131 (2026-09-21) |  |  |  |  |  |  |  |  |  |  | `gemma3-4b-router-v11-avg` |
| [v13-rslora-a112 (killed)](#v13-rslora-a112-killed) |  |  | 112 |  |  | 112 | yes | 0 |  |  |  |  |
| [v13](#v13) | 97.5% on 1,131 (2026-09-22) | [rejected] | 112 | 1172 | 3 | 32 | yes | 0 | 6653 | ended 2026-09-22 |  | `gemma3-4b-router-v13-r112-rslora-a32` |
| [v13-avg](#v13-avg) | 97.2% on 1,131 (2026-09-22) |  |  |  |  |  |  |  |  |  |  | `gemma3-4b-router-v13-avg` |
| [v13-ck225](#v13-ck225) | 83.8% on 1,131 (2026-09-23) |  |  |  |  |  |  | 0 |  |  |  | `gemma3-4b-router-v13-ck225` |
| [v14](#v14) | 87.0% on 1,131 (2026-09-23) |  | 112 | 1172 | 3 | 32 | yes | 0.1 |  |  |  | `gemma3-4b-router-v14-ck225` |
| [v15](#v15) | 96.8% on 1,131 (2026-09-23) | [rejected] | 96 | 1172 | 3 | 30 | yes | 0.1 | 6629 | ended 2026-09-23 |  | `gemma3-4b-router-v15-r96-drop10` |
| [v16](#v16) | 90.2% on 1,148 (2026-09-24) |  | 112 | 873 | 3 | 112 | no | 0 | 6653 | 2026-09-23 to 2026-09-24 |  | `gemma3-4b-router-v16-disjoint` |
| [v17](#v17) | 95.2% on 1,275 (2026-09-25) | [deployed] | 112 | 1189 | 3 | 112 | no | 0 | 6658 | 2026-09-24 to 2026-09-25 | marvin-v1.3 | `gemma3-4b-router-v17-full` |

## Test-set eras {#test-set-eras}

The test set grew over the program. A number from one era does not compare with a number from another.

| Cases | Note | Runnable | Kind | Held out | flags | Measured on |
| --- | --- | --- | --- | --- | --- | --- |
| 550 | 535 regression + 15 hard cases |  |  |  |  | 2026-09-14 |
| 790 | 775 regression + 15 hard; 240 new cases weighted toward known weak capabilities |  |  |  |  | 2026-09-15 |
| 865 | 850 regression + 15 hard |  |  |  |  | 2026-09-15 |
| 1036 | v8 bake; 60 cases whose capability is disabled on the host were counted as failures in the raw summary | 976 |  |  | [inferred] | 2026-09-18 |
| 1081 | disabled-capability cases skipped for every model from here on | 1021 |  |  |  | 2026-09-19 |
| 1125 | first set scored with the harness channel fix (context cases routed on their own channel) |  |  |  |  | 2026-09-19 |
| 1131 | 1,125 + six two-turn-back context cases; 1,021 train-flagged + 110 held-out |  |  |  |  | 2026-09-21 |
| 1191 | all-caps run: 1,131 plus the 60 flag-gated desktop/finance cases enabled |  |  |  |  | 2026-09-21 |
| 1148 | v16 bake with the disjoint-split manifest; 749 train-flagged + 399 held-out |  |  |  |  | 2026-09-24 |
| 1275 | adds 127 never-seen cases (86 adversarial guards, 41 session-capability) |  |  |  |  | 2026-09-24 |
| 174 |  |  | guard set | 48 |  | 2026-09-22 |
| 181 |  |  | guard set |  |  | 2026-09-24 |
| 267 |  |  | guard set, expanded with 86 adversarial guards |  |  | 2026-09-24 |

## Definitions {#definitions}

- **Accuracy definition:** passed / runnable cases. A case passes when the router returns the asserted capability and arguments, and none of the case's forbidden capabilities.
- **Held out definition:** cases with no training label in the corpus the model was trained on. The held-out slice changed size as the case file grew, so compare held-out numbers only within one test-set era.
- **Fair slice definition:** cases carrying no train key at all, which neither compared model ever saw (used for v16 and v17).
- **Guard definition:** must-not-fire cases: utterances that must NOT route to a named capability (negations, quoted or pasted content, vague phrasing). Counts only; case text is excluded.
- **Speed definition:** mean seconds per case through the production routing prompt (~5.6k tokens), model fully on GPU unless noted.
- **Dates:** UTC. measured_on is the file timestamp of the result, or the date the program notes records.
- **Shared recipe:** see recipe.json; per-version config below lists only what differs or what the training log confirms.

## Per-version records {#records}

### v1 {#v1}

- **Ollama tag:** `gemma3-4b-router-finetuned`
- **base model:** `unsloth/gemma-3-4b-it`
- **Corpus examples:** 152
- **trained:** Start: 2026-09-13; End: 2026-09-14T00:26Z
- **what changed:** First LoRA, on 152 curated examples mostly mined from the untuned router's own logged decisions plus a first batch of hand-authored gold cases.
- **outcome:** Regressed: 4.9 pp below the untuned 12B baseline (85.3%). Not caught until v2's bake, because v1 was never evaluated after training.

#### Config {#v1-config}

- **Rank:** 16
- **Alpha:** 16
- **RsLoRA:** no
- **Dropout:** 0
- **Epochs:** 3
- **Optimizer steps:** 57
- **Lr:** 0.0002
- **Measured on:** 2026-09-14

#### Training {#v1-training}

- **Train loss:** 0.3685
- **Peak VRAM MB:** 4926
- **Runtime s:** 4616
- **Measured on:** 2026-09-14

#### Results {#v1-results}

| Test set | Passed | Accuracy | Wall clock s | Measured on |
| --- | --- | --- | --- | --- |
| 550 | 442 | 0.8036 | 1081.2 | 2026-09-14 |

### v2 {#v2}

- **Ollama tag:** `gemma3-4b-router-v2`
- **base model:** `unsloth/gemma-3-4b-it`
- **Corpus examples:** 490
- **trained:** End: 2026-09-14T07:08Z
- **what changed:** Corpus tripled with hand-authored gold cases weighted toward vague and ambiguous phrasing and full per-capability coverage.
- **outcome:** First fine-tune to beat the untuned baseline: +4.3 pp over gemma4:12b-it-qat on the same 550 cases.

#### Config {#v2-config}

- **Rank:** 16
- **Alpha:** 16
- **RsLoRA:** no
- **Dropout:** 0
- **Epochs:** 3
- **Optimizer steps:** 186
- **Lr:** 0.0002
- **Measured on:** 2026-09-14

#### Training {#v2-training}

- **Final segment train loss:** 0.0274
- **Peak VRAM MB:** 5053
- **Relaunches:** 9
- **Note:** train_loss covers only the last resumed segment, not the run
- **Measured on:** 2026-09-14

#### Results {#v2-results}

| Test set | Passed | Accuracy | Wall clock s | Capability | Total | Failure shape | Measured on |
| --- | --- | --- | --- | --- | --- | --- | --- |
| 550 | 493 | 0.8964 | 1028.8 |  |  |  | 2026-09-14 |
| 550 | 27 | 0.562 |  | clarify | 48 |  | 2026-09-14 |
| 550 |  |  |  |  |  | 12 of 57 misses picked a forbidden capability (21%), vs 6/81 for the untuned 12B and 8/108 for v1 | 2026-09-14 |

### v3 {#v3}

- **Ollama tag:** `gemma3-4b-router-v3`
- **base model:** `unsloth/gemma-3-4b-it`
- **Corpus examples:** 696
- **trained:** End: 2026-09-14T23:57Z
- **what changed:** 240 new gold cases for the capabilities v2 missed most; full corpus, fresh from base.
- **outcome:** Held: same score as v2 on a set 240 cases harder. Not comparable to v2's number (different test set).

#### Config {#v3-config}

- **Rank:** 16
- **Alpha:** 16
- **RsLoRA:** no
- **Dropout:** 0
- **Epochs:** 3
- **Optimizer steps:** 261
- **Lr:** 0.0002
- **Measured on:** 2026-09-14

#### Training {#v3-training}

- **Train loss:** 0.1502
- **Peak VRAM MB:** 5040
- **Runtime s:** 21162
- **Relaunches:** 0
- **Measured on:** 2026-09-14

#### Results {#v3-results}

| Test set | Passed | Accuracy | Wall clock s | Capability | Measured on |
| --- | --- | --- | --- | --- | --- |
| 790 | 709 | 0.8975 | 1056.8 |  | 2026-09-15 |
| 790 |  | 0.604 |  | clarify | 2026-09-15 |

### v4 {#v4}

- **Ollama tag:** `gemma3-4b-router-v4`
- **base model:** `v3's adapter (continued, not fresh from base)`
- **Corpus examples:** 59
- **trained:** End: 2026-09-15T02:32Z; Conflict: The program notes and the training script's docstring date the run 2026-09-16; file timestamps (train log 09-15 02:32, bake summary 09-15 02:54) say 2026-09-15. Trust the file timestamps.
- **what changed:** Incremental patch: only the 59 new or corrected examples, 8 epochs, continued from v3's adapter.
- **outcome:** Catastrophic forgetting. Capabilities the patch never touched collapsed to 25-56%. Never deployed.

#### Config {#v4-config}

- **Rank:** 16
- **Alpha:** 16
- **RsLoRA:** no
- **Dropout:** 0
- **Epochs:** 8
- **Optimizer steps:** 64
- **Lr:** 0.0002
- **Continued from:** v3 adapter
- **Measured on:** 2026-09-15

#### Training {#v4-training}

- **Train loss:** 0.0522
- **Peak VRAM MB:** 4889
- **Runtime s:** 4778
- **Measured on:** 2026-09-15

#### Results {#v4-results}

| Test set | Passed | Accuracy | Wall clock s | Total | Vs v3 | Per capability | Before v3 | Measured on |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 865 | 717 | 0.8289 | 1156.9 |  |  |  |  | 2026-09-15 |
| the 790 cases v3 was scored on | 651 | 0.8241 |  | 790 | -0.0734 |  |  | 2026-09-15 |
| 790 subset |  |  |  |  |  | Run agent: 0.25; Resume agent: 0.417; Dispatch code: 0.524; Calendar availability: 0.562; Clarify: 0.525; Gate reject: 1.0; Dispatch answer: 0.931 | Run agent: ~1.0; Resume agent: 0.86-1.0; Dispatch code: 0.91-1.0; Calendar availability: 1.0; Clarify: 0.604 | 2026-09-15 |

### v5 {#v5}

- **alias:** marvin-v1.0
- **Ollama tag:** `gemma3-4b-router-v5`
- **base model:** `unsloth/gemma-3-4b-it`
- **Corpus examples:** 757
- **trained:** End: 2026-09-15T11:32Z
- **what changed:** v4's 59 examples mixed into the full corpus and retrained fresh from base.
- **outcome:** Repaired v4's damage (run_agent, resume_agent, dispatch_code, calendar_availability all back to 100% on the 790 subset). Deployed 15 Sep.

#### Config {#v5-config}

- **Rank:** 16
- **Alpha:** 16
- **RsLoRA:** no
- **Dropout:** 0
- **Epochs:** 3
- **Optimizer steps:** 285
- **Lr:** 0.0002
- **Measured on:** 2026-09-15

#### Training {#v5-training}

- **Train loss:** 0.0729
- **Peak VRAM MB:** 5117
- **Kills by low memory monitor:** 4
- **Note:** train_loss and runtime cover the last resumed segment
- **Measured on:** 2026-09-15

#### Results {#v5-results}

| Test set | Passed | Accuracy | Wall clock s | S per case | Quant | Total | Note | Held out accuracy | flags | Measured on |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 865 | 772 | 0.8925 | 1141.5 |  |  |  |  |  |  | 2026-09-15 |
| 865 | 773 | 0.8936 |  | 1.506 | Q8_0 |  |  |  |  | 2026-09-16 |
| 865 | 770 | 0.8902 |  | 1.333 |  |  |  |  |  | 2026-09-17 |
| old 790 subset | 709 | 0.8975 |  |  |  | 790 | identical to v3 to the case |  |  | 2026-09-15 |
| 75 cases new since v3 | 63 | 0.84 |  |  |  | 75 |  |  |  | 2026-09-15 |
| 865 |  |  |  |  |  |  | ledger-only. Recomputing from the bake progress file with today's train flags gives a different split (flags changed after 09-18), so this could not be reproduced. | 0.849 | [unverified] | 2026-09-18 |

### v5-q4 {#v5-q4}

- **Ollama tag:** `gemma3-4b-router-v5-q4`
- **base model:** `same merged weights as v5`
- **what changed:** Quantization only.
- **outcome:** Rejected: -3.2 pp for ~15% speed. Q8_0 stays.

#### Config {#v5-q4-config}

- **Quant:** Q4_0
- **GGUF size GB:** 2.37
- **Measured on:** 2026-09-16

#### Results {#v5-q4-results}

| Test set | Passed | Accuracy | S per case | Delta vs q8 | Speed ratio vs q8 | Measured on |
| --- | --- | --- | --- | --- | --- | --- |
| 865 | 745 | 0.8613 | 1.278 | -0.0323 | 0.849 | 2026-09-16 |

### v6-wide {#v6-wide}

- **Ollama tag:** `gemma3-4b-router-v6-wide`
- **base model:** `unsloth/gemma-3-4b-it`
- **Corpus examples:** 757
- **trained:** End: 2026-09-17T01:46Z
- **what changed:** Rank 16 to 64 (alpha kept equal to rank). Same 757 examples, same recipe, fresh from base.
- **outcome:** +4.0 pp over v5 from rank alone. The two passes a day apart (93.1% / 93.9%) set the noise floor for this era.

#### Config {#v6-wide-config}

- **Rank:** 64
- **Alpha:** 64
- **RsLoRA:** no
- **Dropout:** 0
- **Epochs:** 3
- **Optimizer steps:** 285
- **Lr:** 0.0002
- **Measured on:** 2026-09-17

#### Training {#v6-wide-training}

- **Train loss:** 0.1227
- **Peak VRAM MB:** 5796
- **Runtime s:** 23944
- **Measured on:** 2026-09-17

#### Results {#v6-wide-results}

| Test set | Passed | Accuracy | S per case | Capability | Accuracy before | Accuracy after | N | Held out accuracy | flags | Measured on |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 865 | 805 | 0.9306 | 1.223 |  |  |  |  |  |  | 2026-09-17 |
| 865 | 812 | 0.9387 | 1.25 |  |  |  |  |  |  | 2026-09-18 |
| 865 |  |  |  | clarify | 0.521 | 0.755 | 94 |  |  | 2026-09-17 |
| 865 |  |  |  |  |  |  |  | 0.855, 0.882 | [unverified] | 2026-09-18 |

### v7 {#v7}

- **alias:** marvin-v1.1
- **Ollama tag:** `gemma3-4b-router-v7-r112`
- **base model:** `unsloth/gemma-3-4b-it`
- **Corpus examples:** 757
- **trained:** End: 2026-09-17T23:06Z
- **what changed:** Rank 64 to 112, the highest rank that fit on the 8 GB card.
- **outcome:** +0.5 pp over v6-wide, inside v6-wide's own day-to-day spread. Shipped 18 Sep, replaced by v9 on 19 Sep.

#### Config {#v7-config}

- **Rank:** 112
- **Alpha:** 112
- **RsLoRA:** no
- **Dropout:** 0
- **Epochs:** 3
- **Optimizer steps:** 285
- **Lr:** 0.0002
- **Measured on:** 2026-09-17

#### Training {#v7-training}

- **Train loss:** 0.1251
- **Peak VRAM MB:** 6647
- **Runtime s:** 24217
- **Measured on:** 2026-09-17

#### Results {#v7-results}

| Test set | Passed | Accuracy | S per case clean | S per case raw | Note | Per capability | Held out passed | Held out total | Held out accuracy | S per case | Train flagged accuracy | flags | Measured on |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 865 | 816 | 0.9434 | 1.26 | 1.707 | raw speed polluted: first 253 cases ran split CPU/GPU |  |  |  |  |  |  |  | 2026-09-18 |
| 865 |  |  |  |  | v6-wide then v7; 27 cases fixed, 23 broken | Clarify: 0.745, 0.851; Context setup cases n31: 0.968, 0.871 |  |  |  |  |  |  | 2026-09-18 |
| 1,021 runnable | 933 | 0.9138 |  |  |  |  | 76 | 91 | 0.8352 | 1.269 |  |  | 2026-09-19 |
| 1,036 raw (v8 era) | 906 | 0.8745 |  |  |  |  | 69 | 90 |  | 1.427 |  |  | 2026-09-18 |
| 108-case held-out slice, harness fixed | 92 | 0.8519 |  |  |  |  |  |  |  |  |  |  | 2026-09-19 |
| 865 |  |  |  |  |  |  |  |  | 0.871 |  | 0.963 | [unverified] | 2026-09-18 |

### v8 {#v8}

- **Ollama tag:** `gemma3-4b-router-v8-r112`
- **base model:** `unsloth/gemma-3-4b-it`
- **Corpus examples:** 1050
- **trained:** End: 2026-09-18T11:19Z
- **what changed:** Enriched corpus: 126 negative-guard cases given targets, setup context rendered into prompts (27 context cases had trained context-free), clarify-chain turns exported, 171 gold cases including the 9 capabilities that had zero examples.
- **outcome:** Mixed, not deployed. Guards rose 19-50 pp, but four new clarify labels on bare confirmation words contradicted older gold and dropped gate_approve from 100% to 70%.

#### Config {#v8-config}

- **Rank:** 112
- **Alpha:** 112
- **RsLoRA:** no
- **Dropout:** 0
- **Epochs:** 3
- **Optimizer steps:** 396
- **Lr:** 0.0002
- **Measured on:** 2026-09-18

#### Training {#v8-training}

- **Train loss:** 0.1117
- **Peak VRAM MB:** 6643
- **Runtime s:** 34881
- **Measured on:** 2026-09-18

#### Results {#v8-results}

| Test set | Passed | Accuracy | Held out passed | Held out total | Held out accuracy | S per case | Conflict | Per capability | Guard families gain pp | flags | Measured on |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 1,036 raw | 920 | 0.888 | 71 | 90 | 0.7889 | 1.249 |  |  |  |  | 2026-09-18 |
| 976 runnable |  | 0.943 |  |  | 0.91 |  | The raw summary scores 88.8% on 1,036 because 60 disabled-capability cases count as failures. 920/976 = 94.3% matches the program notes, so the program notes number is the raw one with those 60 removed. The held-out 91.0% would need ~12 of the 60 to be held-out (71/78); not verifiable from the summary. |  |  | [inferred] | 2026-09-18 |
| 976 runnable |  |  |  |  |  |  |  | Gate approve: 1.0, 0.7 | 19 to 50 |  | 2026-09-18 |

### v9-step175 {#v9-step175}

- **Ollama tag:** `gemma3-4b-router-v9-r112-ckpt175`
- **base model:** `unsloth/gemma-3-4b-it`
- **Corpus examples:** 1080
- **what changed:** Same run as v9, earlier checkpoint.
- **outcome:** Rejected. Tied the final checkpoint on held-out, lost 6.5 pp overall. Eval loss on partial-argument targets does not select for accuracy.

#### Config {#v9-step175-config}

- **Rank:** 112
- **Alpha:** 112
- **RsLoRA:** no
- **Dropout:** 0
- **Checkpoint step:** 175
- **Of steps:** 405
- **Selected by:** lowest held-out eval loss (0.1306)
- **Measured on:** 2026-09-19

#### Results {#v9-step175-results}

| Test set | Passed | Accuracy | Held out passed | Held out total | Held out accuracy | S per case | Measured on |
| --- | --- | --- | --- | --- | --- | --- | --- |
| 1,021 runnable | 884 | 0.8658 | 79 | 91 | 0.8681 | 1.31 | 2026-09-19 |
| 108-case held-out slice, harness fixed | 96 | 0.8889 |  |  |  |  | 2026-09-19 |

### v9 {#v9}

- **alias:** marvin-v1.2
- **Ollama tag:** `gemma3-4b-router-v9-r112`
- **base model:** `unsloth/gemma-3-4b-it`
- **Corpus examples:** 1080
- **Eval examples:** 55
- **trained:** End: 2026-09-19T04:41Z
- **what changed:** v8's corpus with the conflicting labels removed and 35 counterweight cases added; held-out eval loss logged every 25 steps, every checkpoint kept.
- **outcome:** Deployed 19 Sep. First fine-tune to beat the untuned 12B on held-out once the harness was fixed (91.7% vs 88.9%).

#### Config {#v9-config}

- **Rank:** 112
- **Alpha:** 112
- **RsLoRA:** no
- **Dropout:** 0
- **Epochs:** 3
- **Optimizer steps:** 405
- **Lr:** 0.0002
- **Measured on:** 2026-09-19

#### Training {#v9-training}

- **Train loss:** 0.1115
- **Peak VRAM MB:** 6643
- **Runtime s:** 38218
- **Eval loss best:** Step: 175; Loss: 0.1306
- **Eval loss final:** 0.1861
- **Measured on:** 2026-09-19

#### Results {#v9-results}

| Test set | Passed | Accuracy | Held out passed | Held out total | Held out accuracy | S per case | Passed per run | Accuracy per run | Mean accuracy | Spread pp | Held out passed per run | Held out breaches | Per capability | Fixed vs v7 | Broke vs v7 | Measured on |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 1,021 runnable | 950 | 0.9305 | 79 | 91 | 0.8681 | 1.303 |  |  |  |  |  |  |  |  |  | 2026-09-19 |
| 1,125, harness fixed | 1050 | 0.9333 | 99 | 108 |  | 1.264 |  |  |  |  |  |  |  |  |  | 2026-09-19 |
| 1,125, harness fixed | 1047 | 0.9307 | 99 | 108 |  | 1.34 |  |  |  |  |  |  |  |  |  | 2026-09-20 |
| 1,125, harness fixed | 1043 | 0.9271 | 100 | 108 |  | 1.332 |  |  |  |  |  |  |  |  |  | 2026-09-21 |
| 1,191 all-caps x3 |  |  |  | 122 |  |  | 1098, 1112, 1112 | 0.9219, 0.9337, 0.9337 | 0.9298 | 1.18 | 111, 112, 111 |  |  |  |  | 2026-09-21 |
| 174 guards | 167 |  |  |  |  |  |  |  |  |  |  | 1 |  |  |  | 2026-09-22 |
| 1,021 runnable |  |  |  |  |  |  |  |  |  |  |  |  | Gate approve: 0.94; Dispatch answer: 0.59; Guards: 0.96-1.0 | 45 | 28 | 2026-09-19 |

### v10-r64 {#v10-r64}

- **Ollama tag:** `gemma3-4b-router-v10-r64`
- **base model:** `unsloth/gemma-3-4b-it`
- **Corpus examples:** 1080
- **trained:** End: 2026-09-20T06:03Z
- **what changed:** v9's exact data and recipe at rank 64. Rank the only variable.
- **outcome:** -1.4 pp vs v9 at rank 112 on the same data; clarify 86% to 73%. Rank 112 stays.

#### Config {#v10-r64-config}

- **Rank:** 64
- **Alpha:** 64
- **RsLoRA:** no
- **Dropout:** 0
- **Epochs:** 3
- **Optimizer steps:** 405
- **Lr:** 0.0002
- **Measured on:** 2026-09-20

#### Training {#v10-r64-training}

- **Train loss:** 0.1103
- **Peak VRAM MB:** 5767
- **Runtime s:** 37684
- **Eval loss best:** Step: 125; Loss: 0.1372
- **Eval loss final:** 0.1839
- **Measured on:** 2026-09-20

#### Results {#v10-r64-results}

| Test set | Passed | Accuracy | Held out passed | Held out total | Held out accuracy | S per case | Capability | Rank112 same data | Measured on |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 1,125, harness fixed | 1031 | 0.9164 | 98 | 108 | 0.9074 | 1.224 |  |  | 2026-09-20 |
| 1,125, harness fixed |  | 0.73 |  |  |  |  | clarify | 0.86 | 2026-09-20 |

### v11 {#v11}

- **alias:** marvin-v1.3 (25 Sep 06:42 to 08:10 UTC, then re-pointed to v17)
- **Ollama tag:** `gemma3-4b-router-v11-r112`
- **base model:** `unsloth/gemma-3-4b-it`
- **Corpus examples:** 1167
- **Eval examples:** 72
- **trained:** Start: 2026-09-20T13:27Z; End: 2026-09-21T01:18Z
- **what changed:** Corpus 1,167: new compound and correct_last capabilities, 121 context cases, clarify chains, 40 dispatch_answer positives.
- **outcome:** Best and most stable accuracy of the program. Won the accuracy rule against six challengers (v12, v13, v15, v16, v17 and both averages). Released as marvin-v1.3, then replaced the same morning by v17 on guard behaviour.

#### Config {#v11-config}

- **Rank:** 112
- **Alpha:** 112
- **RsLoRA:** no
- **Dropout:** 0
- **Epochs:** 3
- **Optimizer steps:** 438
- **Lr:** 0.0002
- **Measured on:** 2026-09-21

#### Training {#v11-training}

- **Train loss:** 0.1073
- **Peak VRAM MB:** 6648
- **Runtime s:** 42633
- **Eval loss best:** Step: 75; Loss: 0.2727
- **Eval loss final:** 0.3997
- **Measured on:** 2026-09-21

#### Results {#v11-results}

| Test set | Passed | Accuracy | Held out passed | Held out total | S per case | Passes | Accuracy range | Train flagged accuracy | Held out accuracy | Passed per run | Mean accuracy | Spread pp | Held out per run | Fair slice | Held out breaches | Breach categories in rechecks | Per capability changes | Note | Lowest capabilities | Measured on |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 1,125, harness fixed | 1084 | 0.9636 | 104 | 108 | 1.245 |  |  |  |  |  |  |  |  |  |  |  |  |  |  | 2026-09-21 |
| 1,131 |  |  |  |  |  | (Passed: 1090, Held out: 108/110; measured on 2026-09-21); (Passed: 1090, Held out: 108/110; measured on 2026-09-21); (Passed: 1091, Held out: 108/110; measured on 2026-09-21); (Passed: 1090, Held out: 108/110; measured on 2026-09-22); (Passed: 1090, Held out: 107/110; measured on 2026-09-23) | 0.9637, 0.9646 | 0.9618 | 0.9818 |  |  |  |  |  |  |  |  |  |  |  |
| 1,191 all-caps x3 |  |  |  |  |  |  |  |  |  | 1145, 1144, 1145 | 0.9611 | 0.09 | 117/122 each |  |  |  |  |  |  | 2026-09-21 |
| 1,148 (v16 manifest) | 1101 | 0.9591 | 379 | 399 | 1.194 |  |  |  |  |  |  |  |  |  |  |  |  |  |  | 2026-09-24 |
| 82 cases held out from both v11 and v16 |  | 0.963 |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  | 2026-09-24 |
| 317 cases v11 trained on and v16 did not |  | 0.946 |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  | 2026-09-24 |
| 1,275 expanded | 1212 | 0.9506 | 490 | 526 | 1.197 |  |  |  |  |  |  |  |  |  |  |  |  |  |  | 2026-09-24 |
| 1,275 (v17 bake) | 1211 | 0.9498 | 222 | 237 | 1.187 |  |  |  |  |  |  |  |  | 93/100 |  |  |  |  |  | 2026-09-25 |
| 174 guards | 169 |  |  |  |  |  |  |  |  |  |  |  |  |  | 1 |  |  |  |  | 2026-09-22 |
| 181 guards | 172 |  |  |  |  |  |  |  |  |  |  |  |  |  | 1 |  |  |  |  | 2026-09-24 |
| 267 guards |  |  |  |  |  | (Passed: 250, Held out breaches: 5; measured on 2026-09-24); (Passed: 247, Held out breaches: 4; measured on 2026-09-25); (Passed: 250, Held out breaches: 5; measured on 2026-09-25); (Passed: 250, Held out breaches: 5; measured on 2026-09-25); (Passed: 250, Held out breaches: 5; measured on 2026-09-25) |  |  |  |  |  |  |  |  |  | 3 of 5 held-out breaches involve pasted or quoted content; 1 vague phrasing; 1 session-capability |  |  |  |  |
| 1,125-1,131 |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  | Compound: 0.11, 0.89; Correct last: 0.0, 1.0; Gate reject: 0.88, 0.98; Guard:open folder: 1.0, 0.88 | vs marvin-v1.2; open_folder guard n=8 |  | 2026-09-21 |
| 10 pooled passes |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  | Gate approve: Accuracy: 0.766, Ci: 0.719, 0.807, Cases: 35; Clarify: Accuracy: 0.827, Ci: 0.801, 0.849, Cases: 98; Explain issue: Accuracy: 0.855, Cases: 11; Compound: Accuracy: 0.906, Cases: 18 | 2026-09-22 |

### v12-e2 {#v12-e2}

- **Ollama tag:** `gemma3-4b-router-v12-r112-e2`
- **base model:** `unsloth/gemma-3-4b-it`
- **Corpus examples:** 1167
- **trained:** End: 2026-09-21T14:55Z
- **what changed:** v11's exact data and recipe at 2 epochs. Epochs the only variable.
- **outcome:** -0.8 pp and -4 held-out cases vs v11, mostly clarify (wrong answers went to converse). Lowest final eval loss of any run, and still worse. Three epochs stays.

#### Config {#v12-e2-config}

- **Rank:** 112
- **Alpha:** 112
- **RsLoRA:** no
- **Dropout:** 0
- **Epochs:** 2
- **Optimizer steps:** 292
- **Lr:** 0.0002
- **Measured on:** 2026-09-21

#### Training {#v12-e2-training}

- **Train loss:** 0.1417
- **Peak VRAM MB:** 6648
- **Runtime s:** 28440
- **Eval loss best:** Step: 125; Loss: 0.2589
- **Eval loss final:** 0.2934
- **Measured on:** 2026-09-21

#### Results {#v12-e2-results}

| Test set | Passed | Accuracy | Held out passed | Held out total | S per case | Broke vs v11 | Fixed vs v11 | Measured on |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 1,131 | 1081 | 0.9558 | 104 | 110 | 1.236 | 26 | 17 | 2026-09-21 |

### v11+grammar {#v11-grammar}

- **Ollama tag:** `gemma3-4b-router-v11-r112 (same weights)`
- **what changed:** Decoding only.
- **outcome:** +0.1 pp once the enum was correct. Turned on in production with the 25 Sep release.

#### Config {#v11-grammar-config}

- **Change:** Ollama structured output: JSON schema with a capability enum enforced at decode time
- **Measured on:** 2026-09-21

#### Results {#v11-grammar-results}

| Attempt | Test set | Passed | Accuracy | Held out passed | Held out total | S per case | Note | Unparseable outputs | Broke | Fixed | Measured on |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| first (enum missing a router-hidden capability) | 1,125 | 1077 | 0.9573 | 103 | 108 | 1.287 | 10 cases broke, 8 of them the hidden capability forced to clarify |  |  |  | 2026-09-21 |
| fixed enum | 1,131 | 1092 | 0.9655 | 108 | 110 | 1.266 |  | 3, 1 | 1 | 3 | 2026-09-21 |

### v11-avg {#v11-avg}

- **Ollama tag:** `gemma3-4b-router-v11-avg`
- **what changed:** No training; weight averaging only.
- **outcome:** Noise: +0.4 pp on the first bake, +0.0 on the second; held-out identical both times.

#### Config {#v11-avg-config}

- **Method:** exact mean of v11 checkpoints 300, 325, 350, 375, 400, 425, 438 by concatenating LoRA A and B along the rank axis (r = 7 x 112, alpha unchanged)
- **Measured on:** 2026-09-21

#### Results {#v11-avg-results}

| Test set | Passed | Accuracy | Held out passed | Held out total | S per case | Note | Measured on |
| --- | --- | --- | --- | --- | --- | --- | --- |
| 1,131 | 1096 | 0.9691 | 108 | 110 | 1.235 |  | 2026-09-21 |
| 1,131 | 1090 | 0.9637 | 108 | 110 | 3.615 | speed contention-affected | 2026-09-22 |

### v13-rslora-a112 (killed) {#v13-rslora-a112-killed}

- **what changed:** rsLoRA with alpha left equal to rank.
- **outcome:** Killed early. Eval loss flatlined; scale 10.6 is too hot at lr 2e-4.

#### Config {#v13-rslora-a112-killed-config}

- **Rank:** 112
- **Alpha:** 112
- **RsLoRA:** yes
- **Effective scale:** 10.6
- **Dropout:** 0
- **Lr:** 0.0002
- **Measured on:** 2026-09-21

#### Training {#v13-rslora-a112-killed-training}

- **Eval loss step25:** 0.5734
- **Eval loss step50:** 0.577
- **V11 eval loss same steps:** 0.35, 0.33
- **Train loss peak:** 3.85
- **Measured on:** 2026-09-21

### v13 {#v13}

- **Ollama tag:** `gemma3-4b-router-v13-r112-rslora-a32`
- **base model:** `unsloth/gemma-3-4b-it`
- **Corpus examples:** 1172
- **Eval examples:** 74
- **trained:** End: 2026-09-22T06:35Z
- **what changed:** rsLoRA at alpha 32 on the 21 Sep corpus (1,172 examples: 6 new two-turn-back context cases, 2 label fixes, 18 conflicts excluded).
- **outcome:** Rejected. Highest overall number of the program, but every gained case was train-flagged and it lost held-out safety guards v11 passes.

#### Config {#v13-config}

- **Rank:** 112
- **Alpha:** 32
- **RsLoRA:** yes
- **Effective scale:** 3.02
- **Dropout:** 0
- **Epochs:** 3
- **Optimizer steps:** 441
- **Lr:** 0.0002
- **Measured on:** 2026-09-22

#### Training {#v13-training}

- **Train loss:** 0.1246
- **Peak VRAM MB:** 6653
- **Runtime s:** 42946
- **Eval loss best:** Step: 100; Loss: 0.2519
- **Eval loss final:** 0.3497
- **Measured on:** 2026-09-22

#### Results {#v13-results}

| Test set | Passed | Accuracy | Train flagged accuracy | Held out passed | Held out total | S per case | Fixed vs v11 | Broke vs v11 | Gains | Unparseable outputs | Held out breaches | Breach categories | Measured on |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 1,131 | 1103 | 0.9752 | 0.9794 | 103 | 110 | 1.238 |  |  |  |  |  |  | 2026-09-22 |
| 1,131 | 1104 | 0.9761 |  | 103 | 110 |  |  |  |  |  |  |  | 2026-09-22 |
| 1,131 |  |  |  |  |  |  | 29 | 16 | Gate approve: 9; Clarify: 9; Dispatch answer: 3 | 3, 1 |  |  | 2026-09-22 |
| 174 guards | 168 |  |  |  |  |  |  |  |  |  | 3 | 2 of 3 involve pasted or quoted content; 1 is an argument confusion between two knowledge capabilities | 2026-09-22 |

### v13-avg {#v13-avg}

- **Ollama tag:** `gemma3-4b-router-v13-avg`
- **what changed:** No training; weight averaging only.
- **outcome:** -0.4 pp vs v13 with one more held-out miss. Did not recover the guard losses.

#### Config {#v13-avg-config}

- **Method:** exact checkpoint mean, with B scaled by a further 1/sqrt(n) because rsLoRA scales by alpha/sqrt(r)
- **Measured on:** 2026-09-22

#### Results {#v13-avg-results}

| Test set | Passed | Accuracy | Held out passed | Held out total | Measured on |
| --- | --- | --- | --- | --- | --- |
| 1,131 | 1099 | 0.9717 | 102 | 110 | 2026-09-22 |

### v13-ck225 {#v13-ck225}

- **Ollama tag:** `gemma3-4b-router-v13-ck225`
- **what changed:** Mid-run checkpoint.
- **outcome:** Only the delta to v14-ck225 means anything; absolute scores are for a half-trained model.

#### Config {#v13-ck225-config}

- **Checkpoint step:** 225
- **Of steps:** 441
- **Dropout:** 0
- **Note:** about 1.5 epochs; control arm of the dropout A/B
- **Measured on:** 2026-09-23

#### Results {#v13-ck225-results}

| Test set | Passed | Accuracy | Train flagged accuracy | Held out passed | Held out total | Held out breaches | Measured on |
| --- | --- | --- | --- | --- | --- | --- | --- |
| 1,131 | 948 | 0.8382 | 0.8394 | 91 | 110 |  | 2026-09-23 |
| 174 guards | 148 |  |  |  |  | 10 | 2026-09-23 |

### v14 {#v14}

- **Ollama tag:** `gemma3-4b-router-v14-ck225 (only the step-225 checkpoint was deployed)`
- **base model:** `unsloth/gemma-3-4b-it`
- **Corpus examples:** 1172
- **trained:** Attempts: 2026-09-22T20:16Z, 2026-09-23T01:23Z, 2026-09-23T03:06Z
- **what changed:** v13's recipe plus LoRA dropout 0.1.
- **outcome:** Would not fit at rank 112. At equal steps dropout added 3.2 pp, all on train-flagged cases, and widened the train-to-held-out gap from 1.2 to 6.8 pp. Guards improved 148 to 157.

#### Config {#v14-config}

- **Rank:** 112
- **Alpha:** 32
- **RsLoRA:** yes
- **Dropout:** 0.1
- **Epochs:** 3
- **Planned steps:** 441
- **Lr:** 0.0002
- **Measured on:** 2026-09-23

#### Training {#v14-training}

- **Died at steps:** 184, 236, 239
- **Causes:** CUDA illegal memory access (Xid 31 MMU fault per the program notes), CUDA OOM, 6.98 GiB allocated + 394 MiB reserved of 7.65 GiB, CUDA OOM, 6.90 GiB allocated + 457 MiB reserved
- **Measured on:** 2026-09-23

#### Results {#v14-results}

| Checkpoint | Test set | Passed | Accuracy | Train flagged accuracy | Held out passed | Held out total | S per case | Held out breaches | Measured on |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 225 | 1,131 | 984 | 0.87 | 0.8766 | 89 | 110 | 1.22 |  | 2026-09-23 |
| 225 | 174 guards | 157 |  |  |  |  |  | 7 | 2026-09-23 |

### v15 {#v15}

- **Ollama tag:** `gemma3-4b-router-v15-r96-drop10`
- **base model:** `unsloth/gemma-3-4b-it`
- **Corpus examples:** 1172
- **trained:** End: 2026-09-23T17:51Z
- **what changed:** Rank 112 to 96 to pay for dropout's memory; alpha 30 to hold rsLoRA's effective scale near v13's.
- **outcome:** Vetoed on guards. Dropout bought one held-out case over v13 and cost 0.7 pp overall. Closed the rsLoRA branch.

#### Config {#v15-config}

- **Rank:** 96
- **Alpha:** 30
- **RsLoRA:** yes
- **Effective scale:** 3.06
- **Dropout:** 0.1
- **Epochs:** 3
- **Optimizer steps:** 441
- **Lr:** 0.0002
- **Measured on:** 2026-09-23

#### Training {#v15-training}

- **Train loss:** 0.1218
- **Peak VRAM MB:** 6629
- **Predicted peak MB:** 6793
- **Runtime s:** 43420
- **Eval loss best:** Step: 300; Loss: 0.2499
- **Eval loss final:** 0.3385
- **Measured on:** 2026-09-23

#### Results {#v15-results}

| Test set | Passed | Accuracy | Train flagged accuracy | Held out passed | Held out total | S per case | Held out breaches | Breach categories | Measured on |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 1,131 | 1095 | 0.9682 | 0.9706 | 104 | 110 | 1.269 |  |  | 2026-09-23 |
| 174 guards | 165 |  |  |  |  |  | 4 | 3 of 4 involve pasted or quoted content | 2026-09-23 |

### v16 {#v16}

- **Ollama tag:** `gemma3-4b-router-v16-disjoint`
- **base model:** `unsloth/gemma-3-4b-it`
- **Corpus examples:** 873
- **trained:** Start: 2026-09-23T22:18Z; End: 2026-09-24T07:12Z
- **what changed:** v11's recipe on a disjoint corpus with a quarter of the gold set held back.
- **outcome:** -8.5 pp on the fair slice vs v11. Disproved its own premise: v11's headline was not inflated by memorization.

#### Config {#v16-config}

- **Rank:** 112
- **Alpha:** 112
- **RsLoRA:** no
- **Dropout:** 0
- **Epochs:** 3
- **Optimizer steps:** 330
- **Lr:** 0.0002
- **Holdout:** 273-case stratified slice (25%) plus 58 cases dropped because a deterministic pre-router step now answers them
- **Measured on:** 2026-09-24

#### Training {#v16-training}

- **Train loss:** 0.1316
- **Peak VRAM MB:** 6653
- **Runtime s:** 32031
- **Eval loss best:** Step: 100; Loss: 0.2603
- **Eval loss final:** 0.3877
- **Measured on:** 2026-09-24

#### Results {#v16-results}

| Test set | Passed | Accuracy | Held out passed | Held out total | S per case | V11 accuracy | Note | Held out breaches | Groups | Measured on |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 1,148 | 1035 | 0.9016 | 334 | 399 | 1.195 |  |  |  |  | 2026-09-24 |
| 82 cases held out from both v11 and v16 |  | 0.878 |  |  |  | 0.963 |  |  |  | 2026-09-24 |
| 317 cases v11 trained on and v16 did not |  | 0.826 |  |  |  | 0.946 |  |  |  | 2026-09-24 |
| 1,275 expanded | 1139 | 0.8933 | 439 | 526 | 4.231 |  | speed contention-affected |  |  | 2026-09-24 |
| 181 guards | 173 |  |  |  |  |  | the extra breach is a session-capability case that did not exist when v16's corpus was exported | 2 |  | 2026-09-24 |
| 267 guards | 253 |  |  |  |  |  |  | 5 |  | 2026-09-24 |
| 127 never-seen cases by group |  |  |  |  |  |  |  |  | All 127: V11: 0.874, V16: 0.843; Adversarial guards 62: V11: 0.855, V16: 0.903; Guard counterweights 24: V11: 0.833, V16: 0.667; Session injection guards 16: V11: 1.0, V16: 1.0; Session paraphrases and discriminators 25: V11: 0.88, V16: 0.76 | 2026-09-24 |

### v17 {#v17}

- **alias:** marvin-v1.3 (production from 2026-09-25 08:10 UTC)
- **Ollama tag:** `gemma3-4b-router-v17-full`
- **base model:** `unsloth/gemma-3-4b-it`
- **Corpus examples:** 1189
- **Eval examples:** 78
- **trained:** Start: 2026-09-24T17:10Z; End: 2026-09-25T05:38Z
- **what changed:** v11's exact recipe on the largest corpus: 86 adversarial guards, 41 session-capability cases, 33 guard training targets and 6 conflict rulings added.
- **outcome:** Lost the pre-agreed fair-slice rule by 3 cases (90 vs 93), so the scripted decision picked v11. Then chosen deliberately for guards: 256/267 vs 250/267 over three repeated passes. Live as marvin-v1.3 from 25 Sep.

#### Config {#v17-config}

- **Rank:** 112
- **Alpha:** 112
- **RsLoRA:** no
- **Dropout:** 0
- **Epochs:** 3
- **Optimizer steps:** 447
- **Lr:** 0.0002
- **Holdout:** about 10%
- **Measured on:** 2026-09-25

#### Training {#v17-training}

- **Train loss:** 0.1105
- **Peak VRAM MB:** 6658
- **Runtime s:** 44847
- **Eval loss best:** Step: 100; Loss: 0.2526
- **Eval loss final:** 0.4378
- **Measured on:** 2026-09-25

#### Results {#v17-results}

| Test set | Passed | Accuracy | Train flagged accuracy | Held out passed | Held out total | S per case | V11 passed | Passes | Breach categories in rechecks | Measured on |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 1,275 | 1214 | 0.9522 | 0.9653 | 212 | 237 | 1.174 |  |  |  | 2026-09-25 |
| fair slice, 100 cases neither v11 nor v17 saw | 90 | 0.9 |  |  |  |  | 93 |  |  | 2026-09-25 |
| 267 guards |  |  |  |  |  |  |  | (Passed: 257, Held out breaches: 1; measured on 2026-09-25); (Passed: 256, Held out breaches: 4; measured on 2026-09-25); (Passed: 256, Held out breaches: 4; measured on 2026-09-25); (Passed: 256, Held out breaches: 4; measured on 2026-09-25) | 2 of 4 held-out breaches involve pasted or quoted content; 1 vague phrasing; 1 session-capability |  |