Agentic Coding LabData as of 2026-10-16 00:24 UTC, 1,273 runs, commit 4af5566

Research Question: RQ-ptdd-verification-split

Does splitting the PTDD v1 test list into driving tests and a trailing verification group keep Correctness, safety net and code quality at least at the v1 level on Opus 5.5 and GPT-6.1 Sol, while making the phase-chain discipline readable for a test-list workflow?

Factor(s)
model × workflow
Control(s)
  • kata_base: claim-office
  • prompt: example-mapping
Runs
11 runs, 2026-10-01 → 2026-10-06; 2 of 4 cells at 5 replicates
Source
View on GitHub

claim-office-example-mapping

↑ higher is better · ↓ lower is better · ○ informative
n, run datesCorrectnessSizeTest qualityCode qualityTDD disciplineCost
↑ ↑ ○ ○ ↑ ○ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ○ ○ ○ ○ ○ ○ ○ ⚠ ↑ ⚠ ○ ○ ↓ ↓ ↓
model: opus-5-5-no-thinking · exact-ptdd-v1-cctest listn = 5, 2026-10-01100% ± 0100%5/547.8 ± 1.64222 ± 24.1494% ± 5 🏆124 ± 4.907.80 ± 6.30 🏆632 ± 35.980.00 ± 0.002.40 ± 0.55 🏆1.43 ± 0.21 🏆3.40 ± 0.55 🏆4.81 ± 0.62 🏆n/cn/cn/c1.00 ± 0.0024.2 ± 1.6416.6 ± 2.4123.8 ± 1.3063% ± 12.00 ± 4.472.00 ± 4.4715.4 ± 1.3 min 🏆24.4M ± 1.8M$9.79 ± 0.49
model: opus-5-5-no-thinking · exact-ptdd-v1.2-verification-split-cctest list0 runsNot runNot runNot runNot runNot runNot runNot runNot runNot runNot runNot runNot runNot runNot runNot runNot runNot runNot runNot runNot runNot runNot runNot runNot runNot runNot run
model: gpt-6-1-sol-codex · exact-ptdd-v1-pitest listn = 5, 2026-10-04100% ± 0100%5/548.8 ± 1.10140 ± 13.8554% ± 49 🏆113 ± 3.8151.4 ± 54.45 🏆561 ± 18.050.00 ± 0.003.00 ± 1.41 🏆1.47 ± 0.32 🏆3.80 ± 1.10 🏆4.56 ± 0.41 🏆n/cn/cn/c1.00 ± 0.0025.0 ± 1.0019.6 ± 3.3623.0 ± 1.4163% ± 196.0 ± 1.4197.6 ± 2.1936.2 ± 3.2 min9.0M ± 3.4M 🏆$1.47 ± 0.58
model: gpt-6-1-sol-codex · exact-ptdd-v1.2-verification-split-pitest listn = 1 of 5, 2026-10-06100%100%1/157132n/cn/cn/c476 🏆0.003.001.40 🏆4.004.56 🏆370.000.001.0017190.0088% 🏆11311420.0 min10.3M 🏆$0.00 🏆
Runs behind this table