junte/Model
1
0
Fork 0
mirror of https://github.com/Hestia-Homes/Model.git synced 2026-06-30 13:10:47 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Log 2026-06-23 14:48 modelling_e2e failure sweep + e2e candidates in corpus worklist

Browse source 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jun-te Kim 2026-06-23 15:41:57 +00:00
parent 75341da42d
commit 451f0773e0
1 changed files with 40 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

40
.claude/skills/expand-sap-accuracy-corpus/worklist.md
View file

@ -120,6 +120,46 @@ malformed cert (e.g. PV-as-list AttributeError) is skipped+reported too, not jus
missing-field ValueErrors; transient `EpcApiError` (subclasses `Exception`) still
propagates. +regression test.
### 🔧🔍 modelling_e2e failure sweep — later run (2026-06-23 14:48 BST, portfolio 796, scenario 1268)
A second `modelling_e2e` run (filter `task_source='modelling_e2e' AND job_started >
'2026-06-23 14:48:01.806 +0100'`): **200 tasks, 30 FAILED (15%) → 22 unique props**
(8 retried once). The DB only stores bare `property_id` in subtask `outputs`
(`cloud_logs_url` empty) — root causes recovered by replaying each through
`scripts.run_modelling_e2e --scenario-id 1268`. Four root-cause clusters:
**① ✅ FIXED — `KeyError: BuildingPartIdentifier.EXTENSION_1` (14 props, the dominant
failure).** A **Landlord Override** targeted `extension_1` but the lodged/predicted EPC
had no such part; `overlay_applicator.apply_simulations` indexed `parts_by_id[identifier]`
unguarded → crashed the whole property. Fixed: skip the orphaned part (`.get` + `None`
guard). +regression test `test_override_targeting_a_part_absent_from_the_epc_is_skipped`.
Not a mapper gap — a modelling-overlay bug. The 14 (pid / uprn):
710295/100020458237, 710482/100020450179, 713040/100020404702, 715575/100020397529,
715894/100020604961, 717435/22010468, 720142/100020383544, 720560/100021921443,
721241/100020453651, 724945/100021915421, 725415/100020404036, 726517/100020631307,
726592/100021918195, 730800/100021920273. (Observed post-fix eng: 720142 69.2,
710295 63.2→69.8, 720560 61.8→70.4 predicted.)
**② 🔍 mapper/cascade gap — `UnmappedSapCode: fuel_code 10`** (1 prop, 730259 / uprn
100061905741). EPC-less → prediction synthesises a cohort EPC carrying `fuel_code 10`,
absent from the calculator's cascade dispatch dict → FATAL (it's the target prop's own
fuel, not a skippable cohort cert). FOR KHALIM: add fuel-code-10 to the cascade.
**③ ⛔ not-predictable — empty/unresolved postcode cohort** (3 props): 714585/100020612517
(`CR0 0DD`), 718580/10013149015 (`BR6 6BS`), 723881/22005280 (`BN41 2TP`). EPC-less AND
cohort empty after filtering → Prediction Path 3 can't fire. Coverage gap, not a crash bug.
**④ ⚠ transient — pass clean on replay** (4 props): 712401/100020394694 (eng 75.3),
718138/100020397707 (eng 63.5→69.2), 720844/100020472603 (eng 71.8), 723648/100020480302
(eng 70.8). Likely a flaky EPC/Solar API call during the batch — no code defect.
**E2E candidates to pin (best = the lodged-EPC ones, clean to build in Elmhurst):**
`[ ]` 100020383544 (pid 720142, eng 69.2) · `[ ]` 100020458237 (710295) · `[ ]`
100020404702 (713040) · `[ ]` 100020450179 (710482) · `[ ]` 100021915421 (724945) ·
`[ ]` 100021920273 (730800) · `[ ]` 100020394694 (712401, eng 75.3) · `[ ]`
100020397707 (718138) · `[ ]` 100020472603 (720844, eng 71.8) · `[ ]` 100020480302
(723648, eng 70.8). (Schema not populated on the gov-API objects — confirm when keying
each into Elmhurst.)
### 📋 PLAN — close the 8 modelling_e2e mapping gaps (2026-06-23 run, portfolio 796)
The 8 failed prediction targets reduce to **5 distinct mapper-gap classes** (the fix
targets). Per class: fix the mapper GENERICALLY, guard with BOTH the RdSAP-21.0.1