From 26e0584855f2791448ca79631484ba349b8aa066 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 20 Jul 2026 09:16:54 +0000 Subject: [PATCH] Record Elmhurst-validation provenance finding (#1649) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two outputs of validating the Wythenshawe cohort against accredited Elmhurst: 1. Harness note: the DB-oracle residual tail is not engine defects. The two worst under-raters, 6 Barry Road (ours 60.12, oracle 62) and 87 Alderue (ours 61.80, oracle 63), were re-keyed into accredited Elmhurst on our exact inputs — Elmhurst returned 60 and 62, matching our engine and both below the lodged oracle. So our engine computes correct RdSAP; the lodged certs carried a lighter input set (dropped the surveyed 17.4 m2 solid-brick alternative wall, +1.7 SAP). It's input-provenance drift, not a calculation gap — don't chase these as engine bugs or tune the engine to close them. Corrects the earlier "spec-literal alt-wall" note. 2. epc-to-elmhurst skill gotcha: give Elmhurst the RAW surveyed room height, not the engine's storey height. The mapper already adds +0.25 m to upper floors (_UPPER_FLOOR_HEIGHT_ADD_M), so feeding sap_floor_dimensions.room_height_m makes Elmhurst double-add the joist void (~1 SAP too low — it under-scored 6 Barry 60->59 until the first-floor height was corrected 2.76 -> raw 2.51). Comment/doc only; no behaviour change (harness still 89.2%/0.297, pyright clean). Co-Authored-By: Claude Opus 4.8 (1M context) --- .../epc-to-elmhurst-rdsap-inputs/SKILL.md | 10 +++++++ .../test_pashub_sap_accuracy_wythenshawe.py | 27 ++++++++++++++----- 2 files changed, 30 insertions(+), 7 deletions(-) diff --git a/.claude/skills/epc-to-elmhurst-rdsap-inputs/SKILL.md b/.claude/skills/epc-to-elmhurst-rdsap-inputs/SKILL.md index 65fa75267..85050e6ed 100644 --- a/.claude/skills/epc-to-elmhurst-rdsap-inputs/SKILL.md +++ b/.claude/skills/epc-to-elmhurst-rdsap-inputs/SKILL.md @@ -115,6 +115,16 @@ one-line reason. Absent ≠ zero everywhere — note which. - **Water heating** 903 = Electric immersion off-peak → Elmhurst "Water Heater" category, not "Boiler Circulator" (901 = from main system). - **Windows** are synthesised from `glazed_area` band × TFA — not real geometry. +- **Room height — give the RAW surveyed room height, NOT the engine's storey + height.** Elmhurst adds the RdSAP +0.25 m upper-storey (joist-void) allowance + itself. Our mapper already applies that +0.25 to upper floors + (`_UPPER_FLOOR_HEIGHT_ADD_M`), so `sap_floor_dimensions[*].room_height_m` for + a first floor is *already* `raw + 0.25`. Reading that value onto the sheet + makes Elmhurst add a SECOND 0.25 → storey height too tall → ~4-5 m² extra wall + + volume → **~1 SAP too low** (it under-scored 6 Barry Road 60→59 until the + first-floor height was corrected from 2.76 back to the raw 2.51). Pull heights + from the survey/raw `building_measurements`, not the transformed + `sap_floor_dimensions`; the ground/lowest floor takes the raw value either way. - **Age band — pick by construction YEAR, not the engine's band letter.** The live RdSAP-10 tool bands differently from `mapping.md`'s older table: it offers `…K 2007-2011`, **`L 2012-2022`**, **`M 2023 onwards`**. A 2020 build diff --git a/backend/documents_parser/tests/test_pashub_sap_accuracy_wythenshawe.py b/backend/documents_parser/tests/test_pashub_sap_accuracy_wythenshawe.py index ba9917ded..d32889708 100644 --- a/backend/documents_parser/tests/test_pashub_sap_accuracy_wythenshawe.py +++ b/backend/documents_parser/tests/test_pashub_sap_accuracy_wythenshawe.py @@ -55,13 +55,26 @@ _MANIFEST_PATH = _FIXTURES_DIR / "manifest.json" # 2026-07-19 (#1649 A-F): with the six from_site_notes fixes in place, the # **148 DB-oracle-matched fixtures** (39 UPRN + 109 postcode+house-number; 20 # unmatched, excluded) score **within-0.5 89.2% (132/148), MAE 0.297**. The -# residual tail is not fixable engine defects: the DB oracle is a *rounded -# integer* SAP graded against a continuous engine (a +0.5-to-band cluster of -# spec-correct filled-cavity dwellings sits at that rounding floor), plus a -# couple of spec-literal-vs-accredited-engine gaps (e.g. 6 Barry Road's -# uninsulated solid-brick alternative wall at the RdSAP Table 6 U=1.7 default). -# Treat floor/ceiling as a regression tripwire, not a target; re-baseline -# (coverage growth is not loosening) as PasHub sources the 20 unmatched SAPs. +# residual tail is NOT fixable engine defects — two independent causes, both +# established, not inferred: +# 1. Integer-oracle rounding: the DB oracle is a *rounded integer* SAP graded +# against our continuous engine, so a band of spec-correct dwellings sits +# just over a rounding boundary (within-1.0 is 98%). +# 2. Input-provenance drift between the site-notes survey and the lodged cert. +# The two worst under-raters, 6 Barry Road (ours 60.12, oracle 62) and 87 +# Alderue (ours 61.80, oracle 63), were re-keyed into accredited Elmhurst +# on our exact inputs: Elmhurst returned **60 and 62** — matching our +# engine, and both *below* the lodged oracle. So our engine computes +# correct RdSAP; the lodged cert simply carried a lighter input set (it +# dropped the 17.4 m² solid-brick alternative wall the surveyor recorded +# via brick-bond evidence, +1.7 SAP). Accredited engines agree on identical +# inputs — a 2-point gap is an input difference, not a calculation one. +# So do NOT chase these as engine bugs, and do NOT tune the engine to close them +# (that would move the gov-API corpus off 78.9% and diverge from accredited +# Elmhurst). Treat floor/ceiling as a regression tripwire, not a target; +# re-baseline (coverage growth is not loosening) as PasHub sources the 20 +# unmatched SAPs. The clean accuracy proof is engine-vs-accredited on identical +# inputs (done for 6 Barry / 87 Alderue), not engine-vs-lodged-integer. _MIN_WITHIN_HALF: float = 0.88 _MAX_SAP_MAE: float = 0.31