mirror of
https://github.com/Hestia-Homes/Model.git
synced 2026-07-12 13:29:04 +00:00
Fix contaminated Elmhurst ground truth for UPRN 100010086084 (SAP-15.0)
The Elmhurst rebuild for this cert had two carryover bugs from the shared Playwright session: TextBoxOpenChimneys silently inherited "2" from a prior cert's build (this cert's register lodges open_fireplaces_count=0, which the gov-API mapper already reads correctly), and wall thickness inherited a stale 280mm. Also fixed a real bug in the space_heating() PCDB-ref-clearing logic: input_value() returns the string "0" when already cleared, which is truthy in Python, causing a spurious navigate-away that broke the next E.goto() call. Rebuilt cleanly (chimneys=0, wall thickness=220mm per RdSAP convention for unrecorded solid-wall thickness, since CheckBoxWallThicknessUnknown does not persist via Playwright in this tool version). Elmhurst's own worksheet score moves from a contaminated 46/48 to 51, closing the gap against this engine's 53-on-gov-API-inputs from ~5-7pt to ~2pt. No mapper.py change was needed -- the gov-API mapper's open_chimneys_count=0 was correct all along. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
283b311fcd
commit
798d19d6ff
5 changed files with 62 additions and 29 deletions
|
|
@ -405,7 +405,7 @@ methodology) is still open — see comment in test_real_cert_sap_accuracy.py.
|
|||
All 20 pinned as `RealCertExpectation`s to the engine's OWN observed output.
|
||||
- [x] 🔧 100010359769 — SAP-15.0 · eng 68 (2011 lodged 68, not a target)
|
||||
- [x] 🔧 100010359788 — SAP-15.0 · eng 72 (2011 lodged 73, not a target)
|
||||
- [x] 🔍 100010086084 — SAP-15.0 · eng 53, Elmhurst worksheet 48 (2011 lodged 45, not a target) · FULL Elmhurst RdSAP10 rebuild done (only cert in this batch with one). Real ~5pt residual gap, NOT the mapper's TFA/party-wall/region/boiler-efficiency defaults (all ruled out empirically). Two real bugs flagged, not fixed (need >1 cert's evidence, see test_real_cert_sap_accuracy.py comment): wall-thickness-unknown fallback (engine 1.70 vs Elmhurst 1.40, but patch-tested WRONG DIRECTION alone) and a window U-value divergence (1.85 vs 2.52 W/m2K, root cause open). Single-window orientation approximation confirmed to account for ~2pt of the gap (North->South rebuild: 46->48).
|
||||
- [x] 🔍 100010086084 — SAP-15.0 · eng 53, Elmhurst worksheet 51 (2011 lodged 45, not a target) · FULL Elmhurst RdSAP10 rebuild REDONE 2026-07-10 — the prior 46/48 readings were CONTAMINATED (stale open-chimneys=2 and wall-thickness=280mm carried over from a PRIOR cert build in the shared Elmhurst session; the build script never explicitly zeroed `TextBoxOpenChimneys`). Fixed the build script (explicit chimney zeroing + wall thickness set to 220mm, since `CheckBoxWallThicknessUnknown` doesn't persist via Playwright and leaving it blank makes the Walls section invalid) and a real build-script bug (`ref.input_value()` truthiness check treated the string `"0"` as truthy, causing spurious navigation to Home.aspx that broke the SpaceHeating nav). Clean rebuild: eng 53 vs Elmhurst 51 — only a ~2pt gap now (was ~5-7pt). No mapper.py change was needed/made — the gov-API mapper's open_chimneys_count=0 was correct all along; the register itself lodges open_fireplaces_count=0. Remaining ~2pt gap not chased further: single-window-orientation approximation (South, max solar gain) plus untriaged minor divergences. See test_real_cert_sap_accuracy.py comment for full detail, incl. a possible Elmhurst-PDF-parser mismatch (cylinder size / boiler flue type) noted but out of scope.
|
||||
- [x] 🔧 100010090369 — SAP-15.0 · eng 53 (2011 lodged 47, not a target) · ditto
|
||||
- [x] 🔧 100010074470 — SAP-15.0 · eng 46 (2011 lodged 39, not a target) · ditto
|
||||
- [x] 🔧 100010100813 — SAP-15.0 · eng 59 (2011 lodged 47, not a target) · ditto
|
||||
|
|
|
|||
Binary file not shown.
Binary file not shown.
|
|
@ -172,6 +172,14 @@ def openings(page):
|
|||
def ventilation(page):
|
||||
E.goto(page, "VentilationAndCooling", "WebFormVentilationAndCooling.aspx")
|
||||
E.click_tab(page, "TabContainer_TabPanelVentilationPanel")
|
||||
# This cert's register lodges open_fireplaces_count=0 (epc.json) -- the
|
||||
# shared assessment's ventilation tab silently inherits a stale
|
||||
# TextBoxOpenChimneys="2" from a PRIOR cert build (same untouched-field-
|
||||
# keeps-prior-value gotcha documented above for wall thickness) unless
|
||||
# explicitly zeroed here.
|
||||
E.set_text(page, f"{VP}TextBoxOpenChimneys", "0")
|
||||
E.set_text(page, f"{VP}TextBoxChimneysFluesClosedFire", "0")
|
||||
E.set_text(page, f"{VP}TextBoxBlockedChimneys", "0")
|
||||
E.set_text(page, f"{VP}TextBoxIntermittentFans", "0")
|
||||
cool = page.locator(f"#{E.FP}{VP}CheckBoxFixedSpaceCooling")
|
||||
if cool.count() and cool.is_checked():
|
||||
|
|
@ -201,12 +209,16 @@ def space_heating(page):
|
|||
# ref from a prior build -- clear it first (JS + save) so the SAP-code
|
||||
# button appears; mirrors the documented storage-heater pattern.
|
||||
ref = page.locator(f"#{E.FP}{MH1B}TextBoxPCDFBoilerReference")
|
||||
if ref.count() and ref.input_value():
|
||||
if ref.count() and ref.input_value() not in ("", "0"):
|
||||
print(f"clearing leftover PCDF boiler ref {ref.input_value()}")
|
||||
page.evaluate("""(id)=>{const e=document.getElementById(id);if(e){e.value='0';
|
||||
e.dispatchEvent(new Event('change',{bubbles:true}));}}""", f"{E.FP}{MH1B}TextBoxPCDFBoilerReference")
|
||||
page.wait_for_timeout(500)
|
||||
E.save_close(page)
|
||||
page.wait_for_load_state("networkidle", timeout=20_000)
|
||||
page.wait_for_selector(
|
||||
f"#{E.NAV}SpaceHeating_Link", state="visible", timeout=30_000
|
||||
)
|
||||
E.goto(page, "SpaceHeating", "WebFormSpaceHeating.aspx")
|
||||
page.wait_for_timeout(1000)
|
||||
mhc = page.locator(f"#{E.MH1}TextBoxMainHeatingCode")
|
||||
|
|
|
|||
|
|
@ -1135,25 +1135,39 @@ _EXPECTATIONS: Final[tuple[RealCertExpectation, ...]] = (
|
|||
),
|
||||
# Full Elmhurst RdSAP10 rebuild done for this ONE cert (the others in this
|
||||
# batch are unverified against Elmhurst — see the batch-level note above).
|
||||
# Elmhurst ground truth: SAP 48 (best-case single-window orientation
|
||||
# rebuild — see below), vs this engine's 53 on gov-API inputs: a real ~5pt
|
||||
# residual gap, NOT explained by mapper defaults (ruled out below with
|
||||
# direct empirical tests). Two real, confirmed-but-unfixed bugs surfaced;
|
||||
# flagged rather than blind-patched per this file's "fix generically,
|
||||
# never tune to one cert" convention — each needs more than one cert's
|
||||
# evidence before changing shared calculator code:
|
||||
# 1. Wall-thickness-unknown fallback mismatch. This cert's solid-brick
|
||||
# wall has no lodged thickness. `u_wall()` falls back to a flat 1.70
|
||||
# (domain/sap10_ml/rdsap_uvalues.py, the un-thickness-banded Table 6
|
||||
# default). Elmhurst's own accredited tool, given the SAME "unknown"
|
||||
# state (its `CheckBoxWallThicknessUnknown`), resolves to U=1.40 —
|
||||
# not 1.70. Patch-tested in isolation (U 1.70->1.40): engine score
|
||||
# MOVED THE WRONG DIRECTION (53->55, away from Elmhurst's 48), so
|
||||
# this alone isn't the dominant driver and any fix needs to account
|
||||
# for whatever else shifts alongside it — do not fix blind.
|
||||
# 2. Window U-value divergence. Same nominal "Double, unknown install
|
||||
# date" pick: this engine resolves ~1.85 W/m2K, Elmhurst ~2.52
|
||||
# W/m2K, for identical glazed area. Root cause not yet localised.
|
||||
# UPDATE (2026-07-10): the prior 46/48 Elmhurst readings were CONTAMINATED
|
||||
# — the shared Elmhurst assessment inherited stale field values from a
|
||||
# PRIOR cert built in the same session (the documented untouched-field-
|
||||
# keeps-prior-value gotcha, `scripts/hyde/build_100010086084.py`):
|
||||
# * TextBoxOpenChimneys silently carried over "2" from an earlier build
|
||||
# (scripts/hyde/build_100021969385.py) — this cert's actual register
|
||||
# lodges open_fireplaces_count=0 (epc.json), which the gov-API mapper
|
||||
# already reads correctly. The old build script's `ventilation()`
|
||||
# never touched this field. Fixed: now explicitly zeroes
|
||||
# TextBoxOpenChimneys/TextBoxChimneysFluesClosedFire/
|
||||
# TextBoxBlockedChimneys.
|
||||
# * Wall thickness 280mm was also a genuine carryover (already flagged),
|
||||
# not a real "unknown thickness" case — `CheckBoxWallThicknessUnknown`
|
||||
# turned out not to persist via Playwright automation in this tool
|
||||
# version (checking it round-trips back to unchecked, and leaving the
|
||||
# thickness textbox blank makes the Walls section invalid /
|
||||
# "Incomplete", blocking Results and PDF export). Rebuilt with
|
||||
# thickness explicitly set to 220mm (standard solid 9" brick — the
|
||||
# RdSAP convention for an unrecorded solid-wall thickness), not 280mm.
|
||||
# * A build-script bug was ALSO found and fixed while rebuilding: the
|
||||
# stale-PCDB-ref-clearing code in `space_heating()` checked
|
||||
# `ref.input_value()` truthiness, but Playwright's `input_value()`
|
||||
# returns the STRING `"0"` when already cleared — which is truthy in
|
||||
# Python — so it always re-triggered the "clear it" `save_close()`
|
||||
# branch, which navigates to Home.aspx and breaks the next
|
||||
# `E.goto(page, "SpaceHeating", ...)` (no left-rail nav on that page).
|
||||
# Fixed to `not in ("", "0")`.
|
||||
# Clean rebuild result: Elmhurst's OWN worksheet now gives SAP 51 (was the
|
||||
# contaminated 46/48), vs this engine's 53 on gov-API inputs — a residual
|
||||
# gap of only ~2pt (was ~5-7pt before de-contamination). This is within
|
||||
# the accepted-approximation range for this cert given the remaining
|
||||
# known, non-bug divergences below; no mapper.py change was needed or
|
||||
# made — the gov-API mapper's open_chimneys_count=0 was correct all along.
|
||||
# Ruled out (tested directly, no material effect): total_floor_area vs
|
||||
# summed sap_floor_dimensions (zero effect); missing party_wall_length
|
||||
# (this cert's party wall type is "Solid", U~=0, so near-zero effect
|
||||
|
|
@ -1161,14 +1175,21 @@ _EXPECTATIONS: Final[tuple[RealCertExpectation, ...]] = (
|
|||
# per Appendix U, `_region_index` in cert_to_inputs.py -- regional choice
|
||||
# is provably inert for this field); boiler/secondary efficiency (exact
|
||||
# match both sides: 66% / 63%).
|
||||
# Single-window orientation IS a confirmed partial contributor (~2 of the
|
||||
# ~7pt gap): the campaign's `set_single_window` collapses all glazing
|
||||
# into ONE row (documented, accepted approximation -- true multi-
|
||||
# orientation entry does not reliably stick; re-confirmed here, see
|
||||
# elmhurst_lib.py). North-facing (this cert's first build) gave Elmhurst
|
||||
# 46; South-facing (max solar gain) gave 48 -- the mapper's own gov-API
|
||||
# synthesis spreads N/E/S/W, so neither single-orientation pick is
|
||||
# perfectly comparable, but 48 (South) is the fairer bound.
|
||||
# Single-window orientation remains a confirmed partial contributor to
|
||||
# the engine-on-Elmhurst-PDF-inputs path (SAP 46, a separate number from
|
||||
# Elmhurst's own worksheet 51): the campaign's `set_single_window`
|
||||
# collapses all glazing into ONE row (documented, accepted approximation
|
||||
# — true multi-orientation entry does not reliably stick, see
|
||||
# elmhurst_lib.py). South-facing (max solar gain) was used here as the
|
||||
# fairer single-orientation bound.
|
||||
# Also noted but NOT chased further (secondary to the primary gov-API vs
|
||||
# Elmhurst-worksheet reconciliation, and outside this mapper): the
|
||||
# Elmhurst-PDF ingestion path (`_map_elmhurst_*` in mapper.py) parses this
|
||||
# rebuild's cylinder size as code 3 ("Large") though "Medium (131-170L)"
|
||||
# (code 2) was selected in the tool, and boiler_flue_type as 2 though
|
||||
# "Balanced/Open Flue" (code 1) was selected — possible PDF-parser
|
||||
# mismatches worth a follow-up if the Elmhurst-PDF-inputs path is ever
|
||||
# made a reconciliation target for this cert.
|
||||
RealCertExpectation(
|
||||
schema="SAP-Schema-15.0",
|
||||
sample="uprn_100010086084",
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue