Fix contaminated Elmhurst ground truth for UPRN 100010086084 (SAP-15.0)

The Elmhurst rebuild for this cert had two carryover bugs from the shared
Playwright session: TextBoxOpenChimneys silently inherited "2" from a prior
cert's build (this cert's register lodges open_fireplaces_count=0, which
the gov-API mapper already reads correctly), and wall thickness inherited
a stale 280mm. Also fixed a real bug in the space_heating() PCDB-ref-clearing
logic: input_value() returns the string "0" when already cleared, which is
truthy in Python, causing a spurious navigate-away that broke the next
E.goto() call.

Rebuilt cleanly (chimneys=0, wall thickness=220mm per RdSAP convention for
unrecorded solid-wall thickness, since CheckBoxWallThicknessUnknown does not
persist via Playwright in this tool version). Elmhurst's own worksheet score
moves from a contaminated 46/48 to 51, closing the gap against this engine's
53-on-gov-API-inputs from ~5-7pt to ~2pt. No mapper.py change was needed --
the gov-API mapper's open_chimneys_count=0 was correct all along.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Jun-te Kim 2026-07-10 08:36:03 +00:00
parent 283b311fcd
commit 798d19d6ff
5 changed files with 62 additions and 29 deletions

View file

@ -405,7 +405,7 @@ methodology) is still open — see comment in test_real_cert_sap_accuracy.py.
All 20 pinned as `RealCertExpectation`s to the engine's OWN observed output.
- [x] 🔧 100010359769 — SAP-15.0 · eng 68 (2011 lodged 68, not a target)
- [x] 🔧 100010359788 — SAP-15.0 · eng 72 (2011 lodged 73, not a target)
- [x] 🔍 100010086084 — SAP-15.0 · eng 53, Elmhurst worksheet 48 (2011 lodged 45, not a target) · FULL Elmhurst RdSAP10 rebuild done (only cert in this batch with one). Real ~5pt residual gap, NOT the mapper's TFA/party-wall/region/boiler-efficiency defaults (all ruled out empirically). Two real bugs flagged, not fixed (need >1 cert's evidence, see test_real_cert_sap_accuracy.py comment): wall-thickness-unknown fallback (engine 1.70 vs Elmhurst 1.40, but patch-tested WRONG DIRECTION alone) and a window U-value divergence (1.85 vs 2.52 W/m2K, root cause open). Single-window orientation approximation confirmed to account for ~2pt of the gap (North->South rebuild: 46->48).
- [x] 🔍 100010086084 — SAP-15.0 · eng 53, Elmhurst worksheet 51 (2011 lodged 45, not a target) · FULL Elmhurst RdSAP10 rebuild REDONE 2026-07-10 — the prior 46/48 readings were CONTAMINATED (stale open-chimneys=2 and wall-thickness=280mm carried over from a PRIOR cert build in the shared Elmhurst session; the build script never explicitly zeroed `TextBoxOpenChimneys`). Fixed the build script (explicit chimney zeroing + wall thickness set to 220mm, since `CheckBoxWallThicknessUnknown` doesn't persist via Playwright and leaving it blank makes the Walls section invalid) and a real build-script bug (`ref.input_value()` truthiness check treated the string `"0"` as truthy, causing spurious navigation to Home.aspx that broke the SpaceHeating nav). Clean rebuild: eng 53 vs Elmhurst 51 — only a ~2pt gap now (was ~5-7pt). No mapper.py change was needed/made — the gov-API mapper's open_chimneys_count=0 was correct all along; the register itself lodges open_fireplaces_count=0. Remaining ~2pt gap not chased further: single-window-orientation approximation (South, max solar gain) plus untriaged minor divergences. See test_real_cert_sap_accuracy.py comment for full detail, incl. a possible Elmhurst-PDF-parser mismatch (cylinder size / boiler flue type) noted but out of scope.
- [x] 🔧 100010090369 — SAP-15.0 · eng 53 (2011 lodged 47, not a target) · ditto
- [x] 🔧 100010074470 — SAP-15.0 · eng 46 (2011 lodged 39, not a target) · ditto
- [x] 🔧 100010100813 — SAP-15.0 · eng 59 (2011 lodged 47, not a target) · ditto

View file

@ -172,6 +172,14 @@ def openings(page):
def ventilation(page):
E.goto(page, "VentilationAndCooling", "WebFormVentilationAndCooling.aspx")
E.click_tab(page, "TabContainer_TabPanelVentilationPanel")
# This cert's register lodges open_fireplaces_count=0 (epc.json) -- the
# shared assessment's ventilation tab silently inherits a stale
# TextBoxOpenChimneys="2" from a PRIOR cert build (same untouched-field-
# keeps-prior-value gotcha documented above for wall thickness) unless
# explicitly zeroed here.
E.set_text(page, f"{VP}TextBoxOpenChimneys", "0")
E.set_text(page, f"{VP}TextBoxChimneysFluesClosedFire", "0")
E.set_text(page, f"{VP}TextBoxBlockedChimneys", "0")
E.set_text(page, f"{VP}TextBoxIntermittentFans", "0")
cool = page.locator(f"#{E.FP}{VP}CheckBoxFixedSpaceCooling")
if cool.count() and cool.is_checked():
@ -201,12 +209,16 @@ def space_heating(page):
# ref from a prior build -- clear it first (JS + save) so the SAP-code
# button appears; mirrors the documented storage-heater pattern.
ref = page.locator(f"#{E.FP}{MH1B}TextBoxPCDFBoilerReference")
if ref.count() and ref.input_value():
if ref.count() and ref.input_value() not in ("", "0"):
print(f"clearing leftover PCDF boiler ref {ref.input_value()}")
page.evaluate("""(id)=>{const e=document.getElementById(id);if(e){e.value='0';
e.dispatchEvent(new Event('change',{bubbles:true}));}}""", f"{E.FP}{MH1B}TextBoxPCDFBoilerReference")
page.wait_for_timeout(500)
E.save_close(page)
page.wait_for_load_state("networkidle", timeout=20_000)
page.wait_for_selector(
f"#{E.NAV}SpaceHeating_Link", state="visible", timeout=30_000
)
E.goto(page, "SpaceHeating", "WebFormSpaceHeating.aspx")
page.wait_for_timeout(1000)
mhc = page.locator(f"#{E.MH1}TextBoxMainHeatingCode")

View file

@ -1135,25 +1135,39 @@ _EXPECTATIONS: Final[tuple[RealCertExpectation, ...]] = (
),
# Full Elmhurst RdSAP10 rebuild done for this ONE cert (the others in this
# batch are unverified against Elmhurst — see the batch-level note above).
# Elmhurst ground truth: SAP 48 (best-case single-window orientation
# rebuild — see below), vs this engine's 53 on gov-API inputs: a real ~5pt
# residual gap, NOT explained by mapper defaults (ruled out below with
# direct empirical tests). Two real, confirmed-but-unfixed bugs surfaced;
# flagged rather than blind-patched per this file's "fix generically,
# never tune to one cert" convention — each needs more than one cert's
# evidence before changing shared calculator code:
# 1. Wall-thickness-unknown fallback mismatch. This cert's solid-brick
# wall has no lodged thickness. `u_wall()` falls back to a flat 1.70
# (domain/sap10_ml/rdsap_uvalues.py, the un-thickness-banded Table 6
# default). Elmhurst's own accredited tool, given the SAME "unknown"
# state (its `CheckBoxWallThicknessUnknown`), resolves to U=1.40 —
# not 1.70. Patch-tested in isolation (U 1.70->1.40): engine score
# MOVED THE WRONG DIRECTION (53->55, away from Elmhurst's 48), so
# this alone isn't the dominant driver and any fix needs to account
# for whatever else shifts alongside it — do not fix blind.
# 2. Window U-value divergence. Same nominal "Double, unknown install
# date" pick: this engine resolves ~1.85 W/m2K, Elmhurst ~2.52
# W/m2K, for identical glazed area. Root cause not yet localised.
# UPDATE (2026-07-10): the prior 46/48 Elmhurst readings were CONTAMINATED
# — the shared Elmhurst assessment inherited stale field values from a
# PRIOR cert built in the same session (the documented untouched-field-
# keeps-prior-value gotcha, `scripts/hyde/build_100010086084.py`):
# * TextBoxOpenChimneys silently carried over "2" from an earlier build
# (scripts/hyde/build_100021969385.py) — this cert's actual register
# lodges open_fireplaces_count=0 (epc.json), which the gov-API mapper
# already reads correctly. The old build script's `ventilation()`
# never touched this field. Fixed: now explicitly zeroes
# TextBoxOpenChimneys/TextBoxChimneysFluesClosedFire/
# TextBoxBlockedChimneys.
# * Wall thickness 280mm was also a genuine carryover (already flagged),
# not a real "unknown thickness" case — `CheckBoxWallThicknessUnknown`
# turned out not to persist via Playwright automation in this tool
# version (checking it round-trips back to unchecked, and leaving the
# thickness textbox blank makes the Walls section invalid /
# "Incomplete", blocking Results and PDF export). Rebuilt with
# thickness explicitly set to 220mm (standard solid 9" brick — the
# RdSAP convention for an unrecorded solid-wall thickness), not 280mm.
# * A build-script bug was ALSO found and fixed while rebuilding: the
# stale-PCDB-ref-clearing code in `space_heating()` checked
# `ref.input_value()` truthiness, but Playwright's `input_value()`
# returns the STRING `"0"` when already cleared — which is truthy in
# Python — so it always re-triggered the "clear it" `save_close()`
# branch, which navigates to Home.aspx and breaks the next
# `E.goto(page, "SpaceHeating", ...)` (no left-rail nav on that page).
# Fixed to `not in ("", "0")`.
# Clean rebuild result: Elmhurst's OWN worksheet now gives SAP 51 (was the
# contaminated 46/48), vs this engine's 53 on gov-API inputs — a residual
# gap of only ~2pt (was ~5-7pt before de-contamination). This is within
# the accepted-approximation range for this cert given the remaining
# known, non-bug divergences below; no mapper.py change was needed or
# made — the gov-API mapper's open_chimneys_count=0 was correct all along.
# Ruled out (tested directly, no material effect): total_floor_area vs
# summed sap_floor_dimensions (zero effect); missing party_wall_length
# (this cert's party wall type is "Solid", U~=0, so near-zero effect
@ -1161,14 +1175,21 @@ _EXPECTATIONS: Final[tuple[RealCertExpectation, ...]] = (
# per Appendix U, `_region_index` in cert_to_inputs.py -- regional choice
# is provably inert for this field); boiler/secondary efficiency (exact
# match both sides: 66% / 63%).
# Single-window orientation IS a confirmed partial contributor (~2 of the
# ~7pt gap): the campaign's `set_single_window` collapses all glazing
# into ONE row (documented, accepted approximation -- true multi-
# orientation entry does not reliably stick; re-confirmed here, see
# elmhurst_lib.py). North-facing (this cert's first build) gave Elmhurst
# 46; South-facing (max solar gain) gave 48 -- the mapper's own gov-API
# synthesis spreads N/E/S/W, so neither single-orientation pick is
# perfectly comparable, but 48 (South) is the fairer bound.
# Single-window orientation remains a confirmed partial contributor to
# the engine-on-Elmhurst-PDF-inputs path (SAP 46, a separate number from
# Elmhurst's own worksheet 51): the campaign's `set_single_window`
# collapses all glazing into ONE row (documented, accepted approximation
# — true multi-orientation entry does not reliably stick, see
# elmhurst_lib.py). South-facing (max solar gain) was used here as the
# fairer single-orientation bound.
# Also noted but NOT chased further (secondary to the primary gov-API vs
# Elmhurst-worksheet reconciliation, and outside this mapper): the
# Elmhurst-PDF ingestion path (`_map_elmhurst_*` in mapper.py) parses this
# rebuild's cylinder size as code 3 ("Large") though "Medium (131-170L)"
# (code 2) was selected in the tool, and boiler_flue_type as 2 though
# "Balanced/Open Flue" (code 1) was selected — possible PDF-parser
# mismatches worth a follow-up if the Elmhurst-PDF-inputs path is ever
# made a reconciliation target for this cert.
RealCertExpectation(
schema="SAP-Schema-15.0",
sample="uprn_100010086084",