Re-attempted entering this SAP-15.0 cert's 4 real N/E/S/W windows (gov-API
sap_windows, 2.516 m^2 each) as separate Elmhurst Openings rows instead of
the accepted single combined South row, to test whether it closes the
residual ~2pt gap (engine 53 vs Elmhurst worksheet 51).
Reproduced live the documented elmhurst_lib.py window-grid limitation:
adding a 2nd row wipes the previously-added row's width/height to 0.00,
in both same-session and fresh-session-per-window attempts. Not fixable
via current Playwright automation. Restored the assessment to its original
single South 10.06 m^2 row (no PDFs re-downloaded, no worksheet change).
Separately confirmed our own engine already models this cert's windows
per-orientation correctly from gov-API sap_windows via
solar_gains.py's ORIENTATION_BY_SAP10_CODE cascade -- this is an Elmhurst
build-tooling ceiling, not an engine or mapper bug. sap_score=53 remains
unchanged; no calculator/mapper code was touched. Full accuracy suite
re-run clean (66 passed/67 skipped/1 xfailed + corpus test passed).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Both flagged mismatches were Elmhurst input errors (same silent-stale-value
contamination class as the earlier chimneys/wall-thickness bugs), not
parser or calculator bugs:
- cylinder_size: build_100010086084.py's water_heating() selected
DropDownListCylinderSize by raw value "2", but this DOM's option values
ARE their visible litre-band text (no "2" among them) — Playwright
silently no-ops on a non-matching value, leaving a prior cert's "Medium
(131-170L)" selection in place. Fixed to match by text ("Normal"), and
added the missing mapper dict entries (Normal/Large litre-suffixed
labels) to _ELMHURST_CYLINDER_SIZE_LABEL_TO_SAP10 (a real mapper-coverage
gap — the calculator raises UnmappedElmhurstLabel rather than silently
mis-mapping).
- boiler_flue_type: the boiler-code search dialog's combined "Balanced/Open
Flue" Table 4b category doesn't drive the separate, independently
selectable RadioButtonListFlueType field, which was left at an inherited
"Balanced" from a prior cert. Fixed space_heating() to explicitly select
"Open" to match the lodged gov-API code. This field isn't consumed by
Sap10Calculator (ML/generator-only), so it had zero effect on the SAP
score.
Re-downloaded elmhurst_summary.pdf/elmhurst_worksheet.pdf after the fixes
(Recommendations page confirmed clean). Elmhurst-PDF-inputs path moved
46->47 SAP from the cylinder-volume correction; gov-API SAP (53) and
Elmhurst's own worksheet (51) are unchanged. Full accuracy suites re-run
clean (67 passed, 67 skipped, 1 xfailed, same as before).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The Elmhurst rebuild for this cert had two carryover bugs from the shared
Playwright session: TextBoxOpenChimneys silently inherited "2" from a prior
cert's build (this cert's register lodges open_fireplaces_count=0, which
the gov-API mapper already reads correctly), and wall thickness inherited
a stale 280mm. Also fixed a real bug in the space_heating() PCDB-ref-clearing
logic: input_value() returns the string "0" when already cleared, which is
truthy in Python, causing a spurious navigate-away that broke the next
E.goto() call.
Rebuilt cleanly (chimneys=0, wall thickness=220mm per RdSAP convention for
unrecorded solid-wall thickness, since CheckBoxWallThicknessUnknown does not
persist via Playwright in this tool version). Elmhurst's own worksheet score
moves from a contaminated 46/48 to 51, closing the gap against this engine's
53-on-gov-API-inputs from ~5-7pt to ~2pt. No mapper.py change was needed --
the gov-API mapper's open_chimneys_count=0 was correct all along.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Built UPRN 100010086084 (the largest gov-API-vs-lodged outlier from the
20-cert batch) in Elmhurst's accredited RdSAP10 tool to get real ground
truth, per feedback that lodged rating isn't a valid comparison target.
Elmhurst worksheet: SAP 46-48 (orientation-dependent), vs this engine's
53 on gov-API inputs -- a real ~5-7pt residual gap. Ruled out with
direct empirical tests: total_floor_area, missing party_wall_length,
region/climate (SAP rating always uses UK-average weather per Appendix
U -- confirmed inert), boiler/secondary efficiency (exact match, 66%/
63% both sides). Confirmed real but NOT blind-fixed (need more than one
cert's evidence, and a wall-U-value patch tested the wrong direction in
isolation): a wall-thickness-unknown U-value fallback mismatch (this
engine 1.70, Elmhurst's own "unknown" resolution 1.40) and a window
U-value divergence (1.85 vs 2.52 W/m2K for the same nominal glazing
pick). Single-window orientation confirmed a ~2pt partial contributor.
Also fixes a small, separate, evidenced gap the build surfaced: the
Elmhurst summary parser didn't recognise the live tool's "Medium
(131-170L)" cylinder-size label (only the bare "Medium").
Full write-up is in the RealCertExpectation comment for this cert in
test_real_cert_sap_accuracy.py and the worklist. Residual gap is
intentionally left open for follow-up, not force-closed.
Built the property-753950 cert (worklist P4, PR #1503) in the live RdSAP-10
Online tool via Playwright: SAP 44 (E), vs our engine's 42 and the lodged
2014 cert's 45 — a normal RdSAP-2012->SAP-10.2 recalculation spread, not a
defect surfaced by the multiple_glazed_proportion fix.
Also surfaced (and documented as a caveat, not fixed here) a pre-existing
SAP-16.2 gap: the cert lodges party_wall_length_m=0 on both floors despite
being semi-detached, which Elmhurst's validation rejects and this PR's
engine change doesn't touch.
_u_brick_thin_wall_age_a_to_e's `<= 280` check put an uninsulated solid-
brick wall of exactly 280mm in the "200 to 280mm -> 1.7" row; RdSAP 10
§5.7 Table 13 (spec PDF p.41) shares 280 as an unlabelled edge between
that row and "280 to 420mm -> 1.4", so the table text alone doesn't say
which row owns it.
Found by building cert 100031768368 (280mm exactly, band C) in Elmhurst
and comparing every worksheet line to the calculator: volume/ACH/floor/
doors matched exactly, but the wall didn't (1.70 vs Elmhurst's 1.40)
despite an identical input crosswalk. Initially reverted a first attempt
at this fix when it appeared to regress an already-pinned cert
(217091901, band A, also nominally 280mm, previously "confirmed" at
U=1.70) — but that cert's build script never actually set a wall
thickness field, so its shared Elmhurst assessment had silently
inherited a stale 260mm from an earlier build. Fixed that build script
and rebuilt with the correct 280mm entry: Elmhurst's worksheet now also
gives U=1.40, matching 100031768368 and confirming the fix rather than
contradicting it.
Both certs move to (near-)exact lodged matches: 100031768368 59.12 ->
61.21 (lodged 61, within 0.5); 217091901 60.82 -> 61.59 (lodged 62,
exact). Corpus gauge 77.8% -> 78.6% within-0.5, MAE 0.636 -> 0.627 (14
corpus certs lodge exactly 280mm solid brick). TDD'd, 46 pins + full
suite green (2310 passed).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ventilation(): the cert lodges no extract-fan count and RdSAP 10 Table 5's
age-band B default is 0, but the build entered 1 — inflating Elmhurst's own
(8)/(21) infiltration lines relative to both the cert and our engine.
space_heating(): switched from the generic "Combi condens" SAP-Table dialog
code (84% efficiency, no PCDB record) to elmhurst_lib.set_pcdb_boiler(18908)
via clear_main_heating_code — resolves the exact lodged product (Worcester
Greenstar 4000, 88.70%) and its correct keep-hot classification (None),
matching the SAP engine's own PCDB lookup and removing the "Standard Combi"
keep-hot artifact the reused generic code carried.
Also corrected the file's stale docstring (described a different property
entirely — mid-terrace/250mm/DUAL meter/11 windows/lodged 63 — left over
from a copy-paste).
Verified against the refreshed worksheet: (21) shelter-adjusted ACH 0.6938
now EXACT vs our engine's ventilation_from_cert; (61) combi loss monthly
kWh matches ours to ~0.01 kWh (was carrying a ~360 kWh/yr keep-hot artifact).
Both open items from the PR #1439 handoff are resolved.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Cracks the 'PCDB disabled' blocker. The PCDF boiler reference resolves the exact
lodged boiler (e.g. 18908 -> Worcester Greenstar 4000, 88.70%) when: (1) the main
heating CODE is empty server-side (else the field + magnifying glass are
aspNetDisabled), and (2) the index is entered via REAL keyboard + Tab (a JS-set
.value doesn't survive the AutoPostBack). Now Elmhurst uses the same boiler
efficiency as the SAP engine, so worksheet vs calculator compare like-for-like.
Verified on 100061275133: Elmhurst now shows 'Efficiency of main space heating
system 1 = 88.70%' (was generic 84%). Includes updated worksheet evidence.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Third Playwright-entry error found via input reconciliation: my build left
draught-proofed doors at 0 (Elmhurst computed 33% draught proofing), but the
cert lodges percent_draughtproofed=100. Our calculator correctly uses 100% ->
infiltration 42.66 W/K; the ~19 W/K ventilation gap to Elmhurst was purely my
build error. On matched inputs our fabric+TB (109.38) equals Elmhurst's total
fabric (37) EXACTLY, confirming the calculator is RdSAP-faithful. Committed for
build-faithfulness record; the engine's +1.24 vs lodged 76 is not a bug.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Input reconciliation (Elmhurst summary vs EpcPropertyData vs raw cert) exposed
two Playwright-entry errors: lighting entered as 6 incandescent (cert has 5 LED,
100% low-energy) and wall thickness left at Elmhurst's 260mm default (cert lodges
400mm). Fixing both moved Elmhurst 67->69. Confirmed the MAPPER is faithful
(every fabric/heating/lighting field matches the raw cert) and the engine uses
the correct PCDB 18908 efficiency (88.7%). The remaining Elmhurst gap to lodged
76 is the un-enterable PCDB boiler (automation falls back to generic BGW 84% +
keep-hot); the engine (77) is faithful. Not a mapping error.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2-storey semi, solid brick 400mm with EXTERNAL insulation, 300mm loft,
suspended uninsulated ground floor, gas combi, double glazing (glazed_type 13).
Built in accredited Elmhurst RdSAP10: roof (U 0.14, 5.459 W/K) and windows
(15.32 x eff U 1.3258 = 20.311 W/K) match the engine EXACTLY; the
externally-insulated wall is engine U 0.29 vs Elmhurst 0.30, validating the
slice-5 external-insulation default (100mm added over solid brick); suspended
ground floor engine 0.68 vs Elmhurst 0.70. Elmhurst's overall SAP 67 is not
comparable (reused-assessment Standard-Combi keep-hot loss, a build artifact).
Engine 77 vs lodged 76 (within 1.24). Pinned to the observed 77.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Top-floor flat, solid brick uninsulated, uninsulated flat roof, gas combi,
SINGLE glazing (glazed_type 5). Built in accredited Elmhurst RdSAP10 and every
heat-loss element matches the engine EXACTLY: walls 53.18, flat roof 145.57,
floor 0, party 0, and — with the faithful single glazing — windows 34.7114 W/K
(8.62 m2 x eff U 4.0268). Confirms the mapper's glazed_type 5 -> single (raw U
4.8) is spec-correct per epc_codes.csv, not the double it superficially
resembles. Engine 61 vs lodged 62; the -1.18 is a cost-side residual, not a
fabric bug. Pinned to the observed engine 61.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SOLVED the window-grid blocker (per Junte's "keep one and edit, don't
delete all"): set_single_window now ADDS the fresh window (it inserts at
DOM index 0), then deletes the STALE rows BELOW it by index (high->low,
never index 0) via the per-row Delete image button + JS-clicked Yes modal.
The freshly-added row survives as the single window; no 0.00 junk strands
to block the Recommendations gate. Verified end-to-end: cert 100050881708
built, worksheet downloaded, walls (0.70) + roof (0.19) match the engine
EXACTLY. Combined with the earlier fill+Tab (area postback) + frame/gap
(validation) fixes, the per-cert loop now runs autonomously on clean
single-main certs.
Pin: 100050881708 (semi, cavity filled, pitched 225mm, suspended ground
floor, combi) engine 72 = lodged 70 +2; walls/roof accredited-Elmhurst-
matched. NB the build entered the suspended ground floor as an upper
"above unheated space" floor, so Elmhurst's worksheet SAP (61) is not a
clean comparison — the engine uses the correct §3.12 suspended-ground U
(0.74); the pin rests on the walls+roof match + lodged.
No engine change.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Continued hardening for the per-cert Elmhurst loop. Confirmed findings:
- The window Width/Height must be set via Playwright .fill() + Tab (fires
the ASP.NET AutoPostBack so the server commits the area); a JS-set of
.value bypasses the postback and Elmhurst persists a 0.00-area row.
- Frame Type (PVC) + Glazing Gap (12 mm) are REQUIRED per row or the
Recommendations gate blocks the worksheet ("Frame Type/Glazing Gap must
be entered"). add_combined_window now sets both.
KNOWN BLOCKER (documented for the next attempt): the window grid is not
yet reliably automatable when stale rows exist on the shared assessment.
New rows insert at index 0; the delete flow reliably removes only index 0
(the newest), so accumulated 0.00 junk rows BELOW cannot be targeted, and
grid reads are timing-inconsistent across postbacks. A robust handler
needs to either (a) drive the per-row Delete by matching the row's
server-side id rather than DOM index, or (b) start each build from a
freshly-created assessment (not the shared one) so the grid is empty.
Until then the per-cert loop runs only on the FIRST build after a manual
window reset. build_100050881708.py is the WIP template.
No engine change.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Support for the per-cert Elmhurst worksheet loop:
- goto()/save_close(): fall back networkidle -> domcontentloaded on
timeout so an incomplete-validation state (half-set RR, 0-area window)
no longer hangs navigation.
- add_combined_window(): select glazing first (AutoPostBack), then JS
input+change+blur the width/height with settle waits so the Area cell
computes before Add (a plain .fill() raced the postback -> 0.00 row).
- set_single_window(): leave the grid holding exactly one row, deleting
rows carried over from a prior build of the SHARED assessment (the
window grid is the only accumulating page).
- reset_transient_state(): return the shared assessment to neutral after
a run (clear RR age band, secondary = No, meter = Single).
KNOWN RESIDUAL: the window grid still intermittently persists a 0.00 area
through Save & Close (Elmhurst's "each Add wipes the previous row" +
last-row-undeletable behaviour), and the secondary-heating dialog's modal
background can intercept the cascade click. Reliable autonomous grinding of
per-cert builds needs these two hardened further; the loop currently
completes clean single-main gas certs (see the 47084930 flat-roof slice)
but stalls on secondary-heating + stale-window combinations. build_
100021924710.py is the WIP template for that harder shape.
No engine change.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A FLAT roof lodges its insulation thickness in the DEDICATED gov-EPC API
`flat_roof_insulation_thickness` field (e.g. "75mm"), leaving
`roof_insulation_thickness` None (that field is for pitched-loft joists).
`heat_transmission_from_cert` read only the latter, so a measured
flat-roof thickness was ignored and the roof billed at the uninsulated
age-band flat default (age E = 1.5) instead of its Table-16 insulated U.
Fixed by preferring `flat_roof_insulation_thickness` when the part is a
flat roof — the exact mirror of the existing rafter-thickness branch.
An "AB"/"NI" (as-built/unknown) value parses to None and keeps the
age-band default, unchanged; only measured thicknesses move.
PER-CERT ELMHURST VALIDATION (cert 47084930, top-floor flat, flat roof
75 mm): built on the lodged inputs in accredited Elmhurst RdSAP10-Online
(evidence saved: elmhurst_summary.pdf / elmhurst_worksheet.pdf). The
worksheet bills "insulated flat roof" at U 0.5 (floor 0.70 + wall 0.25
also matching the engine). The fix takes the engine roof 96.4 -> 32.1 W/K
(= 64.26 x 0.5, Elmhurst-exact), PE +47 -> +0.2, SAP 69.51 -> 74.34 =
lodged 74.
Gauge: within 77.3% -> 77.7%, SAP MAE 0.648 -> 0.641, CO2 0.074 -> 0.072,
PE 3.1 -> 2.97. Unit-pinned in test_heat_transmission
(flat_roof_insulation_thickness -> U 0.5); RealCertExpectation 47084930
= 74 (Elmhurst-validated). Also adds the build script build_47084930.py.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Built the cert in accredited Elmhurst RdSAP10-Online on the lodged register
inputs; downloaded Input Summary + SAP Worksheets (committed alongside).
Confirmed:
- Slice 5 DIRECTION validated: the externally-insulated solid-brick wall is
billed insulated by both tools (Elmhurst U 0.48, engine U 0.29), far below
the uninsulated 1.40 the pre-fix §5.8 gate produced.
- ROOF matches EXACTLY (U 0.30, 9.35 W/K).
- Engine 71.74 = lodged 71; Elmhurst current SAP 66 (its "unknown external
insulation" default is more conservative + this build used a generic combi
and no keyed wall thickness, so 66 is not a clean engine-comparison).
Follow-up lead (documented in ELMHURST_FINDINGS.md): the engine feeds the
UNMEASURED lodged wall_thickness (360 mm, measured=N) into the §5.6 formula,
whereas RdSAP §5.3 uses measured thickness / Table-3 default otherwise. The
party-wall "difference" is NOT a bug (engine 0.25 is spec-correct, Table 15
row 4). Also adds the build script scripts/hyde/build_200004296092.py.
No engine change in this commit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Captures the calc-facing ventilation-type gap (older RdSAP mappers drop
mechanical_ventilation_kind), the FE-side Main Fuel display, the sweep survivor
clusters, and the predicted-property display path — all separate from ADR-0037.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
In-container Playwright runs were flaky: the chromium renderer crashed
mid-build ("Target crashed") on the 64M /dev/shm, and login intermittently
hung. Added `--disable-dev-shm-usage` + `--no-sandbox` launch args, a
4-attempt login retry loop (domcontentloaded + explicit selector wait),
and an `ELM_GUID` env override so a per-UPRN assessment can be targeted
without editing the module. Tooling only — no calculator impact.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Corpus validation of the modelling_e2e photovoltaic_supply-as-list fix. Cert
6102-6227-8000-0083-2292 (RdSAP-20.0.0 semi, gas combi + 2× 1.14 kW PV arrays)
crashed from_rdsap_schema_20_0_0 on the measured-array list; the fix routes it
through the dict-tolerant _map_schema_21_pv. PV correctly credited: engine 61
(no PV) → 66 (+5). Built in Elmhurst (evidence: epc.json + summary + worksheet,
fabric+heating; the PV "New Technologies" Panel-details grid deferred): worksheet
55 = engine-on-Elmhurst-inputs 55 exactly → calculator faithful. The +6 engine-vs-
Elmhurst base-dwelling residual is the documented RdSAP-default gap (band-C cavity-
uninsulated suspended-floor semi). Pinned engine 66.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Corpus validation of the modelling_e2e built_form fix. Cert 8742-6624-9300-2780-4926
(SAP-Schema-16.0, ground-floor electric-storage-heater flat) omits built_form; the
mapper now derives it from dwelling_type. built_form is ML-only so the fix is
SAP-neutral: engine 66 = lodged 66 exactly. Built in Elmhurst (evidence: epc.json +
summary + worksheet): worksheet 54, engine-on-Elmhurst-inputs 53 ≈ 54 → calculator
faithful. The +12 engine-vs-Elmhurst is a build/input gap (cert size-1 small cylinder
unrepresentable in Elmhurst's Normal/110L-minimum entry → higher HW + reduced-field
16.0 defaults). Pinned engine 66.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two-pass org_ref-matched builder for property_overrides (classify via ChatGPT
into the landlord ledger, validate+apply user edits, write idempotently);
ephemeral-Postgres smoke proves the one-property chain without creds.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Every worklist UPRN now carries schema · engine SAP / lodged · flag. Tally:
64 healthy, 19 MVHR-not-credited (🚩 flag B), 6 heat-pump fuel-39 (🚩 flag A),
4 sparse/NOT MAPPABLE (⛔), 3 Elmhurst-pinned. MVHR is the largest accuracy gap.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Autonomous-run triage of the moderate eng-vs-lodged gaps resolves them into two
patterns, both flagged for owner review (not auto-fixable):
- Heat-pump fuel code 39 mis-priced as gas (over-rates; both gap directions).
- MVHR heat recovery modelled as plain extract loss → systematic UNDER-rating
(~8-12 SAP) on every full-SAP cert carrying a mechanical_vent_system_index_number.
New memory mvhr-heat-recovery-not-modelled; needs the Appendix Q / PCDB MVHR
efficiency model.
findings doc updated with the classification.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Some SAP-Schema-17.x/18.0.0 certs lodge sap_openings width/height in MILLIMETRES
mixed with metre rows in the same array (e.g. a 2025x2100 mm window beside a
3.06x1 m one). The 17.1 mapper read them all as metres → a 4.25M m2 window →
HTC in the millions → SAP clamped to 1.
Fix (TDD, datatypes/epc/domain/mapper.py): _sanitise_opening_dimension_m treats
any dimension > 50 m as mm and divides by 1000; _sap_opening_area_m2 applies it
to areas. Wired into the window, roof-window, and door-area-weighting paths.
The 3 broken certs (uprn_10093117227 / 10090317693 / 10091636031) now score
90 / 81 / 79 instead of 1.
3 RED->GREEN slices + refactor; new test class
TestFromSapSchema17_1OpeningUnitSanitisation + sap_17_1_mm_openings.json fixture;
0 new pyright errors; no regressions.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Schema coverage (datatypes/epc/domain/mapper.py):
- SAP-Schema-18.0.0: full-SAP shape ≡ 17.1 → from_sap_schema_17_1, no normalisation.
- SAP-Schema-16.0: same reduced-field 16.x path; default the omitted `tenure`
field in _normalize_sap_schema_16_x (metadata; SAP cascade never reads it).
Genuinely sparse 16.x certs (missing core fabric fields) still fail loud.
- Regression tests + sap_18_0_0.json / sap_16_0.json fixtures; 0 new pyright errors.
Autonomous triage of the worklist (scripts/hyde/autonomous_run_findings.md):
- Found + diagnosed 2 bugs (flagged, NOT fixed): (1) MAPPER — full-SAP openings
lodged in mm read as m → multi-million-m2 windows → SAP clamps to 1 (uprn_
10093117227 / 10090317693 / 10091636031); (2) CALCULATOR — database heat-pump
fuel code 39 mis-priced as gas, over-rates ~14 (uprn_10093114053).
- Most certs map within +/-4 of lodged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
