diff --git a/.claude/skills/expand-sap-accuracy-corpus/SKILL.md b/.claude/skills/expand-sap-accuracy-corpus/SKILL.md index 1d12ed5b..954264e7 100644 --- a/.claude/skills/expand-sap-accuracy-corpus/SKILL.md +++ b/.claude/skills/expand-sap-accuracy-corpus/SKILL.md @@ -151,6 +151,14 @@ Pattern: `with E.session() as (ctx,page): E.goto(...); E.set_text/set_select(... 2022" ≈ U1.4/g0.72) — the "...known data" option demands per-row Frame-Factor/ Data-Source/Location/Building-Part and fails validation. Trust the grid only after Save & Close + reload (the live grid mis-shows older rows as 0.00). +- **Window-width `.fill()` can DOUBLE the value** → "Openings: Width has to be less + or equal to 50" validation error. `_add_window`'s `page.locator(TextBoxExtWidth) + .fill(str(area))` went through Elmhurst's keystroke/numeric formatter and rendered + `11.1` as `111.10` (uprn_10010215568) — the Recommendations gate then blocks the + download. After building Openings, VERIFY the grid width (reload + read the row); + if wrong, JS-set it: `el.value='11.1'; dispatch input+change+blur`, then Save&Close + (the entry boxes mirror the single row = edit-in-place). JS-set bypasses the + formatter and sticks. Belt-and-braces: set the width by JS in `_add_window` too. - **U-value-known overrides** (walls/roof/floor) don't toggle via automation → accept the natural RdSAP age-band U-value; note the divergence for reconciliation. - **Recommendations page = validation gate.** It must show ZERO errors before the diff --git a/.claude/skills/expand-sap-accuracy-corpus/worklist.md b/.claude/skills/expand-sap-accuracy-corpus/worklist.md index 2fad4b92..3825927b 100644 --- a/.claude/skills/expand-sap-accuracy-corpus/worklist.md +++ b/.claude/skills/expand-sap-accuracy-corpus/worklist.md @@ -78,7 +78,7 @@ Skip the 🚩 MVHR / 🚩 heat-pump-fuel and ⛔ sparse certs. - [⚠] 10093388044 — SAP-17.1 · eng 87 / lodged 93 · 🚩 heat-pump fuel-39 (flagged) - [x] 10090944225 — SAP-17.0 (full-SAP GROUND-FLOOR FLAT, band L 2015, combi PCDB 17045 Ideal Logic Combi ES35, control 2110 CBI time+temp zone, NATURAL vent + 2 extract fans, AP50 3.48, party wall 3.41m² U0, measured U walls 0.27/floor 0.14/window 1.41, TFA 49.97, no cylinder/no secondary) · eng 81 / elm 80 (lodged 82) · PINNED engine 81. Built in Elmhurst (boiler 17045 via search, control CBI/2110 under Boilers→Zone control, water from primary, AP50 Blower Door 3.48+cert, Single meter). engine-on-Elmhurst-inputs 80 = worksheet 80 (exact) → faithful; eng 81 ≈ lodged 82 (−1). Remaining +1 = documented full-SAP→RdSAP residual localised to the FLOOR (cert measured U 0.14 vs Elmhurst RdSAP solid default 0.23; walls/party match) — engine correctly uses measured 0.14, NOT a mapper bug. ⚠ Initial worksheet read 79 due to a spurious leftover REA/691 electric secondary (Table-11 frac 0.1, ~1.5 SAP) the shared assessment retained — REMOVED via present=Yes→clear secondary code to empty→present=No (79→80). 🔧 Build notes: storage→combi reset (clear leftover SEB MainHeatingCode pass-1, reset meter Single); control 2110 is Boilers→**Zone control**→CBI (not Standard). No mapper change. - [⚠] 10090341811 — SAP-17.0 · eng 80 / lodged 89 · 🚩 MVHR idx 500352 not credited (flagged) -- [ ] 10010215568 — RdSAP-17.1 · eng 75 / lodged 74 +- [x] 10010215568 — RdSAP-17.1 (NATIVE; END-TERRACE BUNGALOW single-storey, band J 2003-2006, cavity as-built insulated, pitched 200mm loft, solid insulated floor, mains-gas REGULAR boiler PCDB 15100 Vaillant Ecotec plus 415 + cylinder size2/Normal-110L/foam 25mm/thermostat, control 2106 CBE, water from primary, party wall 9.8m, double glazed, natural vent, 86% LED, TFA 75.46) · eng 75 / elm 73 (lodged 74) · PINNED engine 75. Built in Elmhurst (House+1 storey bungalow, boiler 15100 via search, CBE/2106 control, cylinder Normal/foam/25mm/thermostat, water from primary, window 11.1m² double-unknown-date). engine-on-Elmhurst-inputs 74 ≈ worksheet 73 → calculator faithful; eng 75 ≈ lodged 74 (+1). +2 vs Elmhurst = documented engine-uses-lodged vs Elmhurst-RdSAP-default (wall thickness 250 default vs lodged 310, baths 2 vs lodged 1). Cylinder thickness 25mm correctly carried by RdSAP-17.1 mapper (no cylinder-gap). ⚠ window width fill bug: `.fill("11.1")` rendered 111.10 (keystroke formatter doubled it) → validation "Width ≤ 50"; fixed by JS-setting width. No mapper change. build_10010215568.py. - [⚠] 10093117227 — SAP-17.1 · eng 90 / lodged 80 · 🚩 heat-pump fuel-39 (flagged) - [⚠] 10023444170 — SAP-17.0 · eng 80 / lodged 83 · 🚩 MVHR idx 500167 not credited (flagged) - [ ] 100020980961 — SAP-16.3 · eng 66 / lodged 65 diff --git a/backend/epc_api/json_samples/real_life_examples/RdSAP-Schema-17.1/uprn_10010215568/elmhurst_summary.pdf b/backend/epc_api/json_samples/real_life_examples/RdSAP-Schema-17.1/uprn_10010215568/elmhurst_summary.pdf new file mode 100644 index 00000000..0f61a149 Binary files /dev/null and b/backend/epc_api/json_samples/real_life_examples/RdSAP-Schema-17.1/uprn_10010215568/elmhurst_summary.pdf differ diff --git a/backend/epc_api/json_samples/real_life_examples/RdSAP-Schema-17.1/uprn_10010215568/elmhurst_worksheet.pdf b/backend/epc_api/json_samples/real_life_examples/RdSAP-Schema-17.1/uprn_10010215568/elmhurst_worksheet.pdf new file mode 100644 index 00000000..d3c0c8f8 Binary files /dev/null and b/backend/epc_api/json_samples/real_life_examples/RdSAP-Schema-17.1/uprn_10010215568/elmhurst_worksheet.pdf differ diff --git a/datatypes/epc/domain/mapper.py b/datatypes/epc/domain/mapper.py index 60ec7c85..97528ed6 100644 --- a/datatypes/epc/domain/mapper.py +++ b/datatypes/epc/domain/mapper.py @@ -3296,6 +3296,16 @@ def _normalize_sap_schema_16_x(data: Dict[str, Any]) -> Dict[str, Any]: # still fail loud at from_dict — that is correct, the data is insufficient. d.setdefault("tenure", 0) + # Some 16.x certs (notably 16.3, cert 2628-…-2970) lodge `door_count` but + # omit `insulated_door_count` — RdSapSchema17_1 requires it. It is a door + # *refinement* (which of the lodged doors are the better-insulated type), not + # a core fabric field: an absent value means "no insulated doors recorded", so + # default 0 (the conservative RdSAP assumption — all doors take the standard + # door U-value; the engine's `insulated_share = insulated/door_count` is then + # 0). Mirrors the `tenure` default above. Certs that omit `door_count` itself + # still fail loud — that is correct, the fabric data is insufficient. + d.setdefault("insulated_door_count", 0) + # 16.2 lodges glazing in BOTH `multiple_glazing_type` (frequently the "ND" # not-defined sentinel) AND the windows[].description. When the numeric field # is undefined, honour an explicit "Single glazed" description so it is not diff --git a/datatypes/epc/domain/tests/test_from_sap_schema.py b/datatypes/epc/domain/tests/test_from_sap_schema.py index 64b6ee5c..7b75b84c 100644 --- a/datatypes/epc/domain/tests/test_from_sap_schema.py +++ b/datatypes/epc/domain/tests/test_from_sap_schema.py @@ -519,6 +519,25 @@ class TestFromSapSchema16_2: assert epc.uprn == 100020933894 assert Sap10Calculator().calculate(epc).sap_score == 61 # lodged 56 + def test_16_x_missing_insulated_door_count_defaults_to_zero(self) -> None: + # Some 16.x certs lodge `door_count` but omit `insulated_door_count`, + # which RdSapSchema17_1 requires — previously raised "missing required + # field 'insulated_door_count'", aborting the whole prediction cohort + # (EpcComparablePropertiesRepository.candidates_for maps every cohort + # cert in one list comprehension, so one unmappable cert killed the + # predict_epc path). Regression for cert 2628-3046-6233-4584-2970 + # (SAP-Schema-16.3, uprn 100061905751) seen via modelling_e2e property + # 730259. The normaliser defaults the missing count to 0 — no insulated + # doors recorded, the conservative RdSAP assumption. + data = load("sap_16_3.json") + del data["insulated_door_count"] + assert "insulated_door_count" not in data + + epc = EpcPropertyDataMapper.from_api_response(data) + + assert isinstance(epc, EpcPropertyData) + assert epc.insulated_door_count == 0 + def test_16_2_normalizer_does_not_mutate_caller_dict(self) -> None: # Mirror _normalize_shower_outlets' contract: the caller's dict is # untouched (deep copy), so a re-dispatch sees the original shape. diff --git a/scripts/hyde/elmhurst_download.py b/scripts/hyde/elmhurst_download.py index 3448a231..7688a30f 100644 --- a/scripts/hyde/elmhurst_download.py +++ b/scripts/hyde/elmhurst_download.py @@ -28,7 +28,7 @@ SESSION_DIR = HERE / ".elmhurst-session" SAMPLE_DIR = ( HERE.parent.parent / "backend/epc_api/json_samples/real_life_examples" - / "SAP-Schema-17.0/uprn_10090944225" + / "RdSAP-Schema-17.1/uprn_10010215568" ) ASSESSMENT_GUID = "B44A0DB4-4C08-4241-B818-86F060172105" diff --git a/tests/domain/sap10_calculator/test_real_cert_sap_accuracy.py b/tests/domain/sap10_calculator/test_real_cert_sap_accuracy.py index 681117e0..aa8344b2 100644 --- a/tests/domain/sap10_calculator/test_real_cert_sap_accuracy.py +++ b/tests/domain/sap10_calculator/test_real_cert_sap_accuracy.py @@ -557,6 +557,30 @@ _EXPECTATIONS: Final[tuple[RealCertExpectation, ...]] = ( cert_num="2298-4036-7306-3186-4930", sap_score=81, ), + # UPRN 10010215568 → cert 9978-7098-7226-2633-3994. RdSAP-Schema-17.1 — NATIVE + # RdSAP, END-TERRACE BUNGALOW (single-storey), band J (2003-2006), cavity wall + # as-built insulated (assumed), pitched 200 mm loft, solid insulated floor, + # mains-gas REGULAR boiler (PCDB 15100 Vaillant Ecotec plus 415) + cylinder + # (size 2 = Normal/110 L, factory foam 25 mm, thermostat), control 2106 (CBE + # programmer + room thermostat + TRVs), water from primary, party wall 9.8 m, + # double glazed, natural ventilation, 86% LED, TFA 75.46. Lodged 74; engine 75. + # Built in Elmhurst RdSAP10 on the lodged inputs (evidence saved: + # elmhurst_summary.pdf / elmhurst_worksheet.pdf): worksheet SAP 73. Engine on + # Elmhurst's own parsed inputs = 74 ≈ worksheet 73 → calculator faithful (~1). + # The cylinder insulation thickness (25 mm) IS carried by the RdSAP-17.1 mapper + # (cylinder_insulation_thickness_mm=25) — no mapper gap here (cf. the RdSAP-17.0 + # cylinder-thickness drop on uprn_68151071). The residual engine 75 vs Elmhurst + # 73 (+2) is the documented engine-uses-lodged vs Elmhurst-RdSAP-default gap: + # Elmhurst defaulted the cavity wall thickness to 250 mm (lodged 310 mm) and + # number of baths to 2 (lodged 1, lifting Elmhurst's HW demand). Engine 75 ≈ + # lodged 74 (+1) is the validated anchor. PINNED to the observed engine 75 — + # mapping deliberately untuned. + RealCertExpectation( + schema="RdSAP-Schema-17.1", + sample="uprn_10010215568", + cert_num="9978-7098-7226-2633-3994", + sap_score=75, + ), )