diff --git a/.claude/skills/expand-sap-accuracy-corpus/worklist.md b/.claude/skills/expand-sap-accuracy-corpus/worklist.md index cdd3abbd..58f3b984 100644 --- a/.claude/skills/expand-sap-accuracy-corpus/worklist.md +++ b/.claude/skills/expand-sap-accuracy-corpus/worklist.md @@ -120,6 +120,53 @@ malformed cert (e.g. PV-as-list AttributeError) is skipped+reported too, not jus missing-field ValueErrors; transient `EpcApiError` (subclasses `Exception`) still propagates. +regression test. +### πŸ”§πŸ” modelling_e2e failure sweep β€” later run (2026-06-23 14:48 BST, portfolio 796, scenario 1268) +A second `modelling_e2e` run (filter `task_source='modelling_e2e' AND job_started > +'2026-06-23 14:48:01.806 +0100'`): **200 tasks, 30 FAILED (15%) β†’ 22 unique props** +(8 retried once). The DB only stores bare `property_id` in subtask `outputs` +(`cloud_logs_url` empty) β€” root causes recovered by replaying each through +`scripts.run_modelling_e2e --scenario-id 1268`. Four root-cause clusters: + +**β‘  βœ… FIXED β€” `KeyError: BuildingPartIdentifier.EXTENSION_1` (14 props, the dominant +failure).** A **Landlord Override** targeted `extension_1` but the lodged/predicted EPC +had no such part; `overlay_applicator.apply_simulations` indexed `parts_by_id[identifier]` +unguarded β†’ crashed the whole property. Root cause is a **numbering mismatch**: the +override's `building_part` is a positional index (0=main, 1=extension 1…, ADR-0004), but +the gov-API EPC can label that slot differently (720142's 2nd part lodged as `other`, no +`extension_1`). Fix (per product decision β€” *honour the override, don't drop it*): +`_resolve_part` falls back to the EPC's part **at that position** when the semantic label +is absent, so the landlord's correction lands; only a position the EPC models no part at +is skipped (no geometry to model a wholly-absent part). +regression tests +`test_override_for_an_absent_semantic_part_lands_on_the_part_at_that_position` / +`test_override_with_no_part_at_that_position_is_skipped`. Not a mapper gap β€” a +modelling-overlay bug. The 14 (pid / uprn): +710295/100020458237, 710482/100020450179, 713040/100020404702, 715575/100020397529, +715894/100020604961, 717435/22010468, 720142/100020383544, 720560/100021921443, +721241/100020453651, 724945/100021915421, 725415/100020404036, 726517/100020631307, +726592/100021918195, 730800/100021920273. (Observed post-fix eng: 720142 69.2, +710295 63.2β†’69.8, 720560 61.8β†’70.4 predicted.) + +**β‘‘ πŸ” mapper/cascade gap β€” `UnmappedSapCode: fuel_code 10`** (1 prop, 730259 / uprn +100061905741). EPC-less β†’ prediction synthesises a cohort EPC carrying `fuel_code 10`, +absent from the calculator's cascade dispatch dict β†’ FATAL (it's the target prop's own +fuel, not a skippable cohort cert). FOR KHALIM: add fuel-code-10 to the cascade. + +**β‘’ β›” not-predictable β€” empty/unresolved postcode cohort** (3 props): 714585/100020612517 +(`CR0 0DD`), 718580/10013149015 (`BR6 6BS`), 723881/22005280 (`BN41 2TP`). EPC-less AND +cohort empty after filtering β†’ Prediction Path 3 can't fire. Coverage gap, not a crash bug. + +**β‘£ ⚠ transient β€” pass clean on replay** (4 props): 712401/100020394694 (eng 75.3), +718138/100020397707 (eng 63.5β†’69.2), 720844/100020472603 (eng 71.8), 723648/100020480302 +(eng 70.8). Likely a flaky EPC/Solar API call during the batch β€” no code defect. + +**E2E candidates to pin (best = the lodged-EPC ones, clean to build in Elmhurst):** +`[ ]` 100020383544 (pid 720142, eng 69.2) Β· `[ ]` 100020458237 (710295) Β· `[ ]` +100020404702 (713040) Β· `[ ]` 100020450179 (710482) Β· `[ ]` 100021915421 (724945) Β· +`[ ]` 100021920273 (730800) Β· `[ ]` 100020394694 (712401, eng 75.3) Β· `[ ]` +100020397707 (718138) Β· `[ ]` 100020472603 (720844, eng 71.8) Β· `[ ]` 100020480302 +(723648, eng 70.8). (Schema not populated on the gov-API objects β€” confirm when keying +each into Elmhurst.) + ### πŸ“‹ PLAN β€” close the 8 modelling_e2e mapping gaps (2026-06-23 run, portfolio 796) The 8 failed prediction targets reduce to **5 distinct mapper-gap classes** (the fix targets). Per class: fix the mapper GENERICALLY, guard with BOTH the RdSAP-21.0.1 diff --git a/.devcontainer/backend/requirements.txt b/.devcontainer/backend/requirements.txt index 2db3710a..78131981 100644 --- a/.devcontainer/backend/requirements.txt +++ b/.devcontainer/backend/requirements.txt @@ -24,6 +24,7 @@ ipykernel>=6.25,<7 dotenv psycopg[binary] pytest-postgresql +moto[s3,sqs]==5.0.28 # mock_aws (moto 5.x) for S3/SQS in orchestration tests # Formatting black==26.1.0 boto3-stubs diff --git a/backend/epc_api/json_samples/real_life_examples/RdSAP-Schema-20.0.0/uprn_22086693/elmhurst_summary.pdf b/backend/epc_api/json_samples/real_life_examples/RdSAP-Schema-20.0.0/uprn_22086693/elmhurst_summary.pdf new file mode 100644 index 00000000..56569fde Binary files /dev/null and b/backend/epc_api/json_samples/real_life_examples/RdSAP-Schema-20.0.0/uprn_22086693/elmhurst_summary.pdf differ diff --git a/backend/epc_api/json_samples/real_life_examples/RdSAP-Schema-20.0.0/uprn_22086693/elmhurst_worksheet.pdf b/backend/epc_api/json_samples/real_life_examples/RdSAP-Schema-20.0.0/uprn_22086693/elmhurst_worksheet.pdf new file mode 100644 index 00000000..eb7b60b6 Binary files /dev/null and b/backend/epc_api/json_samples/real_life_examples/RdSAP-Schema-20.0.0/uprn_22086693/elmhurst_worksheet.pdf differ diff --git a/backend/epc_api/json_samples/real_life_examples/SAP-Schema-16.0/uprn_10070004512/elmhurst_summary.pdf b/backend/epc_api/json_samples/real_life_examples/SAP-Schema-16.0/uprn_10070004512/elmhurst_summary.pdf new file mode 100644 index 00000000..eddc4314 Binary files /dev/null and b/backend/epc_api/json_samples/real_life_examples/SAP-Schema-16.0/uprn_10070004512/elmhurst_summary.pdf differ diff --git a/backend/epc_api/json_samples/real_life_examples/SAP-Schema-16.0/uprn_10070004512/elmhurst_worksheet.pdf b/backend/epc_api/json_samples/real_life_examples/SAP-Schema-16.0/uprn_10070004512/elmhurst_worksheet.pdf new file mode 100644 index 00000000..b71391c8 Binary files /dev/null and b/backend/epc_api/json_samples/real_life_examples/SAP-Schema-16.0/uprn_10070004512/elmhurst_worksheet.pdf differ diff --git a/backend/epc_api/json_samples/real_life_examples/SAP-Schema-16.0/uprn_10070004512/epc.json b/backend/epc_api/json_samples/real_life_examples/SAP-Schema-16.0/uprn_10070004512/epc.json new file mode 100644 index 00000000..63dbcfab --- /dev/null +++ b/backend/epc_api/json_samples/real_life_examples/SAP-Schema-16.0/uprn_10070004512/epc.json @@ -0,0 +1,281 @@ +{ + "uprn": 10070004512, + "roofs": [ + { + "description": "(another dwelling above)", + "energy_efficiency_rating": 0, + "environmental_efficiency_rating": 0 + } + ], + "walls": [ + { + "description": "Cavity wall, filled cavity", + "energy_efficiency_rating": 4, + "environmental_efficiency_rating": 4 + } + ], + "floors": [ + { + "description": "To external air, no insulation (assumed)", + "energy_efficiency_rating": 0, + "environmental_efficiency_rating": 0 + } + ], + "status": "entered", + "windows": [ + { + "description": "Fully double glazed", + "energy_efficiency_rating": 3, + "environmental_efficiency_rating": 3 + } + ], + "lighting": { + "description": "Low energy lighting in all fixed outlets", + "energy_efficiency_rating": 5, + "environmental_efficiency_rating": 5 + }, + "postcode": "BR1 4QF", + "hot_water": { + "description": "Electric immersion, off-peak", + "energy_efficiency_rating": 3, + "environmental_efficiency_rating": 1 + }, + "post_town": "BROMLEY", + "created_at": "2012-04-20 10:59:22.000000", + "door_count": 1, + "glazed_area": 1, + "region_code": 14, + "report_type": 2, + "sap_heating": { + "wwhrs": { + "rooms_with_bath_and_or_shower": 1, + "rooms_with_mixer_shower_no_bath": 0, + "rooms_with_bath_and_mixer_shower": 0 + }, + "cylinder_size": 1, + "water_heating_code": 903, + "water_heating_fuel": 29, + "main_heating_details": [ + { + "has_fghrs": "N", + "main_fuel_type": 29, + "heat_emitter_type": 0, + "main_heating_number": 1, + "main_heating_control": 2401, + "main_heating_category": 7, + "main_heating_fraction": 1, + "sap_main_heating_code": 402, + "main_heating_data_source": 2 + } + ], + "immersion_heating_type": 1, + "has_fixed_air_conditioning": "false" + }, + "sap_version": 9.91, + "schema_type": "SAP-Schema-16.0", + "uprn_source": "Energy Assessor", + "country_code": "EAW", + "main_heating": [ + { + "description": "Electric storage heaters", + "energy_efficiency_rating": 3, + "environmental_efficiency_rating": 1 + } + ], + "dwelling_type": "Ground-floor flat", + "language_code": 1, + "property_type": 2, + "address_line_1": "54a, Boyland Road", + "schema_version": "LIG-16.0", + "assessment_type": "RdSAP", + "completion_date": "2012-04-20", + "inspection_date": "2012-04-20", + "extensions_count": 0, + "measurement_type": 1, + "sap_flat_details": { + "level": 1, + "top_storey": "N", + "flat_location": 0, + "heat_loss_corridor": 0 + }, + "total_floor_area": 33, + "transaction_type": 3, + "conservatory_type": 1, + "heated_room_count": 2, + "registration_date": "2012-04-20", + "restricted_access": 1, + "sap_energy_source": { + "main_gas": "N", + "meter_type": 1, + "photovoltaic_supply": { + "percent_roof_area": 0 + }, + "wind_turbines_count": 0, + "wind_turbines_terrain_type": 1 + }, + "secondary_heating": { + "description": "Portable electric heaters (assumed)", + "energy_efficiency_rating": 0, + "environmental_efficiency_rating": 0 + }, + "sap_building_parts": [ + { + "identifier": "Main Dwelling", + "wall_dry_lined": "N", + "floor_heat_loss": 1, + "roof_construction": 3, + "wall_construction": 4, + "building_part_number": 1, + "sap_floor_dimensions": [ + { + "floor": 0, + "room_height": 2.42, + "floor_insulation": 1, + "total_floor_area": 33.24, + "floor_construction": 0, + "heat_loss_perimeter": 10.39 + } + ], + "wall_insulation_type": 2, + "construction_age_band": "B", + "wall_thickness_measured": "N", + "roof_insulation_location": "ND", + "roof_insulation_thickness": "ND" + } + ], + "low_energy_lighting": 100, + "solar_water_heating": "N", + "bedf_revision_number": 321, + "habitable_room_count": 2, + "heating_cost_current": { + "value": 303, + "currency": "GBP" + }, + "insulated_door_count": 0, + "co2_emissions_current": { + "value": 2.9, + "quantity": "tonnes per year" + }, + "energy_rating_average": 60, + "energy_rating_current": 66, + "lighting_cost_current": { + "value": 23, + "currency": "GBP" + }, + "main_heating_controls": [ + { + "description": "Manual charge control", + "energy_efficiency_rating": 2, + "environmental_efficiency_rating": 2 + } + ], + "multiple_glazing_type": 3, + "open_fireplaces_count": 0, + "has_hot_water_cylinder": "false", + "heating_cost_potential": { + "value": 190, + "currency": "GBP" + }, + "hot_water_cost_current": { + "value": 106, + "currency": "GBP" + }, + "mechanical_ventilation": 0, + "percent_draughtproofed": 100, + "suggested_improvements": [ + { + "sequence": 1, + "typical_saving": { + "value": 86, + "currency": "GBP" + }, + "indicative_cost": "\u00a3800 - \u00a31,200", + "improvement_type": "W", + "improvement_details": { + "improvement_number": 47 + }, + "improvement_category": 5, + "energy_performance_rating": 73, + "environmental_impact_rating": 58 + }, + { + "sequence": 2, + "typical_saving": { + "value": 27, + "currency": "GBP" + }, + "indicative_cost": "\u00a3600 - \u00a3800", + "improvement_type": "L", + "improvement_details": { + "improvement_number": 25 + }, + "improvement_category": 5, + "energy_performance_rating": 75, + "environmental_impact_rating": 60 + } + ], + "co2_emissions_potential": { + "value": 2.1, + "quantity": "tonnes per year" + }, + "energy_rating_potential": 75, + "lighting_cost_potential": { + "value": 23, + "currency": "GBP" + }, + "alternative_improvements": [ + { + "sequence": 1, + "typical_saving": { + "value": 24, + "currency": "GBP" + }, + "improvement_type": "J2", + "improvement_details": { + "improvement_number": 54 + }, + "improvement_category": 6, + "energy_performance_rating": 76, + "environmental_impact_rating": 93 + }, + { + "sequence": 2, + "typical_saving": { + "value": 84, + "currency": "GBP" + }, + "improvement_type": "Z1", + "improvement_details": { + "improvement_number": 51 + }, + "improvement_category": 6, + "energy_performance_rating": 80, + "environmental_impact_rating": 81 + } + ], + "hot_water_cost_potential": { + "value": 106, + "currency": "GBP" + }, + "renewable_heat_incentive": { + "water_heating": 1434, + "space_heating_existing_dwelling": 4064 + }, + "seller_commission_report": "Y", + "energy_consumption_current": 497, + "has_fixed_air_conditioning": "false", + "multiple_glazed_proportion": 100, + "calculation_software_version": 4.1, + "energy_consumption_potential": 365, + "environmental_impact_current": 47, + "fixed_lighting_outlets_count": 5, + "current_energy_efficiency_band": "D", + "environmental_impact_potential": 60, + "has_heated_separate_conservatory": "false", + "potential_energy_efficiency_band": "C", + "co2_emissions_current_per_floor_area": { + "value": 88, + "quantity": "kg/m2 per year" + }, + "low_energy_fixed_lighting_outlets_count": 5 +} \ No newline at end of file diff --git a/domain/modelling/scoring/overlay_applicator.py b/domain/modelling/scoring/overlay_applicator.py index 961f4e02..848f1a8f 100644 --- a/domain/modelling/scoring/overlay_applicator.py +++ b/domain/modelling/scoring/overlay_applicator.py @@ -9,10 +9,12 @@ then discarded). See ADR-0016. import copy from dataclasses import fields -from typing import Optional, Sequence +from typing import Final, Optional, Sequence from datatypes.epc.domain.epc_property_data import ( + BuildingPartIdentifier, EpcPropertyData, + SapBuildingPart, SapVentilation, SapWindow, WindowTransmissionDetails, @@ -29,6 +31,39 @@ from domain.modelling.simulation import ( ) +# A Landlord Override's building part is a POSITIONAL index (0=main, 1=extension +# 1…, ADR-0004), translated to a `BuildingPartIdentifier` upstream. This recovers +# that position so an override can fall back onto the part the EPC actually models +# at that slot when the gov-API labelled it differently (e.g. lodged a 2nd part as +# `other` rather than `extension_1`). +_POSITION_BY_IDENTIFIER: Final[dict[BuildingPartIdentifier, int]] = { + BuildingPartIdentifier.MAIN: 0, + BuildingPartIdentifier.EXTENSION_1: 1, + BuildingPartIdentifier.EXTENSION_2: 2, + BuildingPartIdentifier.EXTENSION_3: 3, + BuildingPartIdentifier.EXTENSION_4: 4, +} + + +def _resolve_part( + epc: EpcPropertyData, + parts_by_id: dict[BuildingPartIdentifier, SapBuildingPart], + identifier: BuildingPartIdentifier, +) -> Optional[SapBuildingPart]: + """The building part an overlay targets: the EPC's part with that identifier + when present, else the part at the override's positional index (so a + correction for `extension_1` still lands on the EPC's 2nd part even when the + gov-API lodged it under a different label). ``None`` when the EPC models no + part at that position.""" + part = parts_by_id.get(identifier) + if part is not None: + return part + position = _POSITION_BY_IDENTIFIER.get(identifier) + if position is None or position >= len(epc.sap_building_parts): + return None + return epc.sap_building_parts[position] + + def apply_simulations( baseline: EpcPropertyData, simulations: Sequence[EpcSimulation] ) -> EpcPropertyData: @@ -41,7 +76,12 @@ def apply_simulations( for simulation in simulations: for identifier, overlay in simulation.building_parts.items(): - part = parts_by_id[identifier] + part = _resolve_part(result, parts_by_id, identifier) + # No part at this position β€” the EPC models fewer parts than the + # override's index. We have no geometry to model the missing part, so + # skip it rather than crash the whole property's modelling. + if part is None: + continue for overlay_field in fields(overlay): value = getattr(overlay, overlay_field.name) if value is not None: diff --git a/scripts/hyde/build_10070004512.py b/scripts/hyde/build_10070004512.py new file mode 100644 index 00000000..2210242a --- /dev/null +++ b/scripts/hyde/build_10070004512.py @@ -0,0 +1,288 @@ +"""Elmhurst build for UPRN 10070004512 (SAP-Schema-16.0, GROUND-FLOOR FLAT, +band B, cavity FILLED, ELECTRIC STORAGE HEATERS (SAP 402 SEB, manual charge +control CSA/2401) + electric immersion off-peak (Economy-7 Dual meter) with a +cylinder (size 1), roof = another dwelling above, floor to EXTERNAL AIR, double +glazed, TFA 33.24, window 4.88 mΒ². Engine 66 = lodged 66. + +P1 of the modelling_e2e corpus validation β€” the built_form fix cert (16.0 omitted +built_form; mapper derives it from dwelling_type β†’ flatβ†’modal 4). built_form is +ML-only so SAP-neutral; engine reproduces lodged exactly. Storage-heater build +(see build_10022893721.py). Engine models NO secondary (sap_heating. +secondary_heating_type is None) β†’ secondary present=No to match. Run: + DISPLAY=:99 python scripts/hyde/build_10070004512.py +""" +from __future__ import annotations +import sys +import elmhurst_lib as E + +DIM = "TabContainer_TabPanelMain_WebUserControlDimensionsMain_" +WALL = ("TabContainer_TabPanelMain_InnerTabContainerMain_" + "TabPanelExternalWallMain_WebUserControlWallMain_") +ROOF = "TabContainer_TabPanelMain_WebUserControlRoofMain_" +FLOOR = "TabContainer_TabPanelMain_WebUserControlFloorsMain_" +WP = "TabContainer_TabPanelWindowsPanel_" +DP = "TabContainer_TabPanelDoorsPanel_" +VP = "TabContainer_TabPanelVentilationPanel_" +APT = "TabContainer_TabPanelAirPressureTest_" +LP = "TabContainer_TabPanelLighting_" +MV = "TabContainer_TabPanelMechVent_" +WH = "TabContainer_TabPanelWaterHeating_" + + +def _pick(page, suffix, contains): + val = page.evaluate( + """(a)=>{const s=document.getElementById(a[0]);if(!s)return null; + for(const o of s.options){if(o.text.toLowerCase().includes(a[1].toLowerCase()))return o.value;}return null;}""", + [f"{E.FP}{suffix}", contains]) + if val is not None: + E.set_select(page, suffix, val) + return val + + +def _options(page, suffix): + return page.evaluate( + """(id)=>{const s=document.getElementById(id);if(!s)return []; + return Array.from(s.options).map(o=>o.text);}""", f"{E.FP}{suffix}") + + +def property_description(page): + E.goto(page, "PropertyDescription", "WebFormPropertyDescription.aspx") + E.set_select(page, "DropDownListPropertyType1", "F Flat") + _pick(page, "DropDownListPropertyType2", "mid-terrace") # built_form 4 + E.set_text(page, "TextBoxStoreys", "1") + E.set_text(page, "TextBoxHabitableRooms", "2") + E.set_text(page, "TextBoxHeatedHabitableRooms", "2") + print("date ->", _pick(page, "DropDownListDateBuiltMain", "1900-1929")) # band B + E.set_select(page, "DropDownListDateBuiltFirst", "") + E.set_select(page, "DropDownListRoomInRoofMain", "") + E.save_close(page) + + +def flats(page): + E.goto(page, "Flats", "WebFormFlats.aspx") + E.set_select(page, "DropDownListPositionOfFlat", "Ground Floor") + E.set_text(page, "TextBoxFloor", "0") + E.set_select(page, "RadioButtonListFlatCoridor", "None") + E.save_close(page) + + +def dimensions(page): + E.goto(page, "Dimensions", "WebFormDimensions.aspx") + E.set_text(page, f"{DIM}TextBoxFloorAreaLowestFloor", "33.24") + E.set_text(page, f"{DIM}TextBoxRoomHeightLowestFloor", "2.42") + E.set_text(page, f"{DIM}TextBoxWallPerimeterLowestFloor", "10.39") + # 16.0 lodges no party_wall_length; a flat's side party walls are unmodelled by + # the engine. Try 0 (Elmhurst may require non-zero β€” adjust if Recommendations + # complains). + E.set_text(page, f"{DIM}TextBoxPartyWallLengthLowestFloor", "0") + E.save_close(page) + + +def walls(page): + E.goto(page, "Walls", "WebFormWalls.aspx") + E.set_select(page, f"{WALL}DropDownListType", "CA Cavity") + page.wait_for_timeout(400) + print("insulation ->", _pick(page, f"{WALL}DropDownListInsulation", "filled")) + E.save_close(page) + + +def roofs(page): + E.goto(page, "Roofs", "WebFormRoofs.aspx") + _pick(page, f"{ROOF}DropDownListType", "another dwelling above") + E.save_close(page) + + +def floors(page): + E.goto(page, "Floors", "WebFormFloors.aspx") + # Floor is "to external air" β€” that is the LOCATION/exposure, not a TYPE. + _pick(page, f"{FLOOR}DropDownListLocation", "external air") # E To external air + page.wait_for_timeout(400) + _pick(page, f"{FLOOR}DropDownListType", "solid") # construction; U from exposure + ins = page.locator(f"#{E.FP}{FLOOR}DropDownListInsulation") + if ins.count(): + E.set_select(page, f"{FLOOR}DropDownListInsulation", "A As built") + E.save_close(page) + + +def openings(page): + E.goto(page, "Openings", "WebFormOpenings.aspx") + E.click_tab(page, "TabContainer_TabPanelWindowsPanel") + _add_window(page, 4.88, "North", _glazing(page)) + _delete_zero_rows(page) + E.click_tab(page, "TabContainer_TabPanelDoorsPanel") + E.set_text(page, f"{DP}TextBoxDoors", "1") + E.set_text(page, f"{DP}TextBoxDoorsInsulated", "0") + E.set_text(page, f"{DP}TextBoxDraughtProofedDoors", "0") + E.save_close(page) + + +def _glazing(page): + for needle in ("unknown install date", "before 2002", "pre 2002"): + for opt in _options(page, f"{WP}DropDownListExtGlazing"): + low = opt.lower() + if needle in low and "triple" not in low and "single" not in low and "known data" not in low: + return opt + return "Double post or during 2022" + + +def _add_window(page, area, orientation, glazing): + print("glazing ->", glazing) + E.set_select(page, f"{WP}DropDownListExtGlazing", glazing) + page.wait_for_timeout(400) + ft = page.locator(f"#{E.FP}{WP}DropDownListExtFrameType") + if ft.count(): + ft.select_option("PVC") + gg = page.locator(f"#{E.FP}{WP}DropDownListExtGlazingGap") + if gg.count(): + gg.select_option("12 mm") + wid = f"{E.FP}{WP}TextBoxExtWidth" + page.evaluate( + """(a)=>{const e=document.getElementById(a[0]);if(e){e.value=a[1]; + e.dispatchEvent(new Event('input',{bubbles:true})); + e.dispatchEvent(new Event('change',{bubbles:true})); + e.dispatchEvent(new Event('blur',{bubbles:true}));}}""", [wid, str(area)]) + page.locator(f"#{E.FP}{WP}TextBoxExtHeight").fill("1.00") + page.locator(f"#{E.FP}{WP}DropDownListExtOrientation").select_option(orientation) + page.locator(f"#{E.FP}{WP}DropDownListExtBuildingPartId").select_option("Main") + page.locator(f"#{E.FP}{WP}DropDownListExtLocation").select_option("External wall") + page.wait_for_timeout(300) + before = E.window_row_count(page) + page.evaluate("(id)=>{const e=document.getElementById(id); if(e)e.click();}", f"{E.FP}{WP}ButtonAddWindow") + for _ in range(25): + page.wait_for_timeout(200) + if E.window_row_count(page) > before: + break + + +def _grid_rows(page): + return page.evaluate( + """()=>{const t=document.querySelector("[id*=GridViewExtendedWidows]"); + if(!t)return[];return Array.from(t.querySelectorAll('tr')).slice(1) + .map(r=>Array.from(r.querySelectorAll('td')).map(c=>c.innerText.trim()));}""") + + +def _delete_zero_rows(page): + g = 0 + while g < 6 and E.window_row_count(page) > 1: + g += 1 + rows = _grid_rows(page) + bad = next((i for i, c in enumerate(rows) if len(c) > 1 and c[1] in ("0.00", "0", "0.0")), None) + if bad is None: + break + _delete_row(page, bad) + page.wait_for_timeout(400) + + +def _delete_row(page, idx): + before = E.window_row_count(page) + btn = page.evaluate( + """(i)=>{const b=document.querySelectorAll("[id*='GridViewExtendedWidows_DeleteButton_']");return b[i]?b[i].id:null;}""", idx) + if not btn: + return + page.evaluate("(id)=>{const e=document.getElementById(id); if(e)e.click();}", btn) + page.wait_for_selector(f"#{E.FP}DeleteWindowDialog_LinkButtonYes", state="visible", timeout=5000) + page.evaluate("(id)=>{const e=document.getElementById(id); if(e)e.click();}", f"{E.FP}DeleteWindowDialog_LinkButtonYes") + for _ in range(20): + page.wait_for_timeout(200) + if E.window_row_count(page) < before: + break + + +def ventilation(page): + E.goto(page, "VentilationAndCooling", "WebFormVentilationAndCooling.aspx") + E.click_tab(page, "TabContainer_TabPanelVentilationPanel") + E.set_text(page, f"{VP}TextBoxIntermittentFans", "0") + cool = page.locator(f"#{E.FP}{VP}CheckBoxFixedSpaceCooling") + if cool.count() and cool.is_checked(): + E.commit(page, cool.uncheck) + E.click_tab(page, "TabContainer_TabPanelMechVent") + mv = page.locator(f"#{E.FP}{MV}CheckBoxMechanicalVentilation") + if mv.count() and mv.is_checked(): + E.commit(page, mv.uncheck) + E.click_tab(page, "TabContainer_TabPanelAirPressureTest") + E.set_select(page, f"{APT}DropDownListTestMethod", "Not available") + E.click_tab(page, "TabContainer_TabPanelLighting") + E.set_text(page, f"{LP}TextBoxLightsTotal", "5") + E.set_text(page, f"{LP}TextBoxLedLightsTotal", "5") # 100% low energy + E.set_text(page, f"{LP}TextBoxCflLightsTotal", "0") + E.save_close(page) + + +def space_heating(page): + # Electric storage heaters (SAP 402 = SEB), manual charge control (SAP 2401 = + # CSA). Two passes: clear bound PCDB boiler first, then set the SAP-table code. + E.goto(page, "SpaceHeating", "WebFormSpaceHeating.aspx") + page.wait_for_timeout(1000) + rid = f"{E.MH1}TextBoxPCDFBoilerReference" + ref = page.locator(f"#{rid}").input_value() + if ref not in ("0", ""): + print(f"clearing bound PCDB boiler {ref} -> 0 (rerun space_heating)") + page.evaluate("""(rid)=>{const r=document.getElementById(rid);r.value='0'; + r.dispatchEvent(new Event('change',{bubbles:true}));}""", rid) + page.wait_for_timeout(500) + E.save_close(page) + return + E.set_heating_dialog(page, "TabContainer_TabPanelMainHeating1_WebUserControlMainHeating1_ButtonMainHeatingCode", + "^Electric", "^Electric", "Storage", "SEB Modern slimline") + print("code:", page.locator(f"#{E.MH1}TextBoxMainHeatingCode").input_value()) + E.set_heating_dialog(page, "TabContainer_TabPanelMainHeating1_WebUserControlMainHeating1_ButtonMainHeatingControls", + "Storage Radiator", "CSA Manual charge control") + print("control:", page.locator(f"#{E.MH1}TextBoxMainHeatingControls").input_value()) + E.set_select(page, "DropDownListSecondaryHeatingPresent", "No") + # Economy-7 Dual meter (cert meter_type 1) β€” hidden Meters sub-tab. + E.click_tab(page, "TabContainer_TabPanelMeters") + E.set_select(page, "TabContainer_TabPanelMeters_RadioButtonListElectricityType", "Dual") + print("meter:", page.locator("#ContentBody_ContentPlaceHolder1_TabContainer_TabPanelMeters_RadioButtonListElectricityType").input_value()) + E.save_close(page) + + +def water_heating(page): + # Electric immersion off-peak (Dual) WITH cylinder (size 1 = small). The + # immersion code REQUIRES a cylinder. + E.goto(page, "WaterHeating", "WebFormWaterHeating.aspx") + E.click_tab(page, "TabContainer_TabPanelWaterHeating") + page.wait_for_timeout(600) + E.set_heating_dialog(page, f"{WH}ButtonWaterHeatingCode", + "Water Heater", "^Electric", "Immersion") + print("water code:", page.locator(f"#{E.FP}{WH}TextBoxWaterHeatingCode").input_value()) + cid = f"{E.FP}{WH}CheckBoxHotWaterCylinder" + cyl = page.locator(f"#{cid}") + if cyl.count() and not cyl.is_checked(): + try: + with page.expect_navigation(wait_until="load", timeout=8000): + page.evaluate( + """(id)=>{const c=document.getElementById(id);if(c){c.checked=true; + c.dispatchEvent(new Event('click',{bubbles:true})); + c.dispatchEvent(new Event('change',{bubbles:true}));}}""", cid) + except Exception: + page.wait_for_timeout(2000) + print("cylinder present:", cyl.is_checked() if cyl.count() else "n/a") + print("cyl sizes:", _options(page, f"{WH}DropDownListCylinderSize")) + _pick(page, f"{WH}DropDownListCylinderSize", "small") or \ + E.set_select(page, f"{WH}DropDownListCylinderSize", "Normal") + E.set_select(page, f"{WH}DropDownListInsulated", "Foam") + isuf = f"{WH}DropDownListInsulationThickness" + if page.locator(f"#{E.FP}{isuf}").count(): + _pick(page, isuf, "Unknown") or E.set_select(page, isuf, "25 mm") + imm = page.locator(f"#{E.FP}{WH}RadioButtonListImmersionHeater") + if imm.count(): + E.set_select(page, f"{WH}RadioButtonListImmersionHeater", "Dual") + E.save_close(page) + + +_ORDER = ["property_description", "flats", "dimensions", "walls", "roofs", + "floors", "openings", "ventilation", "space_heating", "water_heating"] + + +def main(): + if len(sys.argv) < 2 or sys.argv[1] not in _ORDER: + print("usage: build_10070004512.py <" + "|".join(_ORDER) + ">") + return 2 + with E.session() as (ctx, page): + globals()[sys.argv[1]](page) + print("done:", sys.argv[1], "->", page.url) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/hyde/build_22086693.py b/scripts/hyde/build_22086693.py new file mode 100644 index 00000000..5f102a45 --- /dev/null +++ b/scripts/hyde/build_22086693.py @@ -0,0 +1,307 @@ +"""Elmhurst build for UPRN 22086693 (RdSAP-Schema-20.0.0, SEMI-DETACHED HOUSE, +2-storey, band C 1930-1949, cavity UNINSULATED, mains-gas COMBI (no PCDB index β†’ +generic SAP Table 4b combi), control 2106 (CBE), pitched 200 mm loft, suspended +uninsulated floor, party wall 6.8 m, double glazed, 2Γ— PV ARRAYS @1.14 kW +(orientations SE/SW, pitch 30Β°, overshading modest), electric SECONDARY room +heater (SAP 691 REA), TFA ~74, window 11.84 mΒ². Engine 66 / lodged 72. + +P2 of the modelling_e2e corpus validation β€” the photovoltaic_supply-as-list fix +cert. The PV adds +5 (engine 61β†’66) so it IS credited; the βˆ’6 vs lodged is a +fabric/heating gap to localise. NEW build elements: generic (no-PCDB) combi + +the Renewables/PV page. Run: + DISPLAY=:99 python scripts/hyde/build_22086693.py +""" +from __future__ import annotations +import sys +import elmhurst_lib as E + +DIM = "TabContainer_TabPanelMain_WebUserControlDimensionsMain_" +WALL = ("TabContainer_TabPanelMain_InnerTabContainerMain_" + "TabPanelExternalWallMain_WebUserControlWallMain_") +PWALL = "TabContainer_TabPanelMain_InnerTabContainerMain_TabPanelPartyWallMain_WebUserControlPartyWallMain_" +ROOF = "TabContainer_TabPanelMain_WebUserControlRoofMain_" +FLOOR = "TabContainer_TabPanelMain_WebUserControlFloorsMain_" +WP = "TabContainer_TabPanelWindowsPanel_" +DP = "TabContainer_TabPanelDoorsPanel_" +VP = "TabContainer_TabPanelVentilationPanel_" +APT = "TabContainer_TabPanelAirPressureTest_" +LP = "TabContainer_TabPanelLighting_" +MV = "TabContainer_TabPanelMechVent_" +WH = "TabContainer_TabPanelWaterHeating_" +MH1B = "TabContainer_TabPanelMainHeating1_WebUserControlMainHeating1_" + + +def _pick(page, suffix, contains): + val = page.evaluate( + """(a)=>{const s=document.getElementById(a[0]);if(!s)return null; + for(const o of s.options){if(o.text.toLowerCase().includes(a[1].toLowerCase()))return o.value;}return null;}""", + [f"{E.FP}{suffix}", contains]) + if val is not None: + E.set_select(page, suffix, val) + return val + + +def _options(page, suffix): + return page.evaluate( + """(id)=>{const s=document.getElementById(id);if(!s)return []; + return Array.from(s.options).map(o=>o.text);}""", f"{E.FP}{suffix}") + + +def property_description(page): + E.goto(page, "PropertyDescription", "WebFormPropertyDescription.aspx") + E.set_select(page, "DropDownListPropertyType1", "H House") + _pick(page, "DropDownListPropertyType2", "semi") # built_form 2 + E.set_text(page, "TextBoxStoreys", "2") + E.set_text(page, "TextBoxHabitableRooms", "4") + E.set_text(page, "TextBoxHeatedHabitableRooms", "4") + print("date ->", _pick(page, "DropDownListDateBuiltMain", "1930-1949")) # band C + E.set_select(page, "DropDownListDateBuiltFirst", "") + E.set_select(page, "DropDownListRoomInRoofMain", "") + E.save_close(page) + + +def dimensions(page): + E.goto(page, "Dimensions", "WebFormDimensions.aspx") + E.set_text(page, f"{DIM}TextBoxFloorAreaLowestFloor", "36.86") + E.set_text(page, f"{DIM}TextBoxRoomHeightLowestFloor", "2.30") + E.set_text(page, f"{DIM}TextBoxWallPerimeterLowestFloor", "13.4") + E.set_text(page, f"{DIM}TextBoxPartyWallLengthLowestFloor", "6.8") + E.set_text(page, f"{DIM}TextBoxFloorArea1stFloor", "36.86") + E.set_text(page, f"{DIM}TextBoxRoomHeight1stFloor", "2.30") + E.set_text(page, f"{DIM}TextBoxWallPerimeter1stFloor", "17.4") + E.set_text(page, f"{DIM}TextBoxPartyWallLength1stFloor", "6.8") + E.save_close(page) + + +def walls(page): + E.goto(page, "Walls", "WebFormWalls.aspx") + E.set_select(page, f"{WALL}DropDownListType", "CA Cavity") + page.wait_for_timeout(400) + print("insulation ->", _pick(page, f"{WALL}DropDownListInsulation", "as built")) # uninsulated + # Semi party wall: cavity. Match "masonry filled" β†’ CF (Uβ‰ˆ0); avoid loose "filled". + pw = _pick(page, f"{PWALL}DropDownListPartyWallType", "masonry filled") \ + or _pick(page, f"{PWALL}DropDownListPartyWallType", "determine") + print("party wall ->", pw) + E.save_close(page) + + +def roofs(page): + E.goto(page, "Roofs", "WebFormRoofs.aspx") + _pick(page, f"{ROOF}DropDownListType", "access to loft") + _pick(page, f"{ROOF}DropDownListInsulationAt", "joists") + E.set_select(page, f"{ROOF}DropDownListThickness", "200 mm") + E.save_close(page) + + +def floors(page): + E.goto(page, "Floors", "WebFormFloors.aspx") + E.set_select(page, f"{FLOOR}DropDownListLocation", "G Ground floor") + _pick(page, f"{FLOOR}DropDownListType", "suspended timber") + E.set_select(page, f"{FLOOR}DropDownListInsulation", "A As built") + E.save_close(page) + + +def openings(page): + E.goto(page, "Openings", "WebFormOpenings.aspx") + E.click_tab(page, "TabContainer_TabPanelWindowsPanel") + _add_window(page, 11.84, "North", _glazing(page)) + _delete_zero_rows(page) + E.click_tab(page, "TabContainer_TabPanelDoorsPanel") + E.set_text(page, f"{DP}TextBoxDoors", "1") + E.set_text(page, f"{DP}TextBoxDoorsInsulated", "0") + E.set_text(page, f"{DP}TextBoxDraughtProofedDoors", "0") + E.save_close(page) + + +def _glazing(page): + for needle in ("unknown install date", "before 2002", "pre 2002"): + for opt in _options(page, f"{WP}DropDownListExtGlazing"): + low = opt.lower() + if needle in low and "triple" not in low and "single" not in low and "known data" not in low: + return opt + return "Double post or during 2022" + + +def _add_window(page, area, orientation, glazing): + print("glazing ->", glazing) + E.set_select(page, f"{WP}DropDownListExtGlazing", glazing) + page.wait_for_timeout(400) + ft = page.locator(f"#{E.FP}{WP}DropDownListExtFrameType") + if ft.count(): + ft.select_option("PVC") + gg = page.locator(f"#{E.FP}{WP}DropDownListExtGlazingGap") + if gg.count(): + gg.select_option("12 mm") + wid = f"{E.FP}{WP}TextBoxExtWidth" + page.evaluate( + """(a)=>{const e=document.getElementById(a[0]);if(e){e.value=a[1]; + e.dispatchEvent(new Event('input',{bubbles:true})); + e.dispatchEvent(new Event('change',{bubbles:true})); + e.dispatchEvent(new Event('blur',{bubbles:true}));}}""", [wid, str(area)]) + page.locator(f"#{E.FP}{WP}TextBoxExtHeight").fill("1.00") + page.locator(f"#{E.FP}{WP}DropDownListExtOrientation").select_option(orientation) + page.locator(f"#{E.FP}{WP}DropDownListExtBuildingPartId").select_option("Main") + page.locator(f"#{E.FP}{WP}DropDownListExtLocation").select_option("External wall") + page.wait_for_timeout(300) + before = E.window_row_count(page) + page.evaluate("(id)=>{const e=document.getElementById(id); if(e)e.click();}", f"{E.FP}{WP}ButtonAddWindow") + for _ in range(25): + page.wait_for_timeout(200) + if E.window_row_count(page) > before: + break + + +def fix_window(page): + """Edit-in-place: overwrite the shared assessment's leftover window width to ours.""" + E.goto(page, "Openings", "WebFormOpenings.aspx") + E.click_tab(page, "TabContainer_TabPanelWindowsPanel") + page.wait_for_timeout(800) + wid = f"{E.FP}{WP}TextBoxExtWidth" + page.evaluate("""(a)=>{const e=document.getElementById(a[0]);if(e){e.value=a[1]; + e.dispatchEvent(new Event('input',{bubbles:true})); + e.dispatchEvent(new Event('change',{bubbles:true})); + e.dispatchEvent(new Event('blur',{bubbles:true}));}}""", [wid, "11.84"]) + print("window width now:", page.locator(f"#{wid}").input_value()) + E.save_close(page) + + +def _grid_rows(page): + return page.evaluate( + """()=>{const t=document.querySelector("[id*=GridViewExtendedWidows]"); + if(!t)return[];return Array.from(t.querySelectorAll('tr')).slice(1) + .map(r=>Array.from(r.querySelectorAll('td')).map(c=>c.innerText.trim()));}""") + + +def _delete_zero_rows(page): + g = 0 + while g < 6 and E.window_row_count(page) > 1: + g += 1 + rows = _grid_rows(page) + bad = next((i for i, c in enumerate(rows) if len(c) > 1 and c[1] in ("0.00", "0", "0.0")), None) + if bad is None: + break + _delete_row(page, bad) + page.wait_for_timeout(400) + + +def _delete_row(page, idx): + before = E.window_row_count(page) + btn = page.evaluate( + """(i)=>{const b=document.querySelectorAll("[id*='GridViewExtendedWidows_DeleteButton_']");return b[i]?b[i].id:null;}""", idx) + if not btn: + return + page.evaluate("(id)=>{const e=document.getElementById(id); if(e)e.click();}", btn) + page.wait_for_selector(f"#{E.FP}DeleteWindowDialog_LinkButtonYes", state="visible", timeout=5000) + page.evaluate("(id)=>{const e=document.getElementById(id); if(e)e.click();}", f"{E.FP}DeleteWindowDialog_LinkButtonYes") + for _ in range(20): + page.wait_for_timeout(200) + if E.window_row_count(page) < before: + break + + +def ventilation(page): + E.goto(page, "VentilationAndCooling", "WebFormVentilationAndCooling.aspx") + E.click_tab(page, "TabContainer_TabPanelVentilationPanel") + E.set_text(page, f"{VP}TextBoxIntermittentFans", "0") + cool = page.locator(f"#{E.FP}{VP}CheckBoxFixedSpaceCooling") + if cool.count() and cool.is_checked(): + E.commit(page, cool.uncheck) + E.click_tab(page, "TabContainer_TabPanelMechVent") + mv = page.locator(f"#{E.FP}{MV}CheckBoxMechanicalVentilation") + if mv.count() and mv.is_checked(): + E.commit(page, mv.uncheck) + E.click_tab(page, "TabContainer_TabPanelAirPressureTest") + E.set_select(page, f"{APT}DropDownListTestMethod", "Not available") + E.click_tab(page, "TabContainer_TabPanelLighting") + E.set_text(page, f"{LP}TextBoxLightsTotal", "10") + E.set_text(page, f"{LP}TextBoxLedLightsTotal", "10") # 100% LED + E.set_text(page, f"{LP}TextBoxCflLightsTotal", "0") + E.save_close(page) + + +def space_heating(page): + # Generic (no-PCDB) mains-gas combi. The shared assessment may carry a prior + # cert's storage SEB / PCDB boiler β€” clear it first if present, then select a + # generic gas combi via the boiler dialog. Two passes if a SAP code is bound. + E.goto(page, "SpaceHeating", "WebFormSpaceHeating.aspx") + page.wait_for_timeout(1000) + # Pass 1: clear a leftover SAP-table MainHeatingCode (e.g. SEB) so the boiler + # search/dialog is usable. + mhc = page.locator(f"#{E.MH1}TextBoxMainHeatingCode") + code = mhc.input_value() if mhc.count() else "" + if code and code not in ("0",): + print(f"clearing leftover MainHeatingCode {code}") + page.evaluate("""(id)=>{const e=document.getElementById(id);if(e){e.value=''; + e.dispatchEvent(new Event('change',{bubbles:true}));}}""", f"{E.MH1}TextBoxMainHeatingCode") + page.wait_for_timeout(400) + E.save_close(page) + return + # Generic (no-PCDB) mains-gas boiler via the cascade: Gas β†’ Mains gas β†’ + # Boilers β†’ Post 1998 β†’ Condensing (combi vs regular is set by the water- + # heating page: "from primary" + no cylinder = combi). ~89% SAP Table 4b. + E.set_heating_dialog(page, f"{MH1B}ButtonMainHeatingCode", + "^Gas", "Mains gas", "Boilers", "Post 1998", "Condensing", + "Combi condens") # L6 = BGW Post 98 Combi condens. (SAP 4b ~89%) + print("code:", page.locator(f"#{E.MH1}TextBoxMainHeatingCode").input_value()) + E.set_heating_dialog(page, f"{MH1B}ButtonMainHeatingControls", + "^Boilers", "^Standard", "CBE Programmer, room thermostat and TRVs") + print("control:", page.locator(f"#{E.MH1}TextBoxMainHeatingControls").input_value()) + E.save_close(page) + + +def secondary(page): + # Lodged secondary: SAP 691 = REA electric panel/convector/radiant room heater. + E.goto(page, "SpaceHeating", "WebFormSpaceHeating.aspx") + page.wait_for_timeout(600) + E.set_select(page, "DropDownListSecondaryHeatingPresent", "Yes") + page.wait_for_timeout(900) + E.set_heating_dialog(page, "ButtonSecondaryHeatingCode", + "Electric", "Electric", "Room Heater", "REA Panel") + tb = page.locator(f"#{E.FP}TextBoxSecondaryHeatingCode") + print("secondary code:", tb.input_value() if tb.count() else "?") + E.save_close(page) + + +def water_heating(page): + E.goto(page, "WaterHeating", "WebFormWaterHeating.aspx") + E.click_tab(page, "TabContainer_TabPanelWaterHeating") + page.wait_for_timeout(400) + E.clear_hot_water_cylinder(page) + E.set_heating_dialog(page, f"{WH}ButtonWaterHeatingCode", + "From Space Heating", "From the primary heating system") + print("water code:", page.locator(f"#{E.FP}{WH}TextBoxWaterHeatingCode").input_value()) + E.save_close(page) + + +def renewables(page): + # 2Γ— PV arrays @1.14 kW. Discover the Renewables/PV page structure. + for url in ("WebFormRenewables.aspx", "WebFormPhotovoltaics.aspx", "WebFormSolar.aspx"): + try: + E.goto(page, url.replace("WebForm", "").replace(".aspx", ""), url) + page.wait_for_timeout(800) + print("PV page:", url, "->", page.url) + inputs = page.evaluate( + """()=>Array.from(document.querySelectorAll("[id*=ContentPlaceHolder1] input,[id*=ContentPlaceHolder1] select")).map(e=>e.id.replace('ContentBody_ContentPlaceHolder1_','')).filter(i=>/pv|photov|solar|peak|orient|pitch|oversh|panel/i.test(i)).slice(0,30)""") + print("PV-ish fields:", inputs) + return + except Exception as e: + print(" ", url, "->", type(e).__name__) + + +_ORDER = ["property_description", "dimensions", "walls", "roofs", "floors", + "openings", "fix_window", "ventilation", "space_heating", "secondary", + "water_heating", "renewables"] + + +def main(): + if len(sys.argv) < 2 or sys.argv[1] not in _ORDER: + print("usage: build_22086693.py <" + "|".join(_ORDER) + ">") + return 2 + with E.session() as (ctx, page): + globals()[sys.argv[1]](page) + print("done:", sys.argv[1], "->", page.url) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/hyde/elmhurst_download.py b/scripts/hyde/elmhurst_download.py index 0c317033..6e9e4805 100644 --- a/scripts/hyde/elmhurst_download.py +++ b/scripts/hyde/elmhurst_download.py @@ -28,7 +28,7 @@ SESSION_DIR = HERE / ".elmhurst-session" SAMPLE_DIR = ( HERE.parent.parent / "backend/epc_api/json_samples/real_life_examples" - / "SAP-Schema-16.0/uprn_10070004512" + / "RdSAP-Schema-20.0.0/uprn_22086693" ) ASSESSMENT_GUID = "B44A0DB4-4C08-4241-B818-86F060172105" diff --git a/tests/domain/modelling/test_overlay_applicator.py b/tests/domain/modelling/test_overlay_applicator.py index 64a237d7..8f6c960f 100644 --- a/tests/domain/modelling/test_overlay_applicator.py +++ b/tests/domain/modelling/test_overlay_applicator.py @@ -55,6 +55,55 @@ def test_apply_writes_targeted_building_part_and_leaves_others_untouched() -> No ) +def test_override_for_an_absent_semantic_part_lands_on_the_part_at_that_position() -> ( + None +): + # `building_part` is a POSITIONAL index (0=main, 1=extension 1…, ADR-0004). The + # gov-API EPC can label its parts differently (e.g. a 2nd part lodged as `other` + # rather than `extension_1`). An `extension_1` override must still land on the + # part at position 1 β€” the landlord's correction is applied, not dropped. + # Arrange β€” build_epc() is [MAIN, EXTENSION_1]; relabel the 2nd part to OTHER so + # the EXTENSION_1 identifier is absent but position 1 still exists. + baseline: EpcPropertyData = build_epc() + baseline.sap_building_parts[1].identifier = BuildingPartIdentifier.OTHER + simulation = EpcSimulation( + building_parts={ + BuildingPartIdentifier.EXTENSION_1: BuildingPartOverlay( + wall_insulation_type=3 + ) + } + ) + + # Act + result: EpcPropertyData = apply_simulations(baseline, [simulation]) + + # Assert β€” the override folded onto the part at position 1 (the OTHER part). + assert _part(result, BuildingPartIdentifier.OTHER).wall_insulation_type == 3 + + +def test_override_with_no_part_at_that_position_is_skipped() -> None: + # When there is genuinely no part at the override's position (the EPC models + # fewer parts than the index), the override is skipped rather than crashing β€” + # we cannot model an extension we have no geometry for. + # Arrange β€” build_epc() has 2 parts (positions 0, 1); position 2 is absent. + baseline: EpcPropertyData = build_epc() + simulation = EpcSimulation( + building_parts={ + BuildingPartIdentifier.MAIN: BuildingPartOverlay(wall_insulation_type=1), + BuildingPartIdentifier.EXTENSION_2: BuildingPartOverlay( + wall_insulation_type=1 + ), + } + ) + + # Act + result: EpcPropertyData = apply_simulations(baseline, [simulation]) + + # Assert β€” the present part got its overlay; nothing was added for position 2. + assert _part(result, BuildingPartIdentifier.MAIN).wall_insulation_type == 1 + assert len(result.sap_building_parts) == len(baseline.sap_building_parts) + + def test_flat_roof_construction_type_folds_onto_the_part() -> None: # ADR-0033: a flat-roof landlord override sets `roof_construction_type` so the # calculator's flat path (`"flat" in roof_construction_type`) fires the diff --git a/tests/domain/sap10_calculator/test_real_cert_sap_accuracy.py b/tests/domain/sap10_calculator/test_real_cert_sap_accuracy.py index 56989c71..5575384f 100644 --- a/tests/domain/sap10_calculator/test_real_cert_sap_accuracy.py +++ b/tests/domain/sap10_calculator/test_real_cert_sap_accuracy.py @@ -603,6 +603,52 @@ _EXPECTATIONS: Final[tuple[RealCertExpectation, ...]] = ( "engine prices 100% at the low rate; see elmhurst_worksheet.pdf (243-246)" ), ), + # UPRN 10070004512 β†’ cert 8742-6624-9300-2780-4926. SAP-Schema-16.0, GROUND- + # FLOOR FLAT, band B, cavity FILLED, ELECTRIC STORAGE HEATERS (SAP 402 SEB, + # manual charge CSA/2401) + electric immersion off-peak (Economy-7 Dual meter) + # with a small cylinder (size 1), roof = another dwelling above, floor to + # EXTERNAL AIR, double glazed, TFA 33.24. Engine 66 = lodged 66 EXACTLY. + # This is the modelling_e2e built_form fix cert: 16.0 omits `built_form`, which + # RdSapSchema17_1 requires; the mapper derives it from dwelling_type (flat β†’ + # modal 4). built_form is ML-only (the SAP calculator never reads it) so the fix + # is SAP-NEUTRAL β€” the engine reproduces the lodged score regardless. Built in + # Elmhurst RdSAP10 (evidence saved: elmhurst_summary.pdf / elmhurst_worksheet.pdf): + # worksheet SAP 54, engine on Elmhurst's own parsed inputs 53 β‰ˆ 54 β†’ calculator + # faithful. The engine 66 vs Elmhurst 54 (+12) is an input/build gap dominated by + # HOT WATER (engine 1272 vs Elmhurst 1948 kWh): the cert lodges a size-1 (small) + # cylinder but Elmhurst's RdSAP entry has no "Small" option (Normal/110 L is the + # smallest), forcing a larger cylinder + more storage loss; plus the reduced-field + # 16.0 floor/party-wall defaults. PINNED to the observed engine 66 (= lodged 66). + RealCertExpectation( + schema="SAP-Schema-16.0", + sample="uprn_10070004512", + cert_num="8742-6624-9300-2780-4926", + sap_score=66, + ), + # UPRN 22086693 β†’ cert 6102-6227-8000-0083-2292. RdSAP-Schema-20.0.0, SEMI- + # DETACHED HOUSE 2-storey, band C, cavity UNINSULATED, mains-gas COMBI (no PCDB + # β†’ generic SAP Table-4b BGW post-98 condensing combi), control 2106 (CBE), + # pitched 200 mm loft, suspended uninsulated floor, party wall 6.8 m, double + # glazed, electric secondary room heater (SAP 691 REA), 2Γ— PV ARRAYS @1.14 kW, + # 100% LED, TFA ~74. Engine 66 / lodged 72. + # This is the modelling_e2e photovoltaic_supply-AS-LIST fix cert: 20.0.0 typed + # `photovoltaic_supply` as the wrapper only, so a measured-array LIST crashed + # `from_rdsap_schema_20_0_0` ("'list' object has no attribute none_or_no_details") + # and sank the whole prediction cohort. The fix routes it through the dict-tolerant + # `_map_schema_21_pv`, capturing the arrays. PV is correctly credited: engine + # WITHOUT pv = 61, WITH pv = 66 (+5). Built in Elmhurst RdSAP10 (evidence saved: + # elmhurst_summary.pdf / elmhurst_worksheet.pdf) β€” fabric+heating only (the PV + # is a separate "New Technologies" Panel-details grid, deferred): worksheet 55, + # engine on Elmhurst's own parsed inputs 55 = 55 EXACTLY β†’ calculator faithful. + # The engine-without-pv 61 vs Elmhurst 55 (+6) is the documented engine-vs- + # Elmhurst-RdSAP-default residual on a band-C cavity-uninsulated suspended-floor + # semi; lodged 72 vs engine 66 (βˆ’6) is that plus PV-credit method. PINNED engine 66. + RealCertExpectation( + schema="RdSAP-Schema-20.0.0", + sample="uprn_22086693", + cert_num="6102-6227-8000-0083-2292", + sap_score=66, + ), )