From 560c912c0b54677c5f34c470c24f826c141c9bf3 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 8 Jun 2026 12:43:02 +0000 Subject: [PATCH 01/87] docs: roof-8 lead closed as data-fidelity (not a bug) + description-vs-code audit MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Records the session-4 audit: walls/heating/controls clean; roof_construction=8 ("Pitched, insulated", no measured thickness) computing U=2.30 is CORRECT, not a bug — confirmed by user worksheet sim-case-29 (band C → Elmhurst SAP 55 ≡ our 56.75; lodged 80 is data-fidelity artifact). Lesson: "insulated (assumed)" = the age-band default insulation level, not "well insulated". DO NOT re-chase roof-8. Co-Authored-By: Claude Opus 4.8 --- docs/HANDOVER_API_PROFILING.md | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/docs/HANDOVER_API_PROFILING.md b/docs/HANDOVER_API_PROFILING.md index 9fd1527c..516abed2 100644 --- a/docs/HANDOVER_API_PROFILING.md +++ b/docs/HANDOVER_API_PROFILING.md @@ -65,6 +65,31 @@ profile-surfaced buckets below. `roof_codes=1` broad bucket is mean −0.15 (the −1.78 was top-floor-electric-flat outliers −29/−25). Remaining gains need per-cert worksheets (start code-3) or the unsupported-schema ticket. +## SESSION-4 AUDIT — description-vs-code cross-check (other element types) +After floor-3, audited every API field that carries BOTH an integer code AND an +independent `…[].description` (roofs, walls, floors, main_heating, controls), by joining +code↔description on **single-element certs** (the multi-element `[]` summary is lossy). +- **Walls / heating / controls: CLEAN** — codes map 1:1 to their description families; + residual error is per-cert scatter, not mis-mapping. +- **Roof `roof_construction=8`: investigated hard, NOT a bug — DO NOT re-chase.** It looked + like a huge lead (n=57, ~189 |err|, mean −2.43) because code-8 sloping ceilings described + "Pitched, insulated" with no numeric thickness compute U=2.30 (e.g. 7921-0052 roof_w 241 + W/K, SAP 56.75 vs lodged 80). **User worksheet (simulated case 29) settled it: at band C, + RdSAP/Elmhurst assumes UNINSULATED for an "insulated (assumed)" roof+wall with no measured + thickness → Elmhurst SAP 55, which MATCHES our 56.75.** The lodged 80 needed real insulation + the API record doesn't carry = data-fidelity artifact (meter-3 class). KEY LESSON (user): + **"insulated (assumed)" = the age-band DEFAULT insulation level, NOT "well insulated"** — at + old bands that default is ~uninsulated. Re-audit by band confirmed: old-band no-numeric = + artifacts (we ≡ Elmhurst); newer-band code-8 already gets correct insulated U (2031 band I + U≈0.18, 1436 band E 0.17, 0536 band B 0.09) — their under-rates are elsewhere; numeric- + thickness = accurate (9884 +0.06). So the force-uninsulated-at-pre-1950 rule is CORRECT. + A roof-8 "fix" would push us AWAY from Elmhurst's faithful 55 toward the unreproducible 80. +- **Roof `roof_construction=3`: latent mis-map (inert).** gov desc "(another dwelling above)" + (party roof) but we map to "Pitched, no access to loft"; masked by dwelling exposure + (mean −0.06). Correct for robustness only if touching the roof mapper; not worth chasing. +- **Worksheet-gen constraints (user, for future repros):** Elmhurst no longer lets you pick + build form for a flat; and a band-C repro defaults to uninsulated walls+roof. + ## KEY INSIGHT (load-bearing, from the user) **The gov EPC API JSON is the published OUTPUT of RdSAP software (Elmhurst), not its input.** So any API field Elmhurst doesn't expose as an *input* is register metadata the RdSAP10 method From 5e7ef5c7ffa3e78ab8a0112a229ebca28595bfab Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 8 Jun 2026 13:17:15 +0000 Subject: [PATCH 02/87] =?UTF-8?q?fix(control):=20no=20boiler=20interlock?= =?UTF-8?q?=20for=20TRVs+bypass=20controls=202107/2111=20(SAP=20=C2=A79.4.?= =?UTF-8?q?11)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `_BOILER_NO_ROOM_THERMOSTAT_CONTROL_CODES` held only {2101, 2102} — it was keyed off the Table 4e "+0.6 °C" annotation rather than the actual interlock criterion. SAP 10.2 §9.4.11 (PDF p.66): "A boiler system with no room thermostat (or a device equivalent in this context, such as a flow switch or boiler energy manager) ... must be considered as having no interlock", and "TRVs alone ... do not perform the boiler interlock function". A fixed bypass likewise provides no interlock (it keeps water circulating when TRVs close). So control 2107 ("Programmer, TRVs and bypass") and 2111 ("TRVs and bypass") lack interlock and must take the Table 4c(2) −5pp Space+DHW seasonal-efficiency adjustment and the Table 4f footnote a) ×1.3 circulation-pump uplift — both of which they previously missed. (2108 flow switch / 2109 boiler energy manager carry interlock-equivalent devices → excluded; 2103-2106/2113 have a room thermostat.) All affected certs are cat-2 gas boilers, where §9.4.11 applies. Eval: 909 computed, 45.3% → 46.9% within 0.5 (+14 certs: 412 → 426), mean|err| 1.659 → 1.633. Bucket means corrected: control 2107 +1.50 → +0.32 (n=38), 2111 +1.48 → +0.16 (n=4). 32 improved / 10 regressed (all small; the six that crossed out of ±0.5 were coincidentally-accurate offsetting-error certs). Co-Authored-By: Claude Opus 4.8 --- .../sap10_calculator/rdsap/cert_to_inputs.py | 44 ++++++++++++------- .../rdsap/test_cert_to_inputs.py | 38 ++++++++++++++++ 2 files changed, 66 insertions(+), 16 deletions(-) diff --git a/domain/sap10_calculator/rdsap/cert_to_inputs.py b/domain/sap10_calculator/rdsap/cert_to_inputs.py index 5298b0c5..29730560 100644 --- a/domain/sap10_calculator/rdsap/cert_to_inputs.py +++ b/domain/sap10_calculator/rdsap/cert_to_inputs.py @@ -385,7 +385,9 @@ def _table_4f_circulation_pump_kwh(main: Optional[MainHeatingDetail]) -> float: 2 → 41 kWh (2013 or later) Table 4f footnote a) then multiplies the row by 1.3 when the room - thermostat is absent (control code 2101 / 2102). + thermostat is absent — the same "no room thermostat" criterion as the + interlock rule, i.e. `_BOILER_NO_ROOM_THERMOSTAT_CONTROL_CODES` + (2101 / 2102 / 2107 / 2111; bypass and TRVs are not a room thermostat). """ if not _is_wet_boiler_main(main): return 0.0 @@ -1267,22 +1269,32 @@ _CONTROL_TYPE_BY_CODE: Final[dict[int, int]] = { } -# SAP 10.2 Table 4e Group 1 (PDF p.171) — boiler control codes providing -# NO thermostatic control of room temperature, i.e. no room thermostat -# ("No time or thermostatic control of room temperature" 2101 / -# "Programmer, no room thermostat" 2102 — the two Group-1 rows carrying -# the "+0.6 °C / Table 4c(2)" annotation). Per RdSAP 10 §3 (PDF p.57) -# boiler interlock is "assumed present if there is a room thermostat and -# (for stored hot water systems heated by the boiler) a cylinder -# thermostat. Otherwise not interlocked." A gas/liquid-fuel boiler under -# one of these controls therefore has NO boiler interlock regardless of -# the cylinder thermostat, triggering the Table 4c(2) (PDF p.169) "No -# thermostatic control of room temperature – regular boiler" -5pp Space -# + DHW seasonal-efficiency adjustment. The combi rows of Table 4c(2) -# take Space -5 / DHW 0; the DHW leg is gated separately on a cylinder -# being present (regular boiler) at the call site. +# SAP 10.2 Table 4e Group 1 (PDF p.171) — boiler control codes with NO +# boiler interlock because they lack a room thermostat (or an equivalent +# device). SAP 10.2 §9.4.11 (PDF p.66) is explicit: "A boiler system with +# no room thermostat (or a device equivalent in this context, such as a +# flow switch or boiler energy manager), even if there is a cylinder +# thermostat, must be considered as having no interlock", and "TRVs alone +# (other than some communicating TRVs) do not perform the boiler interlock +# function". A *fixed bypass* likewise provides no interlock — it exists to +# keep water circulating when the TRVs close. The Group-1 rows without a +# room thermostat / flow switch / boiler energy manager are therefore: +# 2101 "No time or thermostatic control of room temperature" +# 2102 "Programmer, no room thermostat" +# 2107 "Programmer, TRVs and bypass" ← bypass ≠ interlock +# 2111 "TRVs and bypass" ← bypass ≠ interlock +# (2108 "Programmer, TRVs and flow switch" and 2109 "… boiler energy +# manager" carry an interlock-equivalent device, so they are INTERLOCKED +# and excluded; 2103-2106/2113 all carry a room thermostat.) Each of these +# triggers the Table 4c(2) (PDF p.169) "No thermostatic control of room +# temperature – regular boiler" -5pp Space + DHW seasonal-efficiency +# adjustment. The combi rows of Table 4c(2) take Space -5 / DHW 0; the DHW +# leg is gated separately on a cylinder being present at the call site. +# NB this is the interlock criterion only — the separate "+0.6 °C" Table 4e +# temperature adjustment applies to 2101/2102 alone (it lives in +# `_CONTROL_TEMPERATURE_ADJUSTMENT_BY_CODE`, where 2107/2111 stay at 0.0). _BOILER_NO_ROOM_THERMOSTAT_CONTROL_CODES: Final[frozenset[int]] = frozenset( - {2101, 2102} + {2101, 2102, 2107, 2111} ) diff --git a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py index 08a25d75..a00fca28 100644 --- a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py +++ b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py @@ -4720,6 +4720,44 @@ def test_table_4c_no_boiler_interlock_applies_minus_5_dhw_adjustment_when_cylind ) +def test_controls_2107_2111_count_as_no_room_thermostat_per_sap_9_4_11() -> None: + # Arrange — SAP 10.2 §9.4.11 (PDF p.66): a boiler with no room + # thermostat (or an equivalent device — flow switch / boiler energy + # manager) has no interlock; "TRVs alone ... do not perform the boiler + # interlock function" and a fixed bypass exists precisely to keep water + # circulating when the TRVs close. Control 2107 ("Programmer, TRVs and + # bypass") and 2111 ("TRVs and bypass") therefore carry the same + # no-room-thermostat treatment as 2101/2102 — including the Table 4f + # footnote a) ×1.3 circulation-pump uplift — which they previously + # missed (the set held only 2101/2102). Control 2106 ("Programmer, room + # thermostat and TRVs") HAS a room thermostat → interlock → no uplift. + # A 2013+ wet gas boiler pumps 41 kWh (Table 4f); ×1.3 = 53.3. + import dataclasses + + from domain.sap10_calculator.rdsap.cert_to_inputs import _table_4f_circulation_pump_kwh # pyright: ignore[reportPrivateUsage] + + base = _gas_boiler_detail() # cat-2 wet boiler + no_stat_2107 = dataclasses.replace( + base, main_heating_control=2107, central_heating_pump_age=2, + ) + no_stat_2111 = dataclasses.replace( + base, main_heating_control=2111, central_heating_pump_age=2, + ) + with_stat_2106 = dataclasses.replace( + base, main_heating_control=2106, central_heating_pump_age=2, + ) + + # Act + pump_2107 = _table_4f_circulation_pump_kwh(no_stat_2107) # pyright: ignore[reportPrivateUsage] + pump_2111 = _table_4f_circulation_pump_kwh(no_stat_2111) # pyright: ignore[reportPrivateUsage] + pump_2106 = _table_4f_circulation_pump_kwh(with_stat_2106) # pyright: ignore[reportPrivateUsage] + + # Assert — bypass/TRVs-only controls get the ×1.3 uplift; 2106 does not. + assert abs(pump_2107 - 41.0 * 1.3) <= 1e-9 + assert abs(pump_2111 - 41.0 * 1.3) <= 1e-9 + assert abs(pump_2106 - 41.0) <= 1e-9 + + def test_sap_9_4_11_no_boiler_interlock_applies_minus_5_pcdb_space_heating_when_main_is_gas_oil_boiler_with_cylinder_no_thermostat() -> None: """SAP 10.2 §9.4.11 (PDF p.30) "Boiler interlock": From faf29942ba7375203aa49b7ad2d9ecdabd2ad702 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 8 Jun 2026 14:23:17 +0000 Subject: [PATCH 03/87] =?UTF-8?q?fix(secondary):=20apply=20Table=2011=20se?= =?UTF-8?q?condary=20when=20lodged=20via=20description=20only=20(=C2=A7A.2?= =?UTF-8?q?.2)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `_secondary_fraction` keyed "has a secondary" off the integer `secondary_heating_type` code. The gov-API path surfaces the secondary as a DESCRIPTION instead (`secondary_heating.description`, e.g. "Portable electric heaters (assumed)") and leaves the integer code None. So a gas/oil boiler main (not in the §A.2.2 forced-secondary set) with an assumed portable-electric secondary dropped the secondary entirely (sec_kWh=0), under-costing the dwelling and over-rating its SAP. Per RdSAP §A.2.2 / SAP 10.2 Table 11, a lodged secondary is costed at its Table 11 fraction (cat-2 boiler = 0.10, billed at standard-rate electricity per the §A.2.2 assumed portable-electric default). New `_has_lodged_secondary_description` treats a real `secondary_heating.description` as a lodged secondary; passed to `_secondary_fraction` at both call sites. The description is authoritative — same lesson as floor_heat_loss / roof codes. (Electric-storage mains were unaffected: they force the secondary already.) Also adds the Table 11 fraction for main_heating_category=8 (electric underfloor, "Integrated storage/direct-acting electric systems" = 0.10) — the strict-raise surfaced this latent gap once cat-8 mains were routed through the lookup. Eval: 909 computed, 0 raises, 46.9% -> 47.6% within 0.5 (+13 certs: 420 -> 433), mean|err| 1.633 -> 1.586. 13 improved / 1 regressed (2610, a cat-10 room-heater cert with an independent over-count). Bucket "Portable electric heaters" median +2.73 -> ~0 on the gas/cat-2 subset (cat-7 storage was already correct). Co-Authored-By: Claude Opus 4.8 --- .../sap10_calculator/rdsap/cert_to_inputs.py | 46 +++++++++++++++++-- .../rdsap/test_cert_to_inputs.py | 21 +++++++++ 2 files changed, 63 insertions(+), 4 deletions(-) diff --git a/domain/sap10_calculator/rdsap/cert_to_inputs.py b/domain/sap10_calculator/rdsap/cert_to_inputs.py index 29730560..71367992 100644 --- a/domain/sap10_calculator/rdsap/cert_to_inputs.py +++ b/domain/sap10_calculator/rdsap/cert_to_inputs.py @@ -794,6 +794,14 @@ _SECONDARY_HEATING_FRACTION_BY_CATEGORY: Final[dict[int, float]] = { 5: 0.10, 6: 0.10, 7: 0.15, + 8: 0.10, # Electric underfloor heating (direct-acting electric, e.g. + # SAP code 424): SAP 10.2 Table 11 (PDF p.188) row + # "Integrated storage/direct-acting electric systems" / + # "Other electric systems" = 0.10. First exercised when the + # description-lodged-secondary fix routed cat-8 mains (which + # previously short-circuited to 0) through the Table 11 + # lookup (cert 2051-9502, electric underfloor + assumed + # portable-electric secondary). 9: 0.10, # Warm-air systems (NOT heat pump): a gas/oil warm-air unit # is an "All gas, liquid and solid fuel systems" row (0.10), # and electric warm air is "Other electric systems" (also @@ -2305,7 +2313,9 @@ def _hot_water_fuel_cost_gbp_per_kwh( def _secondary_fraction( - main: Optional[MainHeatingDetail], secondary_heating_type: object + main: Optional[MainHeatingDetail], + secondary_heating_type: object, + secondary_lodged: bool = False, ) -> float: """SAP 10.2 Table 11 lookup by main heating category, applied only when (a) the cert has a secondary system lodged OR (b) the main @@ -2313,6 +2323,17 @@ def _secondary_fraction( heaters). Returns 0.0 when neither applies — the most common case for gas/oil main systems whose cert doesn't lodge a secondary. + `secondary_lodged` covers the gov-API path: the register publishes + the secondary as a DESCRIPTION (`secondary_heating.description`, e.g. + "Portable electric heaters (assumed)") even when the integer + `secondary_heating_type` code is absent. The description is + authoritative — a lodged secondary description means RdSAP assessed a + secondary (per §A.2.2 the assumed system is portable electric heaters) + and its Table 11 fraction must be costed. Without this a gas/oil + boiler main with an assumed portable-electric secondary dropped the + secondary entirely (sec_kWh=0), under-costing the dwelling and + over-rating its SAP by a clean systematic +2.7 (median). + `main_heating_fraction` on the cert is NOT consulted here: empirical probe shows it tracks main-system-1 vs main-system-2 allocation in multi-main configurations (99% of corpus has =1, meaning "single @@ -2334,7 +2355,7 @@ def _secondary_fraction( if main is None: return 0.0 code = main.sap_main_heating_code - has_lodged_secondary = secondary_heating_type is not None + has_lodged_secondary = secondary_heating_type is not None or secondary_lodged force = code is not None and code in _FORCE_SECONDARY_FOR_MAIN_CODES if not has_lodged_secondary and not force: return 0.0 @@ -2346,6 +2367,19 @@ def _secondary_fraction( return _secondary_heating_fraction_for_category(main.main_heating_category) +def _has_lodged_secondary_description(epc: EpcPropertyData) -> bool: + """True when the cert lodges a secondary-heating DESCRIPTION (the + gov-API path surfaces the secondary as `secondary_heating.description`, + e.g. "Portable electric heaters (assumed)", even when the integer + `secondary_heating_type` code is None). RdSAP treats a lodged + secondary as costed (§A.2.2), so this gates the Table 11 fraction.""" + sec = epc.secondary_heating + if sec is None: + return False + desc = getattr(sec, "description", None) + return desc is not None and desc not in ("None", "") + + def _secondary_heating_fraction_for_category( main_heating_category: Optional[int], ) -> float: @@ -4290,7 +4324,9 @@ def energy_requirements_section_from_cert( main_category = main.main_heating_category if main is not None else None main_fuel = _main_fuel_code(main) secondary_fraction_value = _secondary_fraction( - main, epc.sap_heating.secondary_heating_type if epc.sap_heating else None + main, + epc.sap_heating.secondary_heating_type if epc.sap_heating else None, + secondary_lodged=_has_lodged_secondary_description(epc), ) # When no secondary system is lodged the worksheet displays (208) = 0; # the per-system fuel formula already collapses to 0 via fraction_201 = 0 @@ -6559,7 +6595,9 @@ def cert_to_inputs( # without recomputing it. Pure function over the cert; same value # later when §9a `space_heating_fuel_monthly_kwh` runs. secondary_fraction_value = _secondary_fraction( - main, epc.sap_heating.secondary_heating_type + main, + epc.sap_heating.secondary_heating_type, + secondary_lodged=_has_lodged_secondary_description(epc), ) # SAP10.2 §4 — compute the worksheet (45..65) values now (they only # depend on the cert dwelling shape, not on water_efficiency). The diff --git a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py index a00fca28..d0f12b4d 100644 --- a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py +++ b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py @@ -522,6 +522,27 @@ def test_main_heating_fraction_does_not_override_table11_secondary_default() -> assert inputs.secondary_heating_fraction == pytest.approx(0.1, abs=0.001) +def test_secondary_fraction_fires_when_secondary_lodged_via_description_only() -> None: + # Arrange — SAP 10.2 Table 11 / RdSAP §A.2.2: a gas boiler main (cat 2, + # not in the §A.2.2 forced-secondary set) whose cert lodges NO integer + # `secondary_heating_type` but DOES carry a secondary DESCRIPTION (the + # gov-API path surfaces the secondary only as a description, e.g. + # "Portable electric heaters (assumed)") must cost the secondary at its + # Table 11 0.10 fraction. Previously this returned 0.0 — the secondary + # was dropped (sec_kWh=0) → a clean systematic SAP over-rate (+2.7 med). + from domain.sap10_calculator.rdsap.cert_to_inputs import _secondary_fraction # pyright: ignore[reportPrivateUsage] + + main = _gas_boiler_detail() # cat 2, code 102 — not forced-secondary + + # Act + no_secondary = _secondary_fraction(main, None, secondary_lodged=False) + description_lodged = _secondary_fraction(main, None, secondary_lodged=True) + + # Assert — a description-only lodged secondary fires the 0.10 fraction. + assert no_secondary == 0.0 + assert abs(description_lodged - 0.10) <= 1e-9 + + def test_main_heating_fraction_missing_falls_back_to_table11_default() -> None: # Arrange — when main_heating_fraction isn't lodged AND the cert # has a secondary system lodged, Table 11's 0.10 default still From d83c431c7d40bc7cdc23eb6b9b0b0a360511e779 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 8 Jun 2026 14:47:31 +0000 Subject: [PATCH 04/87] =?UTF-8?q?docs:=20session-4=20handover=20=E2=80=94?= =?UTF-8?q?=20interlock=20+=20secondary=20fixes,=20robust-audit=20method,?= =?UTF-8?q?=20open=20leads?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Updates the headline (45.1 → 47.6%), records the four shipped fixes + the roof-8 false-lead closure, documents the two methods that worked (description-vs-code audit + outlier-robust categorical sweep by net skew + median), and lists the open robust leads (whc=903 immersion HW, cat-7 storage, dual immersion) with the scatter buckets to avoid. Co-Authored-By: Claude Opus 4.8 --- docs/HANDOVER_API_PROFILING.md | 60 +++++++++++++++++++++++++++------- 1 file changed, 49 insertions(+), 11 deletions(-) diff --git a/docs/HANDOVER_API_PROFILING.md b/docs/HANDOVER_API_PROFILING.md index 516abed2..c9c7bec3 100644 --- a/docs/HANDOVER_API_PROFILING.md +++ b/docs/HANDOVER_API_PROFILING.md @@ -13,17 +13,20 @@ deproven approaches + the meter/shower data-fidelity findings), and the earlier `energy_rating_current`. Headline gauge: `PYTHONPATH=/workspaces/model python scripts/eval_api_sap_accuracy.py`. -| metric | now (`a8e5563a`) | -|--------|------------------| -| **% \|err\| < 0.5** | **45.1%** | -| % \|err\| < 1.0 | 59.4% | -| mean \|err\| | 1.702 | -| mean signed | −0.006 (balanced) | -| computed / raises | **909 / 0** | -| unsupported_schema | 100 (deferred — see below) | +| metric | session-3 (`a8e5563a`) | **session-4 (`faf29942`)** | +|--------|------------------|------------------| +| **% \|err\| < 0.5** | 45.1% | **47.6%** | +| % \|err\| < 1.0 | 59.4% | **62.6%** | +| % \|err\| < 2.0 | 77.7% | **79.6%** | +| mean \|err\| | 1.702 | **1.586** | +| computed / raises | 909 / 0 | **909 / 0** | +| unsupported_schema | 100 (deferred) | 100 (deferred) | -45% is still poor. The systematic bias is gone; remaining error is per-cert scatter + the -profile-surfaced buckets below. +**SESSION-4 shipped (45.1 → 47.6%):** four spec-grounded fixes + closed one false lead. +See the `## SESSION-4 …` blocks below and the auto-memory for full detail. The systematic bias +is gone; the winning method this session was the **description-vs-code audit** + an +**outlier-robust categorical sweep** (rank by net directional skew + MEDIAN, not mean — the +mean-based metric is fooled by multi-cause outliers). 47.6% is still the target's halfway point. ## WHAT SHIPPED THIS SESSION (7 slices, all green, pyright net-zero) 1. `e41a0bc0` **PCDB heat pump w/o SAP code → Table 12a ASHP_APP_N SH split** (0.80 high-rate). @@ -39,7 +42,42 @@ profile-surfaced buckets below. (4)(5)(6) cleared **all 4 raises** — eval now has zero raises. 7. `(profiler)` **`scripts/profile_api_error.py`** — the new diagnostic (below). -## SESSION-4 UPDATE (HEAD `8741fbdf`) — read before re-working the leads below +## SESSION-4 UPDATE (HEAD `faf29942`) — read before re-working the leads below + +### Shipped this session (45.1 → 47.6%) +1. `b40e0f67` **exposed-floor-on-flats** (floor_heat_loss=1) — §3.12; per-BP override of the + dwelling-level flat suppression. +2. `8741fbdf` **floor_heat_loss=3 → above partially heated space, U=0.7** (§3.12/§5.14) + + re-pinned golden 7536 (its "irreducible residual" was THIS bug). +3. `5e7ef5c7` **boiler interlock for TRVs+bypass controls 2107/2111** (§9.4.11) — biggest single + win (+1.6pts). The no-interlock set was keyed off the wrong signal (the "+0.6 °C" annotation); + 2107/2111 lack a room thermostat → −5pp + Table 4f ×1.3 pump. +4. `faf29942` **description-lodged secondary heating** (§A.2.2/Table 11) — gas/oil boilers with an + API-description-only secondary ("Portable electric heaters (assumed)", code field None) + dropped the secondary (sec_kWh=0); now `_has_lodged_secondary_description` fires Table 11. + Also added cat-8 (electric underfloor) Table-11 fraction 0.10. +- `560c912c`/`d0f57a0e` docs: **roof_construction=8 lead CLOSED as data-fidelity** (not a bug — see + the roof-8 section below; user worksheet sim-case-29 proved we ≡ Elmhurst). + +### The two methods that worked (reuse these) +- **Description-vs-code audit:** join each int code (`floor_heat_loss`, `roof_construction`, + `wall_construction`, secondary type) to its authoritative `…[].description`, **on single-element + certs only** (multi-element `[]` arrays are LOSSY). Mis-maps fall out (floor-3, secondary). +- **Outlier-robust categorical sweep** (`/tmp/cat_audit2.py`): rank field-values by **net + directional skew** (#under−0.5 minus #over+0.5) + **MEDIAN** error. The mean-based directionality + metric (`/tmp/cat_audit.py`) gets FOOLED by multi-cause outliers (e.g. "Solid brick no insulation" + looked systematic at mean −1.07 but median is −0.22 = scatter; 2100 −61/RR drove it). + +### Open robust leads (verify with `/tmp/cat_audit2.py` — they shift; check MEDIAN not mean) +- `whc=903` electric-immersion HW: **median +0.87, n=84** — likely off-peak immersion handling + (the handover noted WHC 903 raises NotImplementedError on the Table-12a off-peak-immersion row). +- `main_heat_cat=7` electric storage: median +1.05, n=41 — over-rate (tariff/cost; partly artifact). +- `immersion_type=2` dual: +1.50, n=43 — we OVER-credit (so §12 dual→off-peak would worsen it). +- `dwelling_type=Top-floor flat`: median −1.24, n=99 — under-rate, mostly fabric scatter/artifacts. +- **Low-dir = SCATTER, do NOT single-fix:** non-PCDB main / data_source=2 (n=242, 28% within-0.5), + mains_gas=N electric (n=145), most flats. These are per-cert/data-fidelity, not one bug. + +### Resolved/closed this session (don't re-chase) - **Lead #1 `floor_codes=3` RESOLVED — the code IS authoritative.** The diagnostic that cracked it: join each **single-BP** cert's `floor_heat_loss` code to its independent `floors[].description` (the multi-BP tally was contaminated because a cert's `floors[]` summary From 43d4c67d12071b3280a73d4a9f01a69ed041d62d Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 8 Jun 2026 15:45:41 +0000 Subject: [PATCH 05/87] fix(hw-cost): WHC-903 immersion off-peak HW bills at Table 13 high-rate fraction MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Electric immersion water heating (WHC 903) on an off-peak tariff billed 100% at the low rate, under-costing the dwelling and over-rating it (median +0.98 SAP across the off-peak WHC-903 API cohort, n=57). SAP 10.2 Table 12a "Immersion water heater" row (PDF p.191) routes the water-heating column to Table 13 (PDF p.197): the high-rate fraction is a function of cylinder volume V, assumed occupancy N (Appendix J Table 1b) and single-/dual-immersion. The remainder bills at the low rate. Table 13 Note 2 supplies exact equations equivalent to the rounded grid; `electric_dhw_high_rate_fraction` evaluates them (validated against the published 110 L grid cells). Per Note 1 the 10-hour equations cover any tariff with >=10 hours/day low-rate (so 18-/24-hour use that column). Immersion code mapping CONFIRMED 1=dual, 2=single via RdSAP 10 §10.5 (PDF p.54 — an immersion is "assumed dual" on a dual/off-peak meter) cross-checked against the API cohort (code 1 sits 3.6:1 on dual meters; code 2 on single meters). This INVERTS an earlier handover's unverified "1=single, 2=dual" note — the dual code carries Table 13's small fraction, matching the cohort over-rating direction; the single mapping overshot in a prototype. API SAP eval: 47.6% -> 48.6% within 0.5; <1.0 62.6% -> 63.8%; mean|err| 1.586 -> 1.561; 909 computed, 0 raises. Co-Authored-By: Claude Opus 4.8 --- .../sap10_calculator/rdsap/cert_to_inputs.py | 66 ++++++++++- domain/sap10_calculator/tables/table_13.py | 70 ++++++++++++ .../rdsap/test_cert_to_inputs.py | 55 +++++++++- .../domain/sap10_calculator/test_table_13.py | 103 ++++++++++++++++++ 4 files changed, 288 insertions(+), 6 deletions(-) create mode 100644 domain/sap10_calculator/tables/table_13.py create mode 100644 tests/domain/sap10_calculator/test_table_13.py diff --git a/domain/sap10_calculator/rdsap/cert_to_inputs.py b/domain/sap10_calculator/rdsap/cert_to_inputs.py index 71367992..7b77c669 100644 --- a/domain/sap10_calculator/rdsap/cert_to_inputs.py +++ b/domain/sap10_calculator/rdsap/cert_to_inputs.py @@ -105,6 +105,9 @@ from domain.sap10_calculator.tables.table_12a import ( tariff_from_meter_type, water_heating_high_rate_fraction, ) +from domain.sap10_calculator.tables.table_13 import ( + electric_dhw_high_rate_fraction, +) from domain.sap10_calculator.tables.table_32 import ( additional_standing_charges_gbp, is_electric_fuel_code, @@ -2265,6 +2268,9 @@ def _hot_water_fuel_cost_gbp_per_kwh( *, water_heating_code: Optional[int] = None, inherit_main_for_community_heating: bool = False, + cylinder_volume_l: Optional[float] = None, + occupancy_n: Optional[float] = None, + immersion_single: Optional[bool] = None, ) -> float: """Hot water bills at the *water-heating* fuel's rate. When the water-heating fuel is electric AND tariff is off-peak, bill at the @@ -2278,10 +2284,18 @@ def _hot_water_fuel_cost_gbp_per_kwh( ∈ {901, 902, 914}) and that main is a PCDB Table 362 heat pump, the HW bills per SAP 10.2 Table 12a Grid 1 WH column (PDF p.191) — the ASHP/GSHP-from-database row carries a 0.70 high-rate fraction at - 7-hour and 10-hour, NOT 100% off-peak low rate. Electric IMMERSION - (WHC 903) is a different Table 12a row (off-peak immersion 0.17 / - Table 13) and stays on the 100%-low-rate fallback until that slice - lands. + 7-hour and 10-hour, NOT 100% off-peak low rate. + + Electric IMMERSION exception (WHC 903): Table 12a's "Immersion water + heater" row (PDF p.191) routes the WH column to Table 13 (PDF p.197). + The Table 13 high-rate fraction — a function of cylinder volume, + assumed occupancy and single-/dual-immersion — gives the proportion + billed at the high rate, the remainder at the low rate. Without it + the immersion HW billed 100% at the off-peak low rate, under-costing + the dwelling and over-rating it (median +0.98 SAP across the off-peak + WHC-903 API cohort). Needs `cylinder_volume_l` + `occupancy_n` + + `immersion_single`; absent any of them (no cylinder / volume not + resolvable) it falls back to the 100%-low-rate scalar. `inherit_main_for_community_heating`: per S0380.173, when WHC ∈ {901, 902, 914} AND main is a heat network, ignore the cert- @@ -2306,6 +2320,21 @@ def _hot_water_fuel_cost_gbp_per_kwh( ) blended = high_frac * high_rate + (1.0 - high_frac) * low_rate return blended * _PENCE_TO_GBP + if ( + water_heating_code == _WHC_ELECTRIC_IMMERSION + and cylinder_volume_l is not None + and occupancy_n is not None + and immersion_single is not None + ): + high_rate, low_rate = _tariff_high_low_rates_p_per_kwh(tariff) + high_frac = electric_dhw_high_rate_fraction( + cylinder_volume_l=cylinder_volume_l, + occupancy_n=occupancy_n, + single_immersion=immersion_single, + tariff=tariff, + ) + blended = high_frac * high_rate + (1.0 - high_frac) * low_rate + return blended * _PENCE_TO_GBP return _off_peak_low_rate_gbp_per_kwh(tariff) if water_heating_fuel is not None: return prices.unit_price_p_per_kwh(water_heating_fuel) * _PENCE_TO_GBP @@ -4851,6 +4880,17 @@ _CYLINDER_SIZE_CODE_TO_LITRES: Final[dict[int, float]] = { 2: 110.0, 3: 160.0, 4: 210.0 } +# RdSAP `immersion_heating_type` lodgement codes. Code 1 = DUAL immersion, +# code 2 = SINGLE. Confirmed against RdSAP 10 §10.5 (PDF p.54 — an +# immersion is "assumed dual" on a dual/off-peak meter) cross-checked +# with the API cohort: code 1 sits 3.6:1 on dual meters (40 vs 11 single) +# while code 2 sits on single meters (22 single vs 16 dual). This INVERTS +# the unverified "1=single, 2=dual" annotation in an earlier handover — +# the dual code (1) carries Table 13's small high-rate fraction, matching +# the cohort's over-rating direction; treating code 1 as single overshot. +_IMMERSION_TYPE_DUAL: Final[int] = 1 +_IMMERSION_TYPE_SINGLE: Final[int] = 2 + # RdSAP 10 §10.5 code 7-11: cylinder insulation type. Empirical mapping # from the ASHP cohort (all 7 certs lodge code 1, worksheet shows # "Foam" → factory-applied per SAP 10.2 Table 2 Note 2). @@ -5673,6 +5713,21 @@ def _hot_water_cylinder_volume_l(epc: EpcPropertyData) -> Optional[float]: return _CYLINDER_SIZE_CODE_TO_LITRES.get(size_code) +def _immersion_is_single(epc: EpcPropertyData) -> Optional[bool]: + """True for a single immersion, False for a dual immersion, None when + the cert lodges no recognised `immersion_heating_type`. Maps the + RdSAP code (1 = dual, 2 = single — see `_IMMERSION_TYPE_DUAL`). + None makes the Table 13 high-rate-fraction caller fall back to the + 100%-low-rate scalar rather than guess the immersion configuration. + """ + code = _int_or_none(epc.sap_heating.immersion_heating_type) + if code == _IMMERSION_TYPE_DUAL: + return False + if code == _IMMERSION_TYPE_SINGLE: + return True + return None + + def _table_3a_combi_loss_default_applies(main: Optional[MainHeatingDetail]) -> bool: """Gate for the Table 3a keep-hot 600 kWh/yr fall-through per SAP 10.2 §4 line 7702. Returns True only when the main heating system is in the @@ -7058,6 +7113,9 @@ def cert_to_inputs( prices, water_heating_code=epc.sap_heating.water_heating_code, inherit_main_for_community_heating=_community_hw_inherit, + cylinder_volume_l=_hot_water_cylinder_volume_l(epc), + occupancy_n=wh_result.occupancy if wh_result is not None else None, + immersion_single=_immersion_is_single(epc), ) hw_co2_factor = _hot_water_co2_factor_kg_per_kwh( epc, hw_monthly_kwh_for_factors, _rdsap_tariff(epc), diff --git a/domain/sap10_calculator/tables/table_13.py b/domain/sap10_calculator/tables/table_13.py new file mode 100644 index 00000000..f96bd71d --- /dev/null +++ b/domain/sap10_calculator/tables/table_13.py @@ -0,0 +1,70 @@ +"""SAP 10.2 Table 13 — high-rate fraction for electric DHW heating. + +Sourced verbatim from `domain/sap10_calculator/docs/specs/sap-10-2-full- +specification-2025-03-14.pdf`, page 197 (Table 13). RdSAP10 §10.5 (PDF +p.54) routes electric immersion water heating here via Table 12a's +"Immersion water heater" row, whose Water-heating column reads "Fraction +from Table 13". + +The table gives the fraction of DHW electricity consumed at the HIGH +rate for a cylinder with a single or dual immersion heater on an +off-peak tariff; the remainder is at the low rate. Note 2 of the table +supplies exact equations equivalent to the tabulated (rounded) grid — +this module evaluates those equations, so no floor-area interpolation +is needed: + + 7-hour tariff (>= 7 hours/day at the low rate) + Dual: [(6.8 - 0.024 V) N + 14 - 0.07 V] / 100 + Single: [(14530 - 762 N) / V - 80 + 10 N] / 100 + + 10-hour tariff (>= 10 hours/day at the low rate) + Dual: [(6.8 - 0.036 V) N + 14 - 0.105 V] / 100 + Single: [(14530 - 762 N) / (1.5 V) - 80 + 10 N] / 100 + +where V is the cylinder volume (litres) and N is the assumed occupancy +(Appendix J Table 1b). Per Note 2 the result is clamped to [0, 1]. Per +Note 1 the 10-hour equations apply to any tariff providing at least 10 +hours/day at the low rate (so 18-hour and 24-hour use the 10-hour +column). Heat pumps providing water heating only are treated as dual +immersion (Note 1) — out of scope of this helper (callers route those +via Table 12a). +""" + +from __future__ import annotations + +from domain.sap10_calculator.tables.table_12a import Tariff + + +def electric_dhw_high_rate_fraction( + *, + cylinder_volume_l: float, + occupancy_n: float, + single_immersion: bool, + tariff: Tariff, +) -> float: + """SAP 10.2 Table 13 (PDF p.197) high-rate fraction for an electric + immersion DHW cylinder on an off-peak tariff. + + `single_immersion` selects the single- vs dual-immersion equation + (RdSAP10 §10.5 p.54: an immersion is assumed dual on a dual meter). + The 7-hour tariff uses the 7-hour equations; every other off-peak + tariff (10/18/24-hour, all >= 10 hours low-rate per Note 1) uses the + 10-hour equations. STANDARD has no off-peak split and is rejected — + callers must early-return before this fires. + """ + if tariff is Tariff.STANDARD: + raise ValueError("Table 13 high-rate fraction is undefined for STANDARD") + v = cylinder_volume_l + n = occupancy_n + if tariff is Tariff.SEVEN_HOUR: + if single_immersion: + fraction = ((14530 - 762 * n) / v - 80 + 10 * n) / 100 + else: + fraction = ((6.8 - 0.024 * v) * n + 14 - 0.07 * v) / 100 + else: + # >= 10 hours/day at the low rate (10/18/24-hour) — Note 1. + if single_immersion: + fraction = ((14530 - 762 * n) / (1.5 * v) - 80 + 10 * n) / 100 + else: + fraction = ((6.8 - 0.036 * v) * n + 14 - 0.105 * v) / 100 + return max(0.0, min(1.0, fraction)) diff --git a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py index d0f12b4d..83a65467 100644 --- a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py +++ b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py @@ -3271,8 +3271,9 @@ def test_hot_water_from_pcdb_heat_pump_bills_at_app_n_wh_high_rate() -> None: # 10-hour. `_hot_water_fuel_cost_gbp_per_kwh` previously billed any # electric off-peak HW at 100% low rate (its TODO), over-crediting the # HP-DHW cat-4 cluster. Electric IMMERSION (WHC 903) is a different - # Table 12a row (off-peak immersion 0.17 / Table 13) and must stay on - # the 100%-low-rate fallback here. + # Table 12a row (Table 13) — without the cylinder volume / occupancy / + # immersion-type inputs (not passed here) it falls back to the + # 100%-low-rate scalar; the Table 13 blend is locked separately below. from domain.sap10_calculator.tables.table_12a import Tariff from domain.sap10_calculator.rdsap.cert_to_inputs import ( _hot_water_fuel_cost_gbp_per_kwh, # pyright: ignore[reportPrivateUsage] @@ -3305,6 +3306,56 @@ def test_hot_water_from_pcdb_heat_pump_bills_at_app_n_wh_high_rate() -> None: assert abs(rate_immersion - 0.0750) <= 1e-6 +def test_hot_water_immersion_off_peak_bills_at_table_13_blend() -> None: + # Arrange — SAP 10.2 Table 12a (PDF p.191) "Immersion water heater" + # row routes the WH column to Table 13 (PDF p.197). For an electric + # immersion (WHC 903) on an off-peak tariff with a known cylinder + # volume + occupancy + immersion type, the HW bills at the Table 13 + # high-rate fraction blend, NOT 100% at the off-peak low rate. A dual + # immersion (small fraction) bills only a little above the low rate; a + # single immersion (large fraction) bills much closer to the high rate. + # Pre-slice both billed 100% at the 7-hour low rate 5.50 p (£0.0550), + # under-costing the dwelling and over-rating it (median +0.98 SAP + # across the off-peak WHC-903 API cohort). + from domain.sap10_calculator.tables.table_12a import Tariff + from domain.sap10_calculator.rdsap.cert_to_inputs import ( + _hot_water_fuel_cost_gbp_per_kwh, # pyright: ignore[reportPrivateUsage] + ) + from domain.sap10_calculator.tables.table_13 import ( + electric_dhw_high_rate_fraction, + ) + high_p, low_p = 15.29, 5.50 # Table 32 codes 32 / 31 (7-hour) + n_occupants = 2.7395 # Appendix J Table 1b N at 100 m² + + # Act — 110 L cylinder, occupancy N(100), dual (False) vs single (True). + rate_dual = _hot_water_fuel_cost_gbp_per_kwh( + 29, None, Tariff.SEVEN_HOUR, SAP_10_2_SPEC_PRICES, + water_heating_code=903, cylinder_volume_l=110.0, + occupancy_n=n_occupants, immersion_single=False, + ) + rate_single = _hot_water_fuel_cost_gbp_per_kwh( + 29, None, Tariff.SEVEN_HOUR, SAP_10_2_SPEC_PRICES, + water_heating_code=903, cylinder_volume_l=110.0, + occupancy_n=n_occupants, immersion_single=True, + ) + + # Assert — each rate equals its Table 13 blend; single > dual; both + # strictly above the 100%-low fallback (5.50 p) it replaces. + frac_dual = electric_dhw_high_rate_fraction( + cylinder_volume_l=110.0, occupancy_n=n_occupants, + single_immersion=False, tariff=Tariff.SEVEN_HOUR, + ) + frac_single = electric_dhw_high_rate_fraction( + cylinder_volume_l=110.0, occupancy_n=n_occupants, + single_immersion=True, tariff=Tariff.SEVEN_HOUR, + ) + expected_dual = (frac_dual * high_p + (1 - frac_dual) * low_p) / 100 + expected_single = (frac_single * high_p + (1 - frac_single) * low_p) / 100 + assert abs(rate_dual - expected_dual) <= 1e-9 + assert abs(rate_single - expected_single) <= 1e-9 + assert rate_single > rate_dual > 0.0550 + + def test_space_heating_pcdb_heat_pump_without_sap_code_bills_at_app_n_high_rate() -> None: # Arrange — an API-path heat pump resolves via its PCDB Table 362 # index alone (data_source=1, no Table-4a SAP code lodged), so diff --git a/tests/domain/sap10_calculator/test_table_13.py b/tests/domain/sap10_calculator/test_table_13.py new file mode 100644 index 00000000..d6857cc2 --- /dev/null +++ b/tests/domain/sap10_calculator/test_table_13.py @@ -0,0 +1,103 @@ +"""SAP 10.2 Table 13 — high-rate fraction for electric DHW heating. + +Locks `electric_dhw_high_rate_fraction` against the published table grid +and the Note-2 clamp at +`domain/sap10_calculator/docs/specs/sap-10-2-full-specification-2025-03-14.pdf`, +page 197. Table 12a's "Immersion water heater" row (PDF p.191) routes +electric immersion DHW here. + +The helper evaluates the Note-2 equations, which the spec offers as an +exact alternative to the rounded grid — the pins below check that the +equations reproduce the published 2-dp cells. +""" +from __future__ import annotations + +from domain.sap10_calculator.tables.table_12a import Tariff +from domain.sap10_calculator.tables.table_13 import ( + electric_dhw_high_rate_fraction, +) + +# Appendix J Table 1b occupancy N at a few total floor areas (m²) — the +# anchor for the V/N grid cells below. Computed from the same piecewise +# formula the §4 worksheet uses (water_heating.assumed_occupancy). +_N_AT_TFA_100 = 2.7395 # N(100 m²) +_N_AT_TFA_60 = 1.9816 # N(60 m²) + +# Table 13 high-rate-fraction grid cells (PDF p.197), keyed by (floor +# area row, cylinder litres column, tariff, single?) → published value. +_GRID_TOL = 0.005 # the published grid is rounded to 2 dp + + +def test_table_13_dual_immersion_matches_published_grid() -> None: + # Arrange — SAP 10.2 Table 13 (PDF p.197), 110 L cylinder, dual + # immersion. Floor area 100 m² row: 7-hour = 0.18, 10-hour = 0.10. + + # Act + seven = electric_dhw_high_rate_fraction( + cylinder_volume_l=110.0, occupancy_n=_N_AT_TFA_100, + single_immersion=False, tariff=Tariff.SEVEN_HOUR, + ) + ten = electric_dhw_high_rate_fraction( + cylinder_volume_l=110.0, occupancy_n=_N_AT_TFA_100, + single_immersion=False, tariff=Tariff.TEN_HOUR, + ) + + # Assert + assert abs(seven - 0.18) <= _GRID_TOL + assert abs(ten - 0.10) <= _GRID_TOL + + +def test_table_13_single_immersion_matches_published_grid() -> None: + # Arrange — SAP 10.2 Table 13 (PDF p.197), 110 L cylinder, single + # immersion. Floor area 100 m² row: 7-hour = 0.61, 10-hour = 0.23. + # Single immersion carries a much larger high-rate fraction than dual. + + # Act + seven = electric_dhw_high_rate_fraction( + cylinder_volume_l=110.0, occupancy_n=_N_AT_TFA_100, + single_immersion=True, tariff=Tariff.SEVEN_HOUR, + ) + ten = electric_dhw_high_rate_fraction( + cylinder_volume_l=110.0, occupancy_n=_N_AT_TFA_100, + single_immersion=True, tariff=Tariff.TEN_HOUR, + ) + + # Assert + assert abs(seven - 0.61) <= _GRID_TOL + assert abs(ten - 0.23) <= _GRID_TOL + + +def test_table_13_large_cylinder_single_immersion_clamps_to_zero() -> None: + # Arrange — SAP 10.2 Table 13 Note 2 (PDF p.197): "If these formulae + # give a value less than zero, set the high-rate fraction to zero." A + # 210 L cylinder with single immersion on a 10-hour tariff falls below + # zero (the published 210 L 10-hour column is 0), so the helper clamps. + + # Act + fraction = electric_dhw_high_rate_fraction( + cylinder_volume_l=210.0, occupancy_n=_N_AT_TFA_60, + single_immersion=True, tariff=Tariff.TEN_HOUR, + ) + + # Assert + assert fraction == 0.0 + + +def test_table_13_eighteen_hour_uses_ten_hour_column() -> None: + # Arrange — SAP 10.2 Table 13 Note 1 (PDF p.197): the table applies + # "for tariffs providing at least 10 hours ... at the low rate", so an + # 18-hour tariff resolves to the 10-hour equations, not a separate + # column. + + # Act + eighteen = electric_dhw_high_rate_fraction( + cylinder_volume_l=110.0, occupancy_n=_N_AT_TFA_100, + single_immersion=False, tariff=Tariff.EIGHTEEN_HOUR, + ) + ten = electric_dhw_high_rate_fraction( + cylinder_volume_l=110.0, occupancy_n=_N_AT_TFA_100, + single_immersion=False, tariff=Tariff.TEN_HOUR, + ) + + # Assert + assert abs(eighteen - ten) <= 1e-9 From 152682d80235df4d6e81a7b227a98890e69c506e Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 8 Jun 2026 15:48:25 +0000 Subject: [PATCH 06/87] =?UTF-8?q?docs:=20session-5=20handover=20=E2=80=94?= =?UTF-8?q?=20WHC-903=20immersion=20off-peak=20HW=20(Table=2013)=20closed?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 47.6% → 48.6% within 0.5; immersion code mapping corrected (1=dual, 2=single); next robust leads are under-rating flat/party-fabric scatter. Co-Authored-By: Claude Opus 4.8 --- docs/HANDOVER_API_PROFILING.md | 42 +++++++++++++++++++++++++++------- 1 file changed, 34 insertions(+), 8 deletions(-) diff --git a/docs/HANDOVER_API_PROFILING.md b/docs/HANDOVER_API_PROFILING.md index c9c7bec3..581a123e 100644 --- a/docs/HANDOVER_API_PROFILING.md +++ b/docs/HANDOVER_API_PROFILING.md @@ -13,14 +13,40 @@ deproven approaches + the meter/shower data-fidelity findings), and the earlier `energy_rating_current`. Headline gauge: `PYTHONPATH=/workspaces/model python scripts/eval_api_sap_accuracy.py`. -| metric | session-3 (`a8e5563a`) | **session-4 (`faf29942`)** | -|--------|------------------|------------------| -| **% \|err\| < 0.5** | 45.1% | **47.6%** | -| % \|err\| < 1.0 | 59.4% | **62.6%** | -| % \|err\| < 2.0 | 77.7% | **79.6%** | -| mean \|err\| | 1.702 | **1.586** | -| computed / raises | 909 / 0 | **909 / 0** | -| unsupported_schema | 100 (deferred) | 100 (deferred) | +| metric | session-3 (`a8e5563a`) | session-4 (`faf29942`) | **session-5 (`43d4c67d`)** | +|--------|------------------|------------------|------------------| +| **% \|err\| < 0.5** | 45.1% | 47.6% | **48.6%** | +| % \|err\| < 1.0 | 59.4% | 62.6% | **63.8%** | +| % \|err\| < 2.0 | 77.7% | 79.6% | **79.9%** | +| mean \|err\| | 1.702 | 1.586 | **1.561** | +| computed / raises | 909 / 0 | 909 / 0 | **909 / 0** | +| unsupported_schema | 100 (deferred) | 100 (deferred) | 100 (deferred) | + +## SESSION-5 UPDATE (HEAD `43d4c67d`) — whc=903 immersion off-peak HW closed + +**Shipped (47.6 → 48.6%):** one spec-grounded fix, the session-4 robust-sweep `whc=903` +lead (median +0.87, n=84). +- `43d4c67d` **WHC-903 electric immersion off-peak HW → SAP 10.2 Table 13 high-rate fraction.** + Was billing 100% at the off-peak low rate; Table 12a "Immersion water heater" row (p.191) routes + the WH column to Table 13 (p.197). New `tables/table_13.py` evaluates the Note-2 equations (f of + cylinder volume V, occupancy N, single/dual immersion), clamped [0,1]; 18-/24-hour use the 10-hour + column (Note 1). Wired into `_hot_water_fuel_cost_gbp_per_kwh` (threads V / N / immersion-single + from the caller; absent any → old 100%-low fallback, no regression). Off-peak WHC-903 cohort + (n=57): within-0.5 16% → 33%, median |err| 1.56 → 0.86. +- **IMMERSION CODE MAPPING CORRECTED: `immersion_heating_type` 1 = DUAL, 2 = SINGLE.** The + session-3 handover lead #3's "(1=single, 2=dual)" was UNVERIFIED and BACKWARDS. Confirmed via + RdSAP 10 §10.5 (p.54 — immersion "assumed dual" on a dual/off-peak meter) cross-checked with the + cohort: code 1 sits 3.6:1 on dual meters (40 vs 11), code 2 on single meters (22 vs 16). Dual + carries Table 13's small fraction → matches the over-rating direction; the single mapping + overshot in a prototype (cohort within-0.5 16% → 14%). The description-vs-code-audit lesson + again: skeptical of unverified handover code-semantics claims. +- **Next robust leads (post-fix sweep, ranked by net directional skew + MEDIAN):** all now + UNDER-rate clusters (negative median = fabric/flat scatter, per-cert not one-bug): `property_type=2` + flats −0.31 (n=283), `wall_construction=3` −0.28 (n=221), roof "(another dwelling above)" −0.32 + (n=182), floor "(another dwelling below)" −0.35 (n=185). The remaining OVER-rate buckets are small: + "Cavity wall, as built, insulated" +0.26 (n=145), "Solid, no insulation" +0.13 (n=304). whc=903 has + dropped off the top of the sweep. `main_heat_cat=7` electric-storage (median +1.05, n=41) is still + open (tariff/cost; partly artifact) — was the session-4 #2 lead, untouched this session. **SESSION-4 shipped (45.1 → 47.6%):** four spec-grounded fixes + closed one false lead. See the `## SESSION-4 …` blocks below and the auto-memory for full detail. The systematic bias From 2e466ed1e6c9382234ec787577c894c210f1acf9 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 8 Jun 2026 18:51:17 +0000 Subject: [PATCH 07/87] fix(wall-U): as-built "insulated (assumed)" cavity uses Cavity-as-built row, not Filled cavity MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An as-built cavity wall (wall_insulation_type=4) lodged "Cavity wall, as built, insulated (assumed)" was routed to RdSAP 10 Table 6's "Filled cavity" row. Per Table 6 (England, PDF p.41) the Filled-cavity row carries the "†" footnote ("assumed as built") only at age bands I-M, where it is numerically identical to "Cavity as built"; at bands A-H the Filled-cavity row represents a GENUINE fill, not the as-built assumption. So an as-built cavity must use the "Cavity as built" row at all bands (band G/H = 0.60, not the filled 0.35). This is the same latent A-H bug slice S0380.210 fixed for the "partial insulation (assumed)" variant but left in place for "insulated (assumed)" by a legacy production convention. The API SAP-accuracy cohort over-rated "Cavity wall, as built, insulated (assumed)" certs at bands G/H by a clean +1.38 / +1.61 SAP median (n=37 / n=18); bands I-M were unaffected (rows coincide), confirming the spec mechanism per-band. Retires the `_cavity_described_as_filled` description sniffer — as-built cavities now always use the as-built row regardless of the rendered insulation adjective; a genuine retrofit fill is still caught by the explicit wall_insulation_type=2 branch. API SAP eval: 48.6% -> 52.1% within 0.5; <1.0 63.8% -> 67.2%; median |err| 0.548 -> 0.475; mean|err| 1.561 -> 1.497; 909 computed, 0 raises. Co-Authored-By: Claude Opus 4.8 --- domain/sap10_ml/rdsap_uvalues.py | 54 +++++++---------- domain/sap10_ml/tests/test_rdsap_uvalues.py | 59 ++++++++++++------- .../worksheet/test_heat_transmission.py | 32 ++++++---- 3 files changed, 77 insertions(+), 68 deletions(-) diff --git a/domain/sap10_ml/rdsap_uvalues.py b/domain/sap10_ml/rdsap_uvalues.py index 2ef935ce..845a72e2 100644 --- a/domain/sap10_ml/rdsap_uvalues.py +++ b/domain/sap10_ml/rdsap_uvalues.py @@ -66,39 +66,26 @@ def _described_as_insulated(description: Optional[str]) -> bool: return "insulated" in desc or "partial insulation" in desc -def _cavity_described_as_filled(description: Optional[str]) -> bool: - """True when an as-built cavity wall's description asserts the cavity is - insulated/filled, routing it to the Table 6 "Filled cavity" row. - - Distinguishes the three as-built cavity states the EPC renders by age - band when wall_insulation_type=4 ("as-built / assumed"): - - - "...insulated (assumed)" → Filled cavity (assessor judges - the cavity filled but lodges no - thickness) - - "...partial insulation (assumed)" → "Cavity as built" row (the - as-built partial fill of the age - band, NOT a retrofit cavity fill) - - "...no insulation (assumed)" → "Cavity as built" row - - Narrower than `_described_as_insulated`: it excludes the "partial - insulation" substring so a "partial insulation (assumed)" cavity stays on - the as-built row. RdSAP 10 Table 6 (England) "Cavity as built" band F = - 1.0 vs "Filled cavity" band F = 0.40 — for an as-built band-F cavity the - filled row understates heat loss by 2.5x. A genuine retrofit fill is - lodged distinctly as "Cavity wall, filled cavity" - (wall_insulation_type=2), handled by the explicit-code branch. - - Real-cert evidence: golden cert 0390-2954-3640 (detached, band F, cavity - type 4, "partial insulation (assumed)") closes all four SAP metrics on - the as-built 1.0 row; the filled 0.40 row under-counts PE by ~28 kWh/m². - """ - if description is None: - return False - desc = description.lower() - if "no insulation" in desc: - return False - return "insulated" in desc +# An AS-BUILT cavity wall (wall_insulation_type=4 / "as-built / assumed", +# however the EPC renders the insulation adjective — "insulated", "partial +# insulation" or "no insulation" "(assumed)") routes to Table 6's "Cavity +# as built" row via the bucketed cascade, NOT the "Filled cavity" row. Per +# RdSAP 10 Table 6 (England) the "Filled cavity" row's † footnote ("assumed +# as built") applies only at age bands I-M, where the two rows are +# numerically identical — so at bands A-H the Filled cavity row represents a +# GENUINE fill, not the as-built assumption. A genuine retrofit fill is +# lodged distinctly as "Cavity wall, filled cavity" (wall_insulation_type=2), +# caught by the explicit-code branch in `u_wall`. +# +# Slice S0380.210 first corrected this for "partial insulation (assumed)" +# (golden 0390-2954-3640, band F → as-built 1.0); the "insulated (assumed)" +# variant was left on the filled row by a legacy production convention. That +# was the SAME latent A-H bug: the API SAP-accuracy cohort over-rated +# "Cavity wall, as built, insulated (assumed)" certs at bands G/H by a clean +# +1.4 / +1.6 SAP median (filled 0.35 vs as-built 0.60), while bands I-M +# were unaffected (rows coincide). The `_cavity_described_as_filled` +# description sniffer is therefore retired — as-built cavities always use the +# as-built row regardless of the rendered insulation adjective. # --------------------------------------------------------------------------- @@ -689,7 +676,6 @@ def u_wall( ) if wall_type == WALL_CAVITY and ( wall_insulation_type == WALL_INSULATION_FILLED_CAVITY - or _cavity_described_as_filled(description) ): return _CAVITY_FILLED_ENG[age_idx] bucket = _insulation_bucket(insulation_thickness_mm, insulation_present) diff --git a/domain/sap10_ml/tests/test_rdsap_uvalues.py b/domain/sap10_ml/tests/test_rdsap_uvalues.py index f7e31c70..2d563dd1 100644 --- a/domain/sap10_ml/tests/test_rdsap_uvalues.py +++ b/domain/sap10_ml/tests/test_rdsap_uvalues.py @@ -114,20 +114,26 @@ def test_u_wall_solid_brick_with_ni_thickness_uses_50mm_row_per_table6_footnote( assert result == pytest.approx(0.55, abs=0.001) -def test_u_wall_cavity_as_built_insulated_assumed_routes_to_filled_cavity_row() -> None: - # Arrange — 1 171 corpus certs (~4% of scanned bulk) lodge - # wall_insulation_type=4 ("as-built / assumed") together with the - # description "Cavity wall, as built, insulated (assumed)". The - # assessor is saying: this cavity is filled, but I haven't measured - # the thickness. Spec footnote on Table 6 covers this: "If a wall - # is known to have additional insulation but the insulation thickness - # is unknown, use the row in the table for 50 mm insulation" — but - # legacy convention (used by the production recommendation engine) - # is to route this to the Filled-cavity row, U = 0.7 at A-E. We - # follow the legacy convention here for parity with the cert assessor. +def test_u_wall_cavity_as_built_insulated_assumed_routes_to_as_built_row() -> None: + # Arrange — a cavity lodged "Cavity wall, as built, insulated (assumed)" + # with wall_insulation_type=4 is in its AS-BUILT state, NOT a retrofit + # cavity fill. Per RdSAP 10 Table 6 (England) the "Filled cavity" row's + # † footnote ("assumed as built") applies only at bands I-M, where it + # coincides with "Cavity as built"; at bands A-H the filled row is for a + # GENUINE fill. So an as-built cavity uses the "Cavity as built" row: + # band E = 1.5, NOT the filled 0.7. + # + # Slice S0380.210 corrected this for the "partial insulation (assumed)" + # variant but left "insulated (assumed)" on the filled row by a legacy + # production convention — the SAME latent A-H bug. The API SAP-accuracy + # cohort over-rated band-G/H "insulated (assumed)" cavities by a clean + # +1.4 / +1.6 SAP median (filled 0.35 vs as-built 0.60); bands I-M were + # unaffected (rows coincide). A genuine fill lodges the distinct "Cavity + # wall, filled cavity" (wall_insulation_type=2), caught by the + # explicit-code branch. # Act - result = u_wall( + result_e = u_wall( country=Country.ENG, age_band="E", construction=WALL_CAVITY, @@ -136,19 +142,30 @@ def test_u_wall_cavity_as_built_insulated_assumed_routes_to_filled_cavity_row() wall_insulation_type=4, description="Cavity wall, as built, insulated (assumed)", ) + # Band I: "Cavity as built" and "Filled cavity" rows coincide (0.45), + # so the routing change is a no-op there — the corpus-confirmed pivot. + result_i = u_wall( + country=Country.ENG, + age_band="I", + construction=WALL_CAVITY, + insulation_thickness_mm=None, + insulation_present=False, + wall_insulation_type=4, + description="Cavity wall, as built, insulated (assumed)", + ) - # Assert - assert result == pytest.approx(0.7, abs=0.001) + # Assert — band E → as-built 1.5 (not filled 0.7); band I → 0.45 (rows coincide). + assert abs(result_e - 1.5) <= 0.001 + assert abs(result_i - 0.45) <= 0.001 def test_u_wall_cavity_as_built_no_insulation_stays_at_table6_cavity_as_built_row() -> None: # Arrange — the same wall_insulation_type=4 ("as-built / assumed") # cert population also contains 686 "Cavity wall, as built, no - # insulation (assumed)" entries which must continue to route to the - # Cavity-as-built row of Table 6 (U=1.5 at band E). The "no - # insulation" substring marker takes precedence over the - # "insulated"-substring filled-cavity rule, so this case is - # disambiguated from "Cavity wall, as built, insulated (assumed)". + # insulation (assumed)" entries which route to the Cavity-as-built row + # of Table 6 (U=1.5 at band E) — as do ALL as-built cavity variants + # ("insulated" / "partial insulation" / "no insulation") now that the + # as-built path no longer special-cases the insulation adjective. # Act result = u_wall( @@ -180,8 +197,8 @@ def test_u_wall_cavity_as_built_partial_insulation_routes_to_as_built_row() -> N # four SAP metrics on the as-built row (band F = 1.0) and under-counts # PE by ~28 kWh/m² on the filled row — the legacy parity was a latent # bug at bands A-H (bands I-M coincide per the Table 6 † footnote). - # The "insulated (assumed)" variant still routes to filled (see the - # heat_transmission `_cavity_described_as_filled` sibling test). + # A later slice extended the same fix to the "insulated (assumed)" + # variant (see the as-built-insulated sibling test above). # Act result = u_wall( diff --git a/tests/domain/sap10_calculator/worksheet/test_heat_transmission.py b/tests/domain/sap10_calculator/worksheet/test_heat_transmission.py index 24fbe082..92967423 100644 --- a/tests/domain/sap10_calculator/worksheet/test_heat_transmission.py +++ b/tests/domain/sap10_calculator/worksheet/test_heat_transmission.py @@ -327,16 +327,22 @@ def test_solid_brick_as_built_no_insulation_assumed_stays_at_table6_as_built_row assert result.walls_w_per_k == pytest.approx(170.0, abs=1.0) -def test_cavity_as_built_insulated_assumed_uses_filled_cavity_row() -> None: - # Arrange — the modal RdSAP encoding for a retrofitted-cavity dwelling: - # wall_construction=4 (cavity), wall_insulation_type=4 (as-built / - # assumed), and walls[0].description = "Cavity wall, as built, - # insulated (assumed)". The assessor has determined the cavity is - # filled but hasn't lodged a thickness. Without the description-based - # dispatcher, the cascade would return U=1.5; with it, the Filled- - # cavity row of Table 6 applies: U=0.7 at band E. +def test_cavity_as_built_insulated_assumed_uses_as_built_row() -> None: + # Arrange — wall_construction=4 (cavity), wall_insulation_type=4 + # (as-built / assumed), walls[0].description = "Cavity wall, as built, + # insulated (assumed)". This is the AS-BUILT state, not a retrofit fill: + # per RdSAP 10 Table 6 (England) the "Filled cavity" row's † footnote + # ("assumed as built") applies only at bands I-M, where it coincides + # with "Cavity as built"; at bands A-H the filled row is for a genuine + # fill. So band E uses the "Cavity as built" row U=1.5, NOT filled 0.7. + # + # Prior code special-cased the "insulated" adjective to the filled row + # (legacy convention); the API SAP-accuracy cohort over-rated band-G/H + # "insulated (assumed)" cavities by +1.4 / +1.6 SAP median (filled 0.35 + # vs as-built 0.60). A genuine fill renders the distinct "Cavity wall, + # filled cavity" (wall_insulation_type=2), caught separately. # Geometry: 100 m² floor, 40 m perimeter, 2.5 m height, single storey - # → gross_wall = 100 m². walls_w_per_k expected = 0.7 × 100 = 70 W/K. + # → gross_wall = 100 m². walls_w_per_k expected = 1.5 × 100 = 150 W/K. main = make_building_part( identifier="Main Dwelling", construction_age_band="E", @@ -367,8 +373,8 @@ def test_cavity_as_built_insulated_assumed_uses_filled_cavity_row() -> None: # Act result = heat_transmission_from_cert(epc) - # Assert - assert result.walls_w_per_k == pytest.approx(70.0, abs=1.0) + # Assert — Cavity-as-built row at band E = 1.5 W/m²K (not filled 0.7). + assert result.walls_w_per_k == pytest.approx(150.0, abs=1.0) def test_cavity_as_built_partial_insulation_assumed_uses_as_built_row() -> None: @@ -381,8 +387,8 @@ def test_cavity_as_built_partial_insulation_assumed_uses_as_built_row() -> None: # RdSAP 10 Table 6 (England) "Cavity as built" band F = 1.0 vs # "Filled cavity" band F = 0.40. A genuine fill renders the distinct # "Cavity wall, filled cavity" description (wall_insulation_type=2), - # caught separately. Contrast the "insulated (assumed)" variant above, - # which the assessor judges as filled. + # caught separately. The "insulated (assumed)" variant above now routes + # to the same as-built row (all as-built adjectives coincide at A-H). # # Real-cert evidence: golden cert 0390-2954-3640 (detached, band F, # cavity type 4, "partial insulation (assumed)") closes all four SAP From 898dcfda1815ec28267d9f2e342450c8e3dbf2fd Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 8 Jun 2026 18:53:01 +0000 Subject: [PATCH 08/87] =?UTF-8?q?docs:=20session-5=20handover=20=E2=80=94?= =?UTF-8?q?=20as-built=20cavity-U=20fix=20(48.6=20=E2=86=92=2052.1%)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds the cavity wall-U slice to the SESSION-5 block + headline table; records the by-age-band re-split method that surfaced the G/H spike. Co-Authored-By: Claude Opus 4.8 --- docs/HANDOVER_API_PROFILING.md | 48 ++++++++++++++++++++++++---------- 1 file changed, 34 insertions(+), 14 deletions(-) diff --git a/docs/HANDOVER_API_PROFILING.md b/docs/HANDOVER_API_PROFILING.md index 581a123e..7f5ab34c 100644 --- a/docs/HANDOVER_API_PROFILING.md +++ b/docs/HANDOVER_API_PROFILING.md @@ -13,18 +13,35 @@ deproven approaches + the meter/shower data-fidelity findings), and the earlier `energy_rating_current`. Headline gauge: `PYTHONPATH=/workspaces/model python scripts/eval_api_sap_accuracy.py`. -| metric | session-3 (`a8e5563a`) | session-4 (`faf29942`) | **session-5 (`43d4c67d`)** | +| metric | session-3 (`a8e5563a`) | session-4 (`faf29942`) | **session-5 (`2e466ed1`)** | |--------|------------------|------------------|------------------| -| **% \|err\| < 0.5** | 45.1% | 47.6% | **48.6%** | -| % \|err\| < 1.0 | 59.4% | 62.6% | **63.8%** | -| % \|err\| < 2.0 | 77.7% | 79.6% | **79.9%** | -| mean \|err\| | 1.702 | 1.586 | **1.561** | +| **% \|err\| < 0.5** | 45.1% | 47.6% | **52.1%** | +| % \|err\| < 1.0 | 59.4% | 62.6% | **67.2%** | +| % \|err\| < 2.0 | 77.7% | 79.6% | **80.7%** | +| mean \|err\| | 1.702 | 1.586 | **1.497** | +| median \|err\| | — | — | **0.475** | | computed / raises | 909 / 0 | 909 / 0 | **909 / 0** | | unsupported_schema | 100 (deferred) | 100 (deferred) | 100 (deferred) | -## SESSION-5 UPDATE (HEAD `43d4c67d`) — whc=903 immersion off-peak HW closed +## SESSION-5 UPDATE (HEAD `2e466ed1`) — whc=903 immersion HW + as-built cavity-U both closed -**Shipped (47.6 → 48.6%):** one spec-grounded fix, the session-4 robust-sweep `whc=903` +**Shipped (47.6 → 52.1%, two spec-grounded fixes):** + +**(2) `2e466ed1` as-built "insulated (assumed)" cavity → Cavity-as-built row, not Filled cavity +(48.6 → 52.1%, the bigger win).** Robust-sweep lead: `wall_desc="Cavity wall, as built, insulated +(assumed)"` median +0.26, n=145, but split by age band it was a CLEAN G/H signal (G +1.38 n37, +H +1.61 n18; I-L neutral). RdSAP 10 Table 6 (England, p.41) "Filled cavity" row carries the † footnote +("assumed as built") ONLY at bands I-M, where it equals "Cavity as built"; at A-H the filled row is a +GENUINE fill. An as-built cavity (type 4) must use "Cavity as built" at all bands (G/H 0.60 not 0.35). +This was the SAME latent A-H bug slice S0380.210 fixed for "partial insulation (assumed)" but left for +"insulated (assumed)" by a legacy convention. Retired `_cavity_described_as_filled`; genuine fills +(wall_insulation_type=2) still hit the filled row. Per-band confirmation: I-M unchanged, G/H corrected +exactly. Bucket within-0.5 47% → 66%; eval +32 net (36 improved, 4 regressed — offsetting-error +electric-storage flats). 3 tests updated to the corrected behaviour (the legacy tests literally said +"we follow the legacy convention for parity"). + +**(1) `43d4c67d` WHC-903 electric immersion off-peak HW → SAP 10.2 Table 13 high-rate fraction +(47.6 → 48.6%).** The session-4 robust-sweep `whc=903` lead (median +0.87, n=84). - `43d4c67d` **WHC-903 electric immersion off-peak HW → SAP 10.2 Table 13 high-rate fraction.** Was billing 100% at the off-peak low rate; Table 12a "Immersion water heater" row (p.191) routes @@ -40,13 +57,16 @@ lead (median +0.87, n=84). carries Table 13's small fraction → matches the over-rating direction; the single mapping overshot in a prototype (cohort within-0.5 16% → 14%). The description-vs-code-audit lesson again: skeptical of unverified handover code-semantics claims. -- **Next robust leads (post-fix sweep, ranked by net directional skew + MEDIAN):** all now - UNDER-rate clusters (negative median = fabric/flat scatter, per-cert not one-bug): `property_type=2` - flats −0.31 (n=283), `wall_construction=3` −0.28 (n=221), roof "(another dwelling above)" −0.32 - (n=182), floor "(another dwelling below)" −0.35 (n=185). The remaining OVER-rate buckets are small: - "Cavity wall, as built, insulated" +0.26 (n=145), "Solid, no insulation" +0.13 (n=304). whc=903 has - dropped off the top of the sweep. `main_heat_cat=7` electric-storage (median +1.05, n=41) is still - open (tariff/cost; partly artifact) — was the session-4 #2 lead, untouched this session. +- **Next robust leads (post-BOTH-fixes sweep, ranked by net directional skew + MEDIAN):** every + top bucket is now an UNDER-rate cluster (negative median = fabric/flat scatter, per-cert not one-bug): + `property_type=2` flats med −0.39 (n=283, netDir +75), roof "(another dwelling above)" −0.46 (n=182), + `wall_desc="Solid brick, as built, no insulation"` −0.22 (n=114). No clean OVER-rate single-cause + bucket remains (cavity-insulated dropped to −0.13, main_heat_cat=7 to −0.31, whc=903 off the top — + all addressed). The flats under-rate is the biggest front but DIFFUSE (fabric/tariff per-cert) — likely + needs worksheets, not one rule. The 100 unsupported-schema certs remain the deferred big ticket. + METHOD NOTE: the cavity win came from splitting the +0.26 bucket BY AGE BAND — the mild median hid a + sharp G/H spike. When a description bucket has a modest median but a plausible single mechanism, + re-split by age band / sub-field before dismissing it as scatter. **SESSION-4 shipped (45.1 → 47.6%):** four spec-grounded fixes + closed one false lead. See the `## SESSION-4 …` blocks below and the auto-memory for full detail. The systematic bias From 58cff932e68f99167871dde02f59045f5ea976c3 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 8 Jun 2026 19:26:03 +0000 Subject: [PATCH 09/87] =?UTF-8?q?fix(roof-U):=20flat=20roof,=20undetermine?= =?UTF-8?q?d=20thickness,=20"no/limited=20insulation"=20=E2=86=92=20Table?= =?UTF-8?q?=2018=20age=20default?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A flat roof lodged "Flat, no insulation" / "Flat, limited insulation" with an undetermined insulation thickness ('ND'/'AB' → parsed None) was given the Table 16 row-0/12mm U (2.30 / 1.50) from the description marker, regardless of age band. Per RdSAP 10 §5.11.4 (PDF p.44) "U-values in Table 18 are used when thickness of insulation cannot be determined" — the column (3) flat-roof age-band default applies. The "no/limited insulation" text is RdSAP's as-built rendering: at old bands (A-D) the column (3) default IS 2.30 (so those certs are unchanged), but a newer-band flat roof carries the age-band insulation as built (band H = 0.35, F = 0.68, not 2.30). Confirmed by the description-vs-rating audit: cert 0390-2753 (top-floor flat, band H, "Flat, no insulation", thickness 'ND') lodges roof energy_efficiency_rating = 3 (moderate U), NOT the rating-1 that 2.30 implies — and drove a -31.78 SAP error (roof 202 W/K over 88 m²). Same masked-at-old-bands structure as the cavity-U fix: accurate at A-D where the default coincides with 2.30, catastrophic only where it diverges. Pitched roofs are deliberately NOT rerouted (their "no insulation" text is load-bearing — the broad 'ND'→Table-18 reroute was empirically net-negative for pitched lofts). API SAP eval: 52.1% -> 53.1% within 0.5; <1.0 67.2% -> 68.0%; median |err| 0.475 -> 0.467; mean|err| 1.497 -> 1.424; flat-roof bucket within-0.5 23% -> 35% (11 improved, 2 regressed). Co-Authored-By: Claude Opus 4.8 --- domain/sap10_ml/rdsap_uvalues.py | 25 +++++++++++-- domain/sap10_ml/tests/test_rdsap_uvalues.py | 40 +++++++++++++++++++++ 2 files changed, 63 insertions(+), 2 deletions(-) diff --git a/domain/sap10_ml/rdsap_uvalues.py b/domain/sap10_ml/rdsap_uvalues.py index 845a72e2..1f37b222 100644 --- a/domain/sap10_ml/rdsap_uvalues.py +++ b/domain/sap10_ml/rdsap_uvalues.py @@ -857,9 +857,30 @@ def u_roof( return u if description is not None: desc = description.lower() - if any(marker in desc for marker in _ROOF_NO_INSULATION_MARKERS): + no_insulation = any(marker in desc for marker in _ROOF_NO_INSULATION_MARKERS) + limited_insulation = any( + marker in desc for marker in _ROOF_LIMITED_INSULATION_MARKERS + ) + if (no_insulation or limited_insulation) and is_flat_roof and age_band is not None: + # FLAT roof reached here only when insulation_thickness_mm is None + # — the lodged thickness is undetermined ('ND'/'AB'/absent). Per + # RdSAP 10 §5.11.4 (PDF p.44) "U-values in Table 18 are used when + # thickness of insulation cannot be determined", so the column (3) + # flat-roof age-band default applies — NOT the uninsulated 2.30. + # The "no/limited insulation" text is RdSAP's as-built rendering: + # at old bands (A-D) the column (3) default IS 2.30 (so those certs + # are unchanged), but a newer-band flat roof carries the age-band + # insulation as built (band H = 0.35, F = 0.68, not 2.30). The + # roof's deterministic energy_efficiency_rating confirms it (e.g. + # cert 0390-2753 band H lodges rating 3 = moderate U, not the + # rating-1 that 2.30 implies). PITCHED roofs are deliberately NOT + # routed here — their "no insulation" text is load-bearing (the + # broad 'ND'→Table-18 reroute was empirically net-negative for + # pitched lofts). + return _FLAT_ROOF_BY_AGE.get(age_band.upper(), 0.4) + if no_insulation: return _ROOF_BY_THICKNESS[0][1] # 2.30 W/m^2K - if any(marker in desc for marker in _ROOF_LIMITED_INSULATION_MARKERS): + if limited_insulation: return _ROOF_BY_THICKNESS[1][1] # 1.50 W/m^2K (12mm row) if age_band is None: return 0.4 diff --git a/domain/sap10_ml/tests/test_rdsap_uvalues.py b/domain/sap10_ml/tests/test_rdsap_uvalues.py index 2d563dd1..5783af45 100644 --- a/domain/sap10_ml/tests/test_rdsap_uvalues.py +++ b/domain/sap10_ml/tests/test_rdsap_uvalues.py @@ -878,6 +878,46 @@ def test_u_roof_unknown_flat_insulation_uses_table18_flat_column() -> None: assert abs(result - 0.35) <= 0.01 +def test_u_roof_flat_no_insulation_undetermined_thickness_uses_table18_by_age() -> None: + # Arrange — a flat roof lodged "Flat, no insulation" / "Flat, limited + # insulation" with an UNDETERMINED thickness (parsed to None from + # 'ND'/'AB') must take the Table 18 column (3) flat-roof age-band + # default per RdSAP 10 §5.11.4 (PDF p.44), NOT the uninsulated 2.30. + # The "no/limited insulation" text is RdSAP's as-built rendering — at + # old bands the column (3) default IS 2.30 (so they're unchanged), but + # a newer-band flat roof carries the age-band insulation as built. + # Cert 0390-2753 (top-floor flat, band H, "Flat, no insulation", + # thickness 'ND', roof rating 3 = moderate) drove a -31.78 SAP error at + # the 2.30 value; band H column (3) = 0.35. + + # Act — band H "no insulation" → 0.35; band F "limited insulation" → 0.68; + # band C "no insulation" → unchanged 2.30 (column (3) default at C). + band_h = u_roof( + country=Country.ENG, age_band="H", insulation_thickness_mm=None, + description="Flat, no insulation", is_flat_roof=True, + ) + band_f = u_roof( + country=Country.ENG, age_band="F", insulation_thickness_mm=None, + description="Flat, limited insulation", is_flat_roof=True, + ) + band_c = u_roof( + country=Country.ENG, age_band="C", insulation_thickness_mm=None, + description="Flat, no insulation", is_flat_roof=True, + ) + # A PITCHED roof "no insulation" with undetermined thickness is NOT + # rerouted — its text is load-bearing (2.30 stays). + pitched = u_roof( + country=Country.ENG, age_band="H", insulation_thickness_mm=None, + description="Pitched, no insulation", is_flat_roof=False, + ) + + # Assert + assert abs(band_h - 0.35) <= 0.01 + assert abs(band_f - 0.68) <= 0.01 + assert abs(band_c - 2.30) <= 0.01 + assert abs(pitched - 2.30) <= 0.01 + + def test_u_roof_age_band_j_pitched_returns_table18_value() -> None: # Arrange — Table 18, pitched insulation between joists, age J -> 0.16 W/m^2K. From d90b6f56434c72c1e62fda81b2323936ee13b709 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 8 Jun 2026 19:30:52 +0000 Subject: [PATCH 10/87] =?UTF-8?q?docs:=20session-5=20handover=20=E2=80=94?= =?UTF-8?q?=20flat-roof=20fix=20+=20the=20unknown-insulation=20principle?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds the flat-roof slice (52.1 → 53.1%) and records the unifying principle ("unknown insulation → as-built age default, not uninsulated") plus the cross-element review confirming all element types now conform. Co-Authored-By: Claude Opus 4.8 --- docs/HANDOVER_API_PROFILING.md | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/docs/HANDOVER_API_PROFILING.md b/docs/HANDOVER_API_PROFILING.md index 7f5ab34c..8adb4f05 100644 --- a/docs/HANDOVER_API_PROFILING.md +++ b/docs/HANDOVER_API_PROFILING.md @@ -13,16 +13,33 @@ deproven approaches + the meter/shower data-fidelity findings), and the earlier `energy_rating_current`. Headline gauge: `PYTHONPATH=/workspaces/model python scripts/eval_api_sap_accuracy.py`. -| metric | session-3 (`a8e5563a`) | session-4 (`faf29942`) | **session-5 (`2e466ed1`)** | +| metric | session-3 (`a8e5563a`) | session-4 (`faf29942`) | **session-5 (`58cff932`)** | |--------|------------------|------------------|------------------| -| **% \|err\| < 0.5** | 45.1% | 47.6% | **52.1%** | -| % \|err\| < 1.0 | 59.4% | 62.6% | **67.2%** | -| % \|err\| < 2.0 | 77.7% | 79.6% | **80.7%** | -| mean \|err\| | 1.702 | 1.586 | **1.497** | -| median \|err\| | — | — | **0.475** | +| **% \|err\| < 0.5** | 45.1% | 47.6% | **53.1%** | +| % \|err\| < 1.0 | 59.4% | 62.6% | **68.0%** | +| % \|err\| < 2.0 | 77.7% | 79.6% | **~81%** | +| mean \|err\| | 1.702 | 1.586 | **1.424** | +| median \|err\| | — | — | **0.467** | | computed / raises | 909 / 0 | 909 / 0 | **909 / 0** | | unsupported_schema | 100 (deferred) | 100 (deferred) | 100 (deferred) | +### THE UNIFYING PRINCIPLE (user, load-bearing) — "unknown insulation → as-built, NOT uninsulated" +An EPC insulation field that is UNDETERMINED (thickness `'ND'`/`'AB'`/absent → parsed None, or +description "as built / (assumed)") must map to the **age-band default** ("as built"), which is +INSULATED at newer bands — never to the uninsulated row. The recurring bug shape: a fixed +uninsulated U (cavity Filled-row, roof Table-16 2.30) is MASKED at old bands (where the age +default coincides with uninsulated) and only diverges (catastrophic under-rate) at newer bands. +All three session-5 fixes are instances. Review status across elements (all now conform): +- **Flat roofs** — FIXED `58cff932` (this slice). **Pitched roofs** — "unknown"→Table 18 (`a64e857b`); + "no insulation" only appears at bands A/B where 2.30 IS the age default (verified, no new-band bug). +- **Cavity walls** — FIXED `2e466ed1`. **System/timber/solid walls** — already on the as-built age + row (verified: bidirectional scatter, not a one-cause under-rate). **Floors** — undetermined + thickness already routes to the Table 19 age default (I=25/J=75/K=100 mm; verified). +- CAVEAT: do NOT broadly reroute PITCHED `'ND'`/`'NI'`→Table 18 (the parsed-0 `'NI'` case) — that + was empirically net-negative (pitched "no insulation" lodgements genuinely use 2.30 even at newer + bands; the description is load-bearing for pitched lofts). The principle holds for flat roofs, + cavity, floors; pitched lofts are the documented exception. + ## SESSION-5 UPDATE (HEAD `2e466ed1`) — whc=903 immersion HW + as-built cavity-U both closed **Shipped (47.6 → 52.1%, two spec-grounded fixes):** From 19235d1144bf6bb4e9bb805d432d94ba130f2539 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 8 Jun 2026 20:31:43 +0000 Subject: [PATCH 11/87] fix(fuel): canonicalise colliding gov-API solid-fuel codes (anthracite/coal) at the fuel-type boundary MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A coal main (gov-API main_fuel_type=33) was priced at the electricity 10-hour low rate (7.5 p) and anthracite (5) at the bulk-LPG rate (12.19 p), because the shared price/CO2/PE lookups check Table-32/12-code membership BEFORE translating the API enum — and codes 5/33 collide with a different-fuel Table code. This drove the cohort's single worst cert (2100 anthracite, -61 SAP). `is_electric_fuel_code(33)` also wrongly classified the coal main as electric. The gov-API fuel enum (confirmed by description-vs-code audit on main_heating[].description): 5=anthracite, 33=coal, 9=dual-fuel, 20/25/31=community. The collision can't be resolved inside the shared table functions — code 33 is ALSO the electricity-10h TARIFF code used by the dual-rate CO2/PE split (golden 000565), so normalising there breaks electricity certs. Instead `canonical_fuel_code` normalises the colliding SOLID-fuel enums (5->15 anthracite, 33->11 house coal) at the fuel-TYPE boundary in `_main_fuel_code` / `_water_heating_fuel_code`, where the code is known to be a fuel type (never a tariff code). Scoped to anthracite (5) + coal (33) — the unambiguous large mispricings. Dual-fuel (9, 0.45 p delta) and community (20/25/31, heat-network path) are deferred (noted in `_GOV_API_COLLISION_FUELS`). API SAP eval: mean|err| 1.424 -> 1.329 (the -61 anthracite outlier 2100 -> -11, residual now fabric); within-0.5 53.1% (flat); 909 computed, 0 raises. Golden + Elmhurst regression green (the shared table functions are unchanged, so the electricity-tariff CO2/PE path is untouched). Co-Authored-By: Claude Opus 4.8 --- .../sap10_calculator/rdsap/cert_to_inputs.py | 13 ++++- domain/sap10_calculator/tables/table_12.py | 2 +- domain/sap10_calculator/tables/table_32.py | 34 ++++++++++- .../rdsap/test_cert_to_inputs.py | 58 ++++++++++++++++++- 4 files changed, 101 insertions(+), 6 deletions(-) diff --git a/domain/sap10_calculator/rdsap/cert_to_inputs.py b/domain/sap10_calculator/rdsap/cert_to_inputs.py index 7b77c669..aef8d53f 100644 --- a/domain/sap10_calculator/rdsap/cert_to_inputs.py +++ b/domain/sap10_calculator/rdsap/cert_to_inputs.py @@ -110,6 +110,7 @@ from domain.sap10_calculator.tables.table_13 import ( ) from domain.sap10_calculator.tables.table_32 import ( additional_standing_charges_gbp, + canonical_fuel_code, is_electric_fuel_code, is_liquid_fuel_code, standing_charge_gbp, @@ -1616,7 +1617,9 @@ def _water_heating_fuel_code(epc: EpcPropertyData) -> Optional[int]: PE/cost lookups returned defaults instead of the gas-combi values. """ if epc.sap_heating.water_heating_fuel: - return epc.sap_heating.water_heating_fuel + # Normalise colliding gov-API solid-fuel enum codes (see + # `_main_fuel_code`) before the shared price/CO2/PE lookups. + return canonical_fuel_code(epc.sap_heating.water_heating_fuel) return _main_fuel_code(_water_heating_main(epc)) @@ -1943,7 +1946,13 @@ def _main_fuel_code(main: Optional[MainHeatingDetail]) -> Optional[int]: return None fuel = main.main_fuel_type if isinstance(fuel, int): - return fuel + # Normalise the colliding gov-API solid-fuel enum codes (5 + # anthracite / 9 dual fuel / 33 coal) to their canonical Table + # 32/12 codes here — at the fuel-TYPE boundary — so the shared + # price/CO2/PE table lookups (which also receive electricity + # TARIFF codes 31/33 for the dual-rate split) never confuse a + # coal fuel-type 33 with the electricity-10h tariff code 33. + return canonical_fuel_code(fuel) raise MissingMainFuelType(fuel, main.sap_main_heating_code) diff --git a/domain/sap10_calculator/tables/table_12.py b/domain/sap10_calculator/tables/table_12.py index 2c884128..dc64e57f 100644 --- a/domain/sap10_calculator/tables/table_12.py +++ b/domain/sap10_calculator/tables/table_12.py @@ -214,7 +214,7 @@ API_FUEL_TO_TABLE_12: Final[dict[int, int]] = { 0: 30, 1: 1, 2: 2, 3: 3, 4: 4, 5: 15, 6: 20, 7: 23, 8: 21, 9: 10, 10: 30, 11: 42, 12: 43, 13: 44, 14: 11, 15: 12, 16: 22, 17: 9, 18: 75, 19: 76, 20: 51, 21: 52, 22: 53, 23: 55, 24: 54, 25: 41, - 26: 1, 27: 2, 28: 4, 29: 30, + 26: 1, 27: 2, 28: 4, 29: 30, 33: 11, } diff --git a/domain/sap10_calculator/tables/table_32.py b/domain/sap10_calculator/tables/table_32.py index 955bad9c..33accf46 100644 --- a/domain/sap10_calculator/tables/table_32.py +++ b/domain/sap10_calculator/tables/table_32.py @@ -105,9 +105,41 @@ API_FUEL_TO_TABLE_32: Final[dict[int, int]] = { 0: 30, 1: 1, 2: 2, 3: 3, 4: 4, 5: 15, 6: 20, 7: 23, 8: 21, 9: 10, 10: 30, 11: 42, 12: 43, 13: 44, 14: 11, 15: 12, 16: 22, 17: 9, 18: 75, 19: 76, 20: 51, 21: 52, 22: 53, 23: 55, 24: 54, 25: 41, - 26: 1, 27: 2, 28: 4, 29: 30, + 26: 1, 27: 2, 28: 4, 29: 30, 33: 11, } +# Gov-API `main_fuel_type` enum codes whose value COLLIDES with a +# same-valued Table 32 code of a DIFFERENT fuel. The gov EPC register +# always lodges the API enum, so for these the API translation is +# authoritative and must win over the direct same-value Table-code +# lookup (which otherwise mis-prices solid fuel at the colliding code's +# rate). Confirmed by the description-vs-code audit on +# `main_heating[].description`: +# 5 = anthracite — Table-32 code 5 is bulk LPG (secondary), 12.19 p +# vs anthracite 3.64 p. Drove the cohort's worst cert (2100, +# -61 SAP at the LPG rate). +# 33 = coal — Table-32 code 33 is the electricity 10-hour low rate +# 7.5 p vs house coal 3.67 p (and `is_electric_fuel_code(33)` +# wrongly classified the coal main as electric). +# DEFERRED (not included): API 9 = dual fuel (mineral + wood) is also a +# collision (Table-32 9 = LPG SC11F 3.48 p vs dual fuel 3.99 p) but the +# 0.45 p delta nets neutral-to-negative on the (outlier-dominated) +# dual-fuel certs and shifts them in a direction not yet understood — +# investigate separately. Community heat-network fuels (20/25/31) are +# also out of scope — their standing-charge / CO2 / PE routing is handled +# by the dedicated heat-network path. +_GOV_API_COLLISION_FUELS: Final[frozenset[int]] = frozenset({5, 33}) + + +def canonical_fuel_code(fuel_code: Optional[int]) -> Optional[int]: + """Normalise a colliding gov-API fuel enum (see + `_GOV_API_COLLISION_FUELS`) to its canonical Table 32 code so the + same-value collision can't mis-resolve it. Non-colliding codes and + already-canonical Table codes pass through unchanged.""" + if fuel_code in _GOV_API_COLLISION_FUELS: + return API_FUEL_TO_TABLE_32.get(fuel_code, fuel_code) + return fuel_code + # RdSAP10 Table 32 — annual standing charge in £/yr per Table 32 fuel # code. Only fuels with a published standing charge appear here; diff --git a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py index 83a65467..d48f947c 100644 --- a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py +++ b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py @@ -1439,8 +1439,14 @@ def test_is_electric_main_dual_fuel_table_32_code_10_is_not_electric() -> None: # Act / Assert — dual fuel (Table 32 10) must NOT be electric assert _is_electric_main(dual_fuel_main) is False - # Sanity — Table 32 electric codes 30-40 still classify as electric - for t32_electric in (30, 31, 32, 33, 34, 35, 38, 40, 60): + # Sanity — Table 32 electric codes still classify as electric. Code 33 + # is EXCLUDED here: as a lodged *main fuel-type* the gov-API enum 33 + # means COAL (description-vs-code audit), which `_main_fuel_code` + # canonicalises to House coal (Table code 11) before `_is_electric_main` + # — so a main fuel-type 33 is NOT electric. The Table 32 electricity-10h + # code 33 is only ever used internally for the dual-rate tariff split + # (never as a main fuel-type), so it is unaffected. + for t32_electric in (30, 31, 32, 34, 35, 38, 40, 60): electric_main = MainHeatingDetail( has_fghrs=False, main_fuel_type=t32_electric, heat_emitter_type=1, emitter_temperature=1, main_heating_control=2105, @@ -1489,6 +1495,54 @@ def test_is_electric_water_dual_fuel_table_32_code_10_is_not_electric() -> None: ) +def test_solid_fuel_main_prices_at_table_32_solid_rate_not_colliding_code() -> None: + # Arrange — the gov-API `main_fuel_type` enum (confirmed by the + # description-vs-code audit on `main_heating[].description`) carries + # 5 = anthracite and 33 = coal. Both COLLIDE with a same-valued Table + # 32 code of a different fuel: code 5 = bulk LPG secondary (12.19 p), + # code 33 = electricity 10-hour low rate (7.5 p). The shared price + # lookup checks the Table-32 dict first, so without normalisation an + # anthracite main billed at 12.19 p and a coal main at 7.5 p — driving + # the cohort's worst cert (2100 anthracite, -61 SAP). `_main_fuel_code` + # now canonicalises the colliding gov-API enum to its Table 32 code at + # the fuel-TYPE boundary (5 -> 15 anthracite 3.64 p; 33 -> 11 house + # coal 3.67 p) — and the canonical coal code is no longer mis-flagged + # electric. The electricity-10h TARIFF code 33 (dual-rate split) is + # untouched because it never enters as a main fuel-type. + from domain.sap10_calculator.tables.table_12a import Tariff + from domain.sap10_calculator.rdsap.cert_to_inputs import ( + _main_fuel_code, # pyright: ignore[reportPrivateUsage] + ) + anthracite_main = MainHeatingDetail( + has_fghrs=False, main_fuel_type=5, heat_emitter_type=1, + emitter_temperature=1, main_heating_control=2105, + main_heating_category=2, sap_main_heating_code=158, + ) + coal_main = MainHeatingDetail( + has_fghrs=False, main_fuel_type=33, heat_emitter_type=1, + emitter_temperature=1, main_heating_control=2105, + main_heating_category=2, sap_main_heating_code=158, + ) + + # Act + anthracite_code = _main_fuel_code(anthracite_main) + coal_code = _main_fuel_code(coal_main) + anthracite_rate = _space_heating_fuel_cost_gbp_per_kwh( + anthracite_main, Tariff.STANDARD, SAP_10_2_SPEC_PRICES + ) + coal_rate = _space_heating_fuel_cost_gbp_per_kwh( + coal_main, Tariff.STANDARD, SAP_10_2_SPEC_PRICES + ) + + # Assert — canonical codes + solid-fuel rates (not 12.19 p / 7.5 p), + # and coal is no longer classed as electric. + assert anthracite_code == 15 + assert coal_code == 11 + assert abs(anthracite_rate - 0.0364) <= 1e-6 + assert abs(coal_rate - 0.0367) <= 1e-6 + assert _is_electric_main(coal_main) is False + + def test_responsiveness_solid_fuel_sap_code_160_returns_0p50_per_table_4a() -> None: # Arrange — SAP 10.2 Table 4a (PDF p.169, "Heating systems"): # From 87485bbe3d904bcd5dbe6964b585de431aad86b9 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 8 Jun 2026 20:32:32 +0000 Subject: [PATCH 12/87] =?UTF-8?q?docs:=20session-5=20handover=20=E2=80=94?= =?UTF-8?q?=20fuel-code=20collision=20fix=20(anthracite/coal)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Records the fuel-type-boundary canonicalisation, the goldens-caught constraint (code 33 is also the electricity-10h tariff code), and the deferred dual-fuel/community/fabric follow-ups. Co-Authored-By: Claude Opus 4.8 --- docs/HANDOVER_API_PROFILING.md | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/docs/HANDOVER_API_PROFILING.md b/docs/HANDOVER_API_PROFILING.md index 8adb4f05..f062ca91 100644 --- a/docs/HANDOVER_API_PROFILING.md +++ b/docs/HANDOVER_API_PROFILING.md @@ -13,16 +13,34 @@ deproven approaches + the meter/shower data-fidelity findings), and the earlier `energy_rating_current`. Headline gauge: `PYTHONPATH=/workspaces/model python scripts/eval_api_sap_accuracy.py`. -| metric | session-3 (`a8e5563a`) | session-4 (`faf29942`) | **session-5 (`58cff932`)** | +| metric | session-3 (`a8e5563a`) | session-4 (`faf29942`) | **session-5 (`19235d11`)** | |--------|------------------|------------------|------------------| | **% \|err\| < 0.5** | 45.1% | 47.6% | **53.1%** | | % \|err\| < 1.0 | 59.4% | 62.6% | **68.0%** | | % \|err\| < 2.0 | 77.7% | 79.6% | **~81%** | -| mean \|err\| | 1.702 | 1.586 | **1.424** | +| mean \|err\| | 1.702 | 1.586 | **1.329** | | median \|err\| | — | — | **0.467** | | computed / raises | 909 / 0 | 909 / 0 | **909 / 0** | | unsupported_schema | 100 (deferred) | 100 (deferred) | 100 (deferred) | +### SESSION-5 — fuel-code collision (anthracite/coal), `19235d11` +The re-audit traced the cohort's WORST cert (2100 anthracite, −61) + the −20/−21 coal cluster to +the **fuel-code collision** (`reference_fuel_code_collision`): the shared price/CO2/PE lookups check +Table-32/12-code membership BEFORE translating the gov-API fuel enum, so API-5 (anthracite) priced at +the bulk-LPG rate (12.19 p) and API-33 (coal) at the electricity-10h rate (7.5 p). KEY constraint +(goldens caught it): code 33 is ALSO the electricity-10h TARIFF code used by the dual-rate CO2/PE +split — so the fix CANNOT live in the shared table functions (breaks golden 000565). Instead +`canonical_fuel_code` (table_32) normalises the colliding SOLID-fuel enums at the **fuel-TYPE +boundary** (`_main_fuel_code`/`_water_heating_fuel_code`): 5→15 anthracite 3.64 p, 33→11 house coal +3.67 p; also fixes `is_electric_fuel_code(33)` mis-flagging coal as electric. Scoped to {5, 33} +(unambiguous large mispricings). mean|err| 1.424→1.329 (2100 −61→−11, residual now FABRIC: 110 905 +kWh demand = a separate area over-statement); within-0.5 flat at 53.1%. **DEFERRED follow-ups:** +dual-fuel API-9 (0.45 p delta, net-neutral, shifts certs in an un-understood direction — needs its own +look); community API-20/25/31 (route through the heat-network standing/CO2 path, NOT `unit_price` — +cert 8536 fuel-31 still mis-prices at 5.5 p electricity → −17). Method: the `decompose_api_cost_error.py` +heat:high/low tail + field-by-field audit of the worst certs surfaced it; `is_electric_fuel_code` +collisions are the tell. + ### THE UNIFYING PRINCIPLE (user, load-bearing) — "unknown insulation → as-built, NOT uninsulated" An EPC insulation field that is UNDETERMINED (thickness `'ND'`/`'AB'`/absent → parsed None, or description "as built / (assumed)") must map to the **age-band default** ("as built"), which is From a7761ea83fa73d2ee48f050a57193a0249cb0bc8 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 8 Jun 2026 21:55:48 +0000 Subject: [PATCH 13/87] fix(fuel): map gov-API community fuels 30/31/32 (waste/biomass/biogas) to Table-12 community rows, gated on heat-network context MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The gov-API `main_fuel_type`/`water_heating_fuel` enum (epc_codes.csv) codes 30="waste combustion (community)", 31="biomass (community)", 32="biogas (community)" collide in VALUE with the Table-32 electricity codes 30 (standard rate), 31 (7-hour low) and 32 (7-hour high). All three sit in `_ELECTRIC_FUEL_CODES`, so `is_electric_fuel_code` flagged a community-scheme main as electric and `_is_electric_main` routed its cost through the off-peak electricity branch — BYPASSING the heat-network rate in `_heat_network_factor_fuel_code`. Cert 8536 (biomass community, SAP code 301) was billing at 5.5 p/kWh grid electricity instead of the 4.24 p/kWh heat-network rate → -17.2 SAP. Per RdSAP 10 §C / SAP 10.2 Table 12 (PDF p.191) the community waste/biomass/biogas rows are codes 42/43/44 (the same rows the backwards-compat enum codes 11/12/13 already map to). Add 30->42, 31->43, 32->44 to both API fuel-translation tables. The remap CANNOT be global (`canonical_fuel_code`): the cascade uses the bare Table-32 code 30 internally as `_STANDARD_ELECTRICITY_FUEL_CODE` (the RdSAP no-water-heating immersion default writes `water_heating_fuel=30`), so a blanket remap mis-prices genuine grid electricity as community waste (cert 2211 regressed +16 SAP in a prototype). Instead `_heat_network_community_fuel_code` translates only when `_is_heat_network_main` is true, at the `_main_fuel_code` / `_water_heating_fuel_code` fuel-TYPE boundary, where the community meaning is unambiguous. Per the strict-raise principle ([[reference-unmapped-sap-code]]), a heat-network main lodging a colliding community fuel the table doesn't cover raises `UnmappedSapCode` rather than silently falling through to the same-numbered electricity code. Eval (API SAP vs lodged): cert 8536 -17.25 -> -6.51, cert 5036 -6.29 -> +1.36; mean|err| 1.329 -> 1.312, within-1.0 67.88% -> 67.99%, within-2.0 81.74% -> 81.85%, within-0.5 held at 53.14%, 909 computed / 0 raises. No golden / calculator regressions. Co-Authored-By: Claude Opus 4.8 --- .../sap10_calculator/rdsap/cert_to_inputs.py | 63 ++++++++++++++- domain/sap10_calculator/tables/table_12.py | 2 +- domain/sap10_calculator/tables/table_32.py | 20 ++++- .../rdsap/test_cert_to_inputs.py | 77 +++++++++++++++++++ 4 files changed, 156 insertions(+), 6 deletions(-) diff --git a/domain/sap10_calculator/rdsap/cert_to_inputs.py b/domain/sap10_calculator/rdsap/cert_to_inputs.py index aef8d53f..31709822 100644 --- a/domain/sap10_calculator/rdsap/cert_to_inputs.py +++ b/domain/sap10_calculator/rdsap/cert_to_inputs.py @@ -1617,9 +1617,17 @@ def _water_heating_fuel_code(epc: EpcPropertyData) -> Optional[int]: PE/cost lookups returned defaults instead of the gas-combi values. """ if epc.sap_heating.water_heating_fuel: + fuel = epc.sap_heating.water_heating_fuel + # When DHW is on a heat network, the colliding community fuels + # 30/31/32 take their Table 12 community row rather than the + # same-numbered electricity code — see + # `_heat_network_community_fuel_code`. + community = _heat_network_community_fuel_code(fuel, _water_heating_main(epc)) + if community is not None: + return community # Normalise colliding gov-API solid-fuel enum codes (see # `_main_fuel_code`) before the shared price/CO2/PE lookups. - return canonical_fuel_code(epc.sap_heating.water_heating_fuel) + return canonical_fuel_code(fuel) return _main_fuel_code(_water_heating_main(epc)) @@ -1929,6 +1937,53 @@ _RESPONSIVENESS_BY_EMITTER_CODE: Final[dict[int, float]] = { } +# Gov-API community fuel enum codes (waste 30 / biomass 31 / biogas 32) +# whose VALUE collides with a Table-32 electricity tariff code of the same +# number (30 standard / 31 7-hour-low / 32 7-hour-high). Per +# `epc_codes.csv` these are unambiguously "(community)" fuels, but the +# bare Table-32 codes 30/31/32 are ALSO used internally as grid +# electricity (e.g. `_STANDARD_ELECTRICITY_FUEL_CODE = 30` written by the +# no-water-heating immersion default), so the community meaning is only +# authoritative when the main is a heat network — see +# `_heat_network_community_fuel_code`. +_API_COMMUNITY_COLLISION_FUELS: Final[frozenset[int]] = frozenset({30, 31, 32}) + + +def _heat_network_community_fuel_code( + fuel: int, main: Optional[MainHeatingDetail] +) -> Optional[int]: + """Translate a gov-API community fuel enum to its SAP Table 12 + community fuel code WHEN the main is a heat network; else return None + so the caller keeps `fuel` unchanged. + + Community fuels 30 (waste) / 31 (biomass) / 32 (biogas) collide in + value with the Table-32 electricity codes 30/31/32. Without this + translation `is_electric_fuel_code` flags a community-scheme main as + electric and `_is_electric_main` routes its cost through the off-peak + electricity branch — bypassing the heat-network rate + (`_heat_network_factor_fuel_code`) entirely. Per RdSAP 10 §C / SAP + 10.2 Table 12 the community waste/biomass/biogas rows are codes + 42/43/44 (the same rows the backwards-compat enum codes 11/12/13 map + to). Cert 8536 (biomass community, SAP code 301) closed -17.2 → -6.5. + + Gating on `_is_heat_network_main` keeps the bare Table-32 code 30 the + cascade uses internally as grid electricity untouched on + non-community certs (e.g. cert 2211 whose whc=999 default writes + `water_heating_fuel=30`). + + Raises `UnmappedSapCode` when a heat-network main lodges a colliding + community fuel the translation table doesn't cover — surfacing the + gap loudly instead of silently mis-pricing it as grid electricity, + per the strict-raise principle ([[reference-unmapped-sap-code]]). + """ + if fuel not in _API_COMMUNITY_COLLISION_FUELS or not _is_heat_network_main(main): + return None + translated = API_FUEL_TO_TABLE_12.get(fuel) + if translated is None: + raise UnmappedSapCode("heat_network_community_fuel", fuel) + return translated + + def _main_fuel_code(main: Optional[MainHeatingDetail]) -> Optional[int]: """Resolve `MainHeatingDetail.main_fuel_type` to a SAP fuel code. @@ -1946,6 +2001,12 @@ def _main_fuel_code(main: Optional[MainHeatingDetail]) -> Optional[int]: return None fuel = main.main_fuel_type if isinstance(fuel, int): + # Heat-network community fuels 30/31/32 collide with electricity + # Table-32 codes — translate to the community row before anything + # else so the main isn't mis-classified as electric. + community = _heat_network_community_fuel_code(fuel, main) + if community is not None: + return community # Normalise the colliding gov-API solid-fuel enum codes (5 # anthracite / 9 dual fuel / 33 coal) to their canonical Table # 32/12 codes here — at the fuel-TYPE boundary — so the shared diff --git a/domain/sap10_calculator/tables/table_12.py b/domain/sap10_calculator/tables/table_12.py index dc64e57f..7ea27585 100644 --- a/domain/sap10_calculator/tables/table_12.py +++ b/domain/sap10_calculator/tables/table_12.py @@ -214,7 +214,7 @@ API_FUEL_TO_TABLE_12: Final[dict[int, int]] = { 0: 30, 1: 1, 2: 2, 3: 3, 4: 4, 5: 15, 6: 20, 7: 23, 8: 21, 9: 10, 10: 30, 11: 42, 12: 43, 13: 44, 14: 11, 15: 12, 16: 22, 17: 9, 18: 75, 19: 76, 20: 51, 21: 52, 22: 53, 23: 55, 24: 54, 25: 41, - 26: 1, 27: 2, 28: 4, 29: 30, 33: 11, + 26: 1, 27: 2, 28: 4, 29: 30, 30: 42, 31: 43, 32: 44, 33: 11, } diff --git a/domain/sap10_calculator/tables/table_32.py b/domain/sap10_calculator/tables/table_32.py index 33accf46..8377fe86 100644 --- a/domain/sap10_calculator/tables/table_32.py +++ b/domain/sap10_calculator/tables/table_32.py @@ -105,7 +105,7 @@ API_FUEL_TO_TABLE_32: Final[dict[int, int]] = { 0: 30, 1: 1, 2: 2, 3: 3, 4: 4, 5: 15, 6: 20, 7: 23, 8: 21, 9: 10, 10: 30, 11: 42, 12: 43, 13: 44, 14: 11, 15: 12, 16: 22, 17: 9, 18: 75, 19: 76, 20: 51, 21: 52, 22: 53, 23: 55, 24: 54, 25: 41, - 26: 1, 27: 2, 28: 4, 29: 30, 33: 11, + 26: 1, 27: 2, 28: 4, 29: 30, 30: 42, 31: 43, 32: 44, 33: 11, } # Gov-API `main_fuel_type` enum codes whose value COLLIDES with a @@ -125,9 +125,21 @@ API_FUEL_TO_TABLE_32: Final[dict[int, int]] = { # collision (Table-32 9 = LPG SC11F 3.48 p vs dual fuel 3.99 p) but the # 0.45 p delta nets neutral-to-negative on the (outlier-dominated) # dual-fuel certs and shifts them in a direction not yet understood — -# investigate separately. Community heat-network fuels (20/25/31) are -# also out of scope — their standing-charge / CO2 / PE routing is handled -# by the dedicated heat-network path. +# investigate separately. +# +# COMMUNITY FUELS (handled elsewhere, NOT here): API 30 (waste +# combustion), 31 (biomass) and 32 (biogas) — all "(community)" in the +# enum — collide in VALUE with the Table-32 electricity codes 30 (standard +# rate), 31 (7-hour low) and 32 (7-hour high). They must NOT be +# canonicalised globally: the cascade uses the bare Table-32 code 30 +# internally as `_STANDARD_ELECTRICITY_FUEL_CODE` (e.g. the RdSAP +# no-water-heating immersion default writes `water_heating_fuel=30`), so a +# blanket remap would mis-price genuine grid electricity as community +# waste. The translation is therefore done at the fuel-TYPE boundary +# GATED on heat-network context (`_heat_network_community_fuel_code` in +# cert_to_inputs), where the community meaning is unambiguous. Community +# fuels 20/25 do not collide with an electricity code, so they resolve +# correctly through the heat-network path without any special handling. _GOV_API_COLLISION_FUELS: Final[frozenset[int]] = frozenset({5, 33}) diff --git a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py index d48f947c..3c961919 100644 --- a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py +++ b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py @@ -50,10 +50,12 @@ from domain.sap10_calculator.rdsap.cert_to_inputs import ( SAP_10_2_SPEC_PRICES, _has_suspended_timber_floor_per_spec, # pyright: ignore[reportPrivateUsage] _heat_network_code_302_effective_factor, # pyright: ignore[reportPrivateUsage] + _heat_network_community_fuel_code, # pyright: ignore[reportPrivateUsage] _heat_network_distribution_electricity, # pyright: ignore[reportPrivateUsage] _heat_network_dlf, # pyright: ignore[reportPrivateUsage] _heat_network_factor_fuel_code, # pyright: ignore[reportPrivateUsage] _heat_network_standing_charge_gbp, # pyright: ignore[reportPrivateUsage] + _main_fuel_code, # pyright: ignore[reportPrivateUsage] _is_electric_main, # pyright: ignore[reportPrivateUsage] _is_heat_network_electric_main, # pyright: ignore[reportPrivateUsage] _is_electric_water, # pyright: ignore[reportPrivateUsage] @@ -1543,6 +1545,81 @@ def test_solid_fuel_main_prices_at_table_32_solid_rate_not_colliding_code() -> N assert _is_electric_main(coal_main) is False +def test_heat_network_biomass_community_fuel_resolves_to_table12_community_code() -> None: + # Arrange — a heat-network main (SAP Table 4a code 301 = community + # heating; main_heating_category=6) lodging gov-API `main_fuel_type` + # 31 = "biomass (community)" per epc_codes.csv. The enum value 31 + # COLLIDES with the Table-32 7-hour-low-rate electricity code 31, so + # without the gated translation `is_electric_fuel_code(31)` flags this + # community-scheme main as electric and `_is_electric_main` routes its + # cost through the off-peak electricity branch — bypassing the + # heat-network rate (cert 8536 biomass-community, -17.2 SAP). Per + # RdSAP 10 §C / SAP 10.2 Table 12 the community biomass row is code 43 + # (the same row the backwards-compat enum 12 maps to). + biomass_community_main = MainHeatingDetail( + has_fghrs=False, main_fuel_type=31, heat_emitter_type=0, + emitter_temperature="NA", main_heating_control=2306, + main_heating_category=6, sap_main_heating_code=301, + ) + + # Act + fuel_code = _main_fuel_code(biomass_community_main) + factor_code = _heat_network_factor_fuel_code(biomass_community_main) + + # Assert — community biomass Table-12 row, NOT electricity, and the + # main is no longer mis-classified as electric. + assert fuel_code == 43 + assert factor_code == 43 + assert _is_electric_main(biomass_community_main) is False + + +def test_non_heat_network_electricity_code_30_is_not_remapped_to_community() -> None: + # Arrange — the colliding community translation must be GATED on + # heat-network context: the cascade writes the bare Table-32 code 30 + # (`_STANDARD_ELECTRICITY_FUEL_CODE`) as genuine grid electricity on + # non-community certs (e.g. the RdSAP no-water-heating immersion + # default — cert 2211). A non-heat-network main carrying code 30 must + # stay electric, NOT be remapped to community waste (Table-12 42). + electric_main = MainHeatingDetail( + has_fghrs=False, main_fuel_type=30, heat_emitter_type=1, + emitter_temperature=1, main_heating_control=2106, + main_heating_category=2, sap_main_heating_code=191, + ) + + # Act + fuel_code = _main_fuel_code(electric_main) + + # Assert — unchanged grid-electricity code, still electric. + assert fuel_code == 30 + assert _is_electric_main(electric_main) is True + + +def test_heat_network_unmapped_community_collision_fuel_raises() -> None: + # Arrange — a heat-network main lodging a colliding community fuel the + # translation table does NOT cover must raise rather than silently + # mis-price it as the same-numbered grid-electricity code (the + # strict-raise principle — a community fuel that "falls through" is a + # mapping gap to surface, not a default to swallow). Simulate the gap + # by removing biomass (31) from the translation table. + from unittest.mock import patch + + import domain.sap10_calculator.rdsap.cert_to_inputs as cti + + main = MainHeatingDetail( + has_fghrs=False, main_fuel_type=31, heat_emitter_type=0, + emitter_temperature="NA", main_heating_control=2306, + main_heating_category=6, sap_main_heating_code=301, + ) + patched = dict(cti.API_FUEL_TO_TABLE_12) + del patched[31] + + # Act / Assert — the gap surfaces loudly instead of resolving to the + # colliding electricity code 31. + with patch.object(cti, "API_FUEL_TO_TABLE_12", patched): + with pytest.raises(UnmappedSapCode): + _heat_network_community_fuel_code(31, main) + + def test_responsiveness_solid_fuel_sap_code_160_returns_0p50_per_table_4a() -> None: # Arrange — SAP 10.2 Table 4a (PDF p.169, "Heating systems"): # From e1adc8d3d5a149bf6f116aed6dfd39a654758b06 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 8 Jun 2026 21:56:56 +0000 Subject: [PATCH 14/87] =?UTF-8?q?docs:=20session-6=20handover=20=E2=80=94?= =?UTF-8?q?=20community=20fuel=20collision=20(waste/biomass/biogas)=20fixe?= =?UTF-8?q?d=20gated=20on=20heat-network?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 4.8 --- docs/HANDOVER_API_PROFILING.md | 50 ++++++++++++++++++++++++++++------ 1 file changed, 41 insertions(+), 9 deletions(-) diff --git a/docs/HANDOVER_API_PROFILING.md b/docs/HANDOVER_API_PROFILING.md index f062ca91..7b76b07b 100644 --- a/docs/HANDOVER_API_PROFILING.md +++ b/docs/HANDOVER_API_PROFILING.md @@ -13,15 +13,47 @@ deproven approaches + the meter/shower data-fidelity findings), and the earlier `energy_rating_current`. Headline gauge: `PYTHONPATH=/workspaces/model python scripts/eval_api_sap_accuracy.py`. -| metric | session-3 (`a8e5563a`) | session-4 (`faf29942`) | **session-5 (`19235d11`)** | -|--------|------------------|------------------|------------------| -| **% \|err\| < 0.5** | 45.1% | 47.6% | **53.1%** | -| % \|err\| < 1.0 | 59.4% | 62.6% | **68.0%** | -| % \|err\| < 2.0 | 77.7% | 79.6% | **~81%** | -| mean \|err\| | 1.702 | 1.586 | **1.329** | -| median \|err\| | — | — | **0.467** | -| computed / raises | 909 / 0 | 909 / 0 | **909 / 0** | -| unsupported_schema | 100 (deferred) | 100 (deferred) | 100 (deferred) | +| metric | session-3 (`a8e5563a`) | session-4 (`faf29942`) | session-5 (`19235d11`) | **session-6 (`a7761ea8`)** | +|--------|------------------|------------------|------------------|------------------| +| **% \|err\| < 0.5** | 45.1% | 47.6% | 53.1% | **53.14%** | +| % \|err\| < 1.0 | 59.4% | 62.6% | 68.0% | **67.99%** | +| % \|err\| < 2.0 | 77.7% | 79.6% | ~81% | **81.85%** | +| mean \|err\| | 1.702 | 1.586 | 1.329 | **1.312** | +| median \|err\| | — | — | 0.467 | **0.467** | +| computed / raises | 909 / 0 | 909 / 0 | 909 / 0 | **909 / 0** | +| unsupported_schema | 100 (deferred) | 100 (deferred) | 100 (deferred) | 100 (deferred) | + +### SESSION-6 — community fuels 30/31/32 collide with electricity codes, `a7761ea8` +Picked up the deferred "fuel-collision part 2". The profiler's strongly-biased +`main_control=2306` bucket (n=11, signed −3.75, nearly uniform) was a PROXY: every +cert in it is `sap_main_heating_code=301` (community heating). ROOT: gov-API +`main_fuel_type` codes **30=waste-combustion / 31=biomass / 32=biogas** (all +"(community)" in `epc_codes.csv`) collide in VALUE with the Table-32 electricity +codes 30 (standard) / 31 (7h-low) / 32 (7h-high). All three sit in +`_ELECTRIC_FUEL_CODES`, so `is_electric_fuel_code` flagged a community-scheme main as +electric and `_is_electric_main` routed its cost through the off-peak electricity +branch — BYPASSING `_heat_network_factor_fuel_code`. Cert 8536 (biomass community) +billed 5.5 p grid electricity instead of the 4.24 p heat-network rate → −17.2 SAP. +Per SAP 10.2 Table 12 the community waste/biomass/biogas rows are 42/43/44 (the same +rows the backwards-compat enum codes 11/12/13 already map to). Added 30→42, 31→43, +32→44 to `API_FUEL_TO_TABLE_12` + `API_FUEL_TO_TABLE_32`. +**CRITICAL — the remap is GATED, NOT global.** A prototype putting 30/31/32 in +`canonical_fuel_code` regressed certs 2211 (+16 SAP) and 3420 (+7): the cascade uses +the bare Table-32 code 30 internally as `_STANDARD_ELECTRICITY_FUEL_CODE` (the RdSAP +no-water-heating whc=999 immersion default writes `water_heating_fuel=30`), so a +blanket remap mis-prices genuine grid electricity as community waste. New +`_heat_network_community_fuel_code(fuel, main)` translates only when +`_is_heat_network_main(main)` is true, wired into `_main_fuel_code` AND +`_water_heating_fuel_code` (water gated via `_water_heating_main(epc)`). +**STRICT-RAISE (user-requested):** a heat-network main lodging a colliding community +fuel the table doesn't cover raises `UnmappedSapCode` rather than silently falling +through to the same-numbered electricity code (currently can't fire — all of +{30,31,32} are mapped — but guards future community fuels 56/57/58/99). Cert 8536 +−17.25 → −6.51 (residual now flat fabric), 5036 −6.29 → +1.36; mean|err| +1.329→1.312, within-1.0/2.0 up, within-0.5 held. 3 AAA tests, regression green +(only pre-existing 2 stone-wall U + a flaky historic-epc ordering test), pyright +net-zero. **STILL DEFERRED: dual-fuel API-9; the per-cert FABRIC tail (8536's −6.5 +is now flat fabric, 2100 demand 110905); the 100 unsupported-schema certs.** ### SESSION-5 — fuel-code collision (anthracite/coal), `19235d11` The re-audit traced the cohort's WORST cert (2100 anthracite, −61) + the −20/−21 coal cluster to From 3e05c95e65bcf66668f8931e63e9101db350b70b Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 8 Jun 2026 22:49:42 +0000 Subject: [PATCH 15/87] fix(wall-U): apply RdSAP Table 4 "Sheltered" R=0.5 to alternative walls MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Audit of the API-SAP error (53% within 0.5) localised the systematic under-rate to ELECTRIC FLATS (houses sit at 60-66% within 0.5; electric flats 13-19%). Decomposing the flat error showed it tracks space-heating demand per m² — the worst certs reach 130-289 kWh/m² (accurate certs sit at 14-110), i.e. a grossly over-stated fabric heat loss, amplified ~4x by the electricity unit price and the steep low-band SAP log curve. Root cause: the gov-EPC API lodges `sheltered_wall="Y"` on alternative wall sub-areas (a sub-area adjacent to an unheated buffer — stair core, adjoining structure), but the field was dropped by the schema + domain dataclasses and the calculator billed the alt sub-area at its full exposed U. RdSAP 10 Table 4 (PDF p.22) "Sheltered": such a wall carries an added external surface resistance R=0.5 m²K/W → U_sheltered = 1/(1/U + 0.5) — the SAME adjustment the main wall already applies for `gable_wall_type=2` (`gable_wall_sheltered`, `_SHELTERED_GABLE_ADDED_RESISTANCE_M2K_W`). Cert 0340-2976 (band-A flat, 42 m² sheltered timber-frame alt) over-stated its wall channel by ~58 W/K → walls 128 -> 70 W/K. Threads the field end-to-end: schema dataclasses (21.0.0/21.0.1) + domain `SapAlternativeWall.is_sheltered` (default False — the Summary/ Elmhurst path leaves it False, sheltering rides through its lodged U-value there, so goldens are untouched) + `from_api_response` mapping `"Y"->True` + `_alt_wall_w_per_k` applying the 0.5 resistance on the cascade path (lodged-U and basement alt-walls return before it). 140 certs (15% of the corpus) carry a sheltered alt-wall; they under- rated at median -0.82 / mean signed -1.33 / 23% within 0.5. Eval: 102 improved, 38 regressed (offsetting-error cases — fix is spec-uniform per [[feedback_software_no_special_handling]]); within-0.5 53.14% -> 54.24%, within-1.0 67.99% -> 69.64%, within-2.0 81.85% -> 83.50%, mean|err| 1.312 -> 1.248, 909 computed / 0 raises. Goldens (6035, 000565) and full calc/epc/parser regression green; pyright net-zero. Co-Authored-By: Claude Opus 4.8 --- datatypes/epc/domain/epc_property_data.py | 8 +++++ datatypes/epc/domain/mapper.py | 4 +++ datatypes/epc/schema/rdsap_schema_21_0_0.py | 1 + datatypes/epc/schema/rdsap_schema_21_0_1.py | 1 + .../worksheet/heat_transmission.py | 12 +++++++ .../worksheet/test_heat_transmission.py | 35 +++++++++++++++++++ 6 files changed, 61 insertions(+) diff --git a/datatypes/epc/domain/epc_property_data.py b/datatypes/epc/domain/epc_property_data.py index 4b45e598..6649ade1 100644 --- a/datatypes/epc/domain/epc_property_data.py +++ b/datatypes/epc/domain/epc_property_data.py @@ -440,6 +440,14 @@ class SapAlternativeWall: # Mirrors `SapBuildingPart.wall_thickness_mm` per the # [[feedback-no-misleading-insulation-type]] convention. wall_thickness_mm: Optional[int] = None + # RdSAP 10 Table 4 (p.22) "Sheltered" wall: a sub-area adjacent to + # an unheated buffer space (stair core, adjoining structure) carries + # an added external surface resistance of R=0.5 m²K/W, so its U is + # reduced to U_sheltered = 1/(1/U + 0.5) — the same adjustment the + # main wall applies for `gable_wall_type=2`. The gov-EPC API lodges + # this per alt-wall as `sheltered_wall="Y"`; the Summary/Elmhurst path + # leaves it False (sheltering rides through the lodged U-value there). + is_sheltered: bool = False # Explicit basement determination. RdSAP10 `wall_construction == 6` is # canonically SYSTEM-BUILT (`WALL_SYSTEM_BUILT`) — the basement # heuristic hijacked it because Elmhurst lodges both "SY System build" diff --git a/datatypes/epc/domain/mapper.py b/datatypes/epc/domain/mapper.py index 04f39ba6..978711c5 100644 --- a/datatypes/epc/domain/mapper.py +++ b/datatypes/epc/domain/mapper.py @@ -1469,6 +1469,7 @@ class EpcPropertyDataMapper: wall_insulation_type=bp.sap_alternative_wall_1.wall_insulation_type, wall_thickness_measured=bp.sap_alternative_wall_1.wall_thickness_measured, wall_insulation_thickness=bp.sap_alternative_wall_1.wall_insulation_thickness, + is_sheltered=bp.sap_alternative_wall_1.sheltered_wall == "Y", ) if bp.sap_alternative_wall_1 else None @@ -1481,6 +1482,7 @@ class EpcPropertyDataMapper: wall_insulation_type=bp.sap_alternative_wall_2.wall_insulation_type, wall_thickness_measured=bp.sap_alternative_wall_2.wall_thickness_measured, wall_insulation_thickness=bp.sap_alternative_wall_2.wall_insulation_thickness, + is_sheltered=bp.sap_alternative_wall_2.sheltered_wall == "Y", ) if bp.sap_alternative_wall_2 else None @@ -1745,6 +1747,7 @@ class EpcPropertyDataMapper: wall_insulation_type=bp.sap_alternative_wall_1.wall_insulation_type, wall_thickness_measured=bp.sap_alternative_wall_1.wall_thickness_measured, wall_insulation_thickness=bp.sap_alternative_wall_1.wall_insulation_thickness, + is_sheltered=bp.sap_alternative_wall_1.sheltered_wall == "Y", ) if bp.sap_alternative_wall_1 else None @@ -1757,6 +1760,7 @@ class EpcPropertyDataMapper: wall_insulation_type=bp.sap_alternative_wall_2.wall_insulation_type, wall_thickness_measured=bp.sap_alternative_wall_2.wall_thickness_measured, wall_insulation_thickness=bp.sap_alternative_wall_2.wall_insulation_thickness, + is_sheltered=bp.sap_alternative_wall_2.sheltered_wall == "Y", ) if bp.sap_alternative_wall_2 else None diff --git a/datatypes/epc/schema/rdsap_schema_21_0_0.py b/datatypes/epc/schema/rdsap_schema_21_0_0.py index da2125be..8fceb878 100644 --- a/datatypes/epc/schema/rdsap_schema_21_0_0.py +++ b/datatypes/epc/schema/rdsap_schema_21_0_0.py @@ -220,6 +220,7 @@ class SapAlternativeWall: wall_insulation_type: int wall_thickness_measured: str wall_insulation_thickness: Optional[str] = None + sheltered_wall: Optional[str] = None @dataclass diff --git a/datatypes/epc/schema/rdsap_schema_21_0_1.py b/datatypes/epc/schema/rdsap_schema_21_0_1.py index c5f456de..c12bf31c 100644 --- a/datatypes/epc/schema/rdsap_schema_21_0_1.py +++ b/datatypes/epc/schema/rdsap_schema_21_0_1.py @@ -258,6 +258,7 @@ class SapAlternativeWall: wall_insulation_type: int wall_thickness_measured: str wall_insulation_thickness: Optional[str] = None + sheltered_wall: Optional[str] = None @dataclass diff --git a/domain/sap10_calculator/worksheet/heat_transmission.py b/domain/sap10_calculator/worksheet/heat_transmission.py index 3e4c9a4e..620deded 100644 --- a/domain/sap10_calculator/worksheet/heat_transmission.py +++ b/domain/sap10_calculator/worksheet/heat_transmission.py @@ -1339,4 +1339,16 @@ def _alt_wall_w_per_k( # dry-lined) → U=2.34 via §5.6 + §5.8 chain. wall_thickness_mm=alt_wall.wall_thickness_mm, ) + if alt_wall.is_sheltered: + # RdSAP 10 Table 4 (p.22) "Sheltered" wall: an alt sub-area + # adjacent to an unheated buffer (stair core, adjoining + # structure) carries an added external surface resistance of + # R=0.5 m²K/W → U_sheltered = 1/(1/U + 0.5). Mirrors the main + # wall's `gable_wall_sheltered` branch + # (`_SHELTERED_GABLE_ADDED_RESISTANCE_M2K_W`). The gov-EPC API + # lodges this per alt-wall as `sheltered_wall="Y"`; without it a + # sheltered timber/cavity alt sub-area billed its full exposed U + # (cert 0340-2976: a 42 m² sheltered timber-frame alt at U=2.5 + # over-stated the wall channel by ~58 W/K → -5 SAP under-rate). + alt_u = 1.0 / (1.0 / alt_u + _SHELTERED_GABLE_ADDED_RESISTANCE_M2K_W) return alt_u * net_alt_area diff --git a/tests/domain/sap10_calculator/worksheet/test_heat_transmission.py b/tests/domain/sap10_calculator/worksheet/test_heat_transmission.py index 92967423..6dfbff44 100644 --- a/tests/domain/sap10_calculator/worksheet/test_heat_transmission.py +++ b/tests/domain/sap10_calculator/worksheet/test_heat_transmission.py @@ -36,6 +36,7 @@ from domain.sap10_calculator.worksheet.heat_transmission import ( heat_transmission_from_cert, ) from domain.sap10_calculator.worksheet.heat_transmission import ( + _alt_wall_w_per_k, # pyright: ignore[reportPrivateUsage] _joined_main_roof_descriptions, # pyright: ignore[reportPrivateUsage] _part_geometry, # pyright: ignore[reportPrivateUsage] _round_half_up, # pyright: ignore[reportPrivateUsage] @@ -1193,6 +1194,40 @@ def test_alternative_wall_uses_own_construction_and_deducts_from_main_wall_area( ) +def test_sheltered_alternative_wall_applies_table4_0p5_resistance() -> None: + # Arrange — RdSAP 10 Table 4 (PDF p.22) "Sheltered" wall: a sub-area + # adjacent to an unheated buffer carries an added external surface + # resistance R=0.5 m²K/W, so its U reduces to 1/(1/U + 0.5). The gov + # EPC API lodges this per alt-wall as `sheltered_wall="Y"` (mapped to + # `is_sheltered`). Two identical timber-frame as-built alt sub-areas + # (cavity-band-A → uninsulated cascade U), one exposed, one sheltered. + from dataclasses import replace + + from domain.sap10_ml.rdsap_uvalues import Country + + area = 42.0 + exposed = SapAlternativeWall( + wall_area=area, wall_dry_lined="N", wall_construction=5, + wall_insulation_type=4, wall_thickness_measured="Y", + wall_insulation_thickness="NI", is_sheltered=False, + ) + sheltered = replace(exposed, is_sheltered=True) + + # Act + exposed_wpk = _alt_wall_w_per_k( + alt_wall=exposed, country=Country.ENG, age_band="A", wall_description=None, + ) + sheltered_wpk = _alt_wall_w_per_k( + alt_wall=sheltered, country=Country.ENG, age_band="A", wall_description=None, + ) + + # Assert — the sheltered U is the exposed U with R=0.5 added. + exposed_u = exposed_wpk / area + expected_sheltered_u = 1.0 / (1.0 / exposed_u + 0.5) + assert abs(sheltered_wpk - expected_sheltered_u * area) <= 1e-9 + assert sheltered_wpk < exposed_wpk + + def test_window_uses_effective_u_value_with_curtain_resistance_per_sap10_2_section_3_2() -> None: """SAP10.2 §3.2: the window U-value used for heat-transmission is the effective form `U_eff = 1/(1/U_raw + 0.04)` — the 0.04 m²K/W is the From 943f83ed0173ba74644354f84b3a2dde0f8ca39f Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 8 Jun 2026 22:50:55 +0000 Subject: [PATCH 16/87] =?UTF-8?q?docs:=20session-7=20handover=20=E2=80=94?= =?UTF-8?q?=20sheltered=20alternative=20walls=20(RdSAP=20Table=204=20R=3D0?= =?UTF-8?q?.5)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 4.8 --- docs/HANDOVER_API_PROFILING.md | 52 ++++++++++++++++++++++++++++++---- 1 file changed, 46 insertions(+), 6 deletions(-) diff --git a/docs/HANDOVER_API_PROFILING.md b/docs/HANDOVER_API_PROFILING.md index 7b76b07b..bb842098 100644 --- a/docs/HANDOVER_API_PROFILING.md +++ b/docs/HANDOVER_API_PROFILING.md @@ -13,16 +13,56 @@ deproven approaches + the meter/shower data-fidelity findings), and the earlier `energy_rating_current`. Headline gauge: `PYTHONPATH=/workspaces/model python scripts/eval_api_sap_accuracy.py`. -| metric | session-3 (`a8e5563a`) | session-4 (`faf29942`) | session-5 (`19235d11`) | **session-6 (`a7761ea8`)** | +| metric | session-4 (`faf29942`) | session-5 (`19235d11`) | session-6 (`a7761ea8`) | **session-7 (`3e05c95e`)** | |--------|------------------|------------------|------------------|------------------| -| **% \|err\| < 0.5** | 45.1% | 47.6% | 53.1% | **53.14%** | -| % \|err\| < 1.0 | 59.4% | 62.6% | 68.0% | **67.99%** | -| % \|err\| < 2.0 | 77.7% | 79.6% | ~81% | **81.85%** | -| mean \|err\| | 1.702 | 1.586 | 1.329 | **1.312** | -| median \|err\| | — | — | 0.467 | **0.467** | +| **% \|err\| < 0.5** | 47.6% | 53.1% | 53.14% | **54.24%** | +| % \|err\| < 1.0 | 62.6% | 68.0% | 67.99% | **69.64%** | +| % \|err\| < 2.0 | 79.6% | ~81% | 81.85% | **83.50%** | +| mean \|err\| | 1.586 | 1.329 | 1.312 | **1.248** | +| median \|err\| | — | 0.467 | 0.467 | **0.457** | | computed / raises | 909 / 0 | 909 / 0 | 909 / 0 | **909 / 0** | | unsupported_schema | 100 (deferred) | 100 (deferred) | 100 (deferred) | 100 (deferred) | +### SESSION-7 — sheltered alternative walls (RdSAP Table 4 R=0.5), `3e05c95e` +The headline-moving audit. User: 53% is poor enough to indicate a MAJOR error +— audit again. The decisive diagnostic CHAIN (reusable): +1. **Error by `dwelling_type`** → flats are the drag (houses 60-66% within 0.5, + flats 28-47%; top-floor flat −1.19, mid-floor 28% within 0.5). +2. **Split flats by `mains_gas`** → ELECTRIC flats are the killer (gas flats + ~45%, electric flats 13-19%; top-floor electric mean|err| 3.62). +3. **Invert the SAP equation** (ECF = 0.42·cost/(TFA+45) → `sap_rating`) to get + `our_cost / lodged_cost` → electric flats over-cost ~3% median (houses 1.00), + amplified by the 4× electric price + steep low-band log curve (+3% ≈ +1.5 SAP + at band 40). +4. **Under-rate tracks space-heating kWh/m² precisely** — accurate certs 14-110, + under-rating certs 130-289 (cert 2021: 11 275 kWh for 39 m²) → over-stated + FABRIC, not tariff (storage flats on the correct 5.5p rate under-rate just as + much as room-heater flats at 13p). +5. **Field-by-field on the worst** → cert 0340-2976 (band-A flat) computed wall + 128 W/K though its main wall is a FILLED cavity (U 0.7); the excess was a + SECOND `u_wall` call — a `sap_alternative_wall_1` timber-frame sub-area at + U=2.5 lodging **`sheltered_wall="Y"`**. + +ROOT: the gov-EPC API lodges `sheltered_wall="Y"` per alt-wall, but it was +DROPPED by the schema + domain dataclasses, so `_alt_wall_w_per_k` billed the +sub-area at its full exposed U. RdSAP 10 Table 4 (PDF p.22) "Sheltered": added +external resistance R=0.5 m²K/W → U_sheltered = 1/(1/U + 0.5) — the SAME +adjustment the MAIN wall already applies for `gable_wall_type=2` +(`gable_wall_sheltered`, `_SHELTERED_GABLE_ADDED_RESISTANCE_M2K_W`). Threaded +end-to-end: schema (21.0.0/21.0.1) + domain `SapAlternativeWall.is_sheltered` +(default False → Summary/Elmhurst path unchanged, goldens untouched) + +`from_api_response` `"Y"→True` + `_alt_wall_w_per_k` applies the 0.5 resistance +(lodged-U + basement alts return before it). 140 certs (15% of corpus) carry a +sheltered alt-wall; they under-rated at median −0.82 / signed −1.33 / 23% within +0.5. Eval: 102 improved / 38 regressed (offsetting-error cases — applied +spec-uniformly per the determinism principle); +10 net within-0.5. Goldens + +full regression green, pyright net-zero. **OPEN audit leftovers: electric-flat +tail persists (2021 −3.9 genuine uninsulated solid brick = data-fidelity; +top-floor "Flat, limited insulation" roofs); detached houses 45% (balanced +bidirectional scatter); the 38 sheltering regressions = offsetting errors +elsewhere; main-wall (non-gable) sheltering not audited; multi-element wall +description joined-string-to-all-BPs (113 certs, mild).** + ### SESSION-6 — community fuels 30/31/32 collide with electricity codes, `a7761ea8` Picked up the deferred "fuel-collision part 2". The profiler's strongly-biased `main_control=2306` bucket (n=11, signed −3.75, nearly uniform) was a PROXY: every From 71b378b9e5c5e0012bf03681443fcf4510bcef32 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 8 Jun 2026 23:02:04 +0000 Subject: [PATCH 17/87] =?UTF-8?q?fix(ventilation):=20map=20API=20mechanica?= =?UTF-8?q?l=5Fventilation=20enum=20to=20=C2=A72=20MV-kind=20dispatch?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The profiler flagged `mechanical_ventilation=2` as a clean systematic over-rate: 20 certs, signed +1.90 SAP, only 5% within 0.5 (every one positive). Root cause: the API path (`from_api_response`) dropped the doc-level `mechanical_ventilation` field, so `sap_ventilation. mechanical_ventilation_kind` was always None and the §2 cascade defaulted to NATURAL — under-stating the ventilation air-change rate (and hence heat loss) for every mechanical system. (Only the Elmhurst/ Summary path mapped it, via `_ELMHURST_MV_TYPE_TO_KIND`.) RdSAP-Schema-21 `mechanical_ventilation` enum (epc_codes.csv) → MechanicalVentilationKind picking the SAP 10.2 §2 (24a..d) effective-ach formula: 0 natural -> NATURAL (24d) 1 MV (no heat recovery) -> MV (24b) 2 mechanical extract, dc (MEV) -> EXTRACT_OR_PIV_OUTSIDE (24c) 3 mechanical extract, c (MEV) -> EXTRACT_OR_PIV_OUTSIDE (24c) 5 positive input from loft -> NATURAL (loft-sourced PIV adds no system air change per RdSAP 10 §2.6) 6 positive input from outside -> EXTRACT_OR_PIV_OUTSIDE (24c) Code 4 (MVHR, 24a) is DEFERRED — its formula needs the lodged heat-recovery efficiency (PCDB Table 326) the API→cascade path doesn't yet plumb; mapping it to MVHR with a null efficiency would mis-model it as MV, so it stays NATURAL (3 scattered certs, accurate at the median). Unmapped integers raise `UnmappedApiCode` (mirror of `_api_sheltered_ sides` / `_api_type_1_gable_kind`). Eval: the extract cohort (mech_vent 2/3/6) moved +1.90 -> +0.9 median (within-0.5 5% -> 35%); 20 improved / 3 regressed (offsetting). Headline within-0.5 54.24% -> 55.01%, within-1.0 69.64% -> 70.08%, mean|err| 1.248 -> 1.233, 909 computed / 0 raises. The +0.9 residual on MEV is the fan electricity (§2.6.4 SFP, PCDB Table 322) — a separate follow-up. 2 AAA tests; goldens + full calc/epc/parser regression green; pyright net-zero. Co-Authored-By: Claude Opus 4.8 --- datatypes/epc/domain/mapper.py | 57 +++++++++++++++++++ .../rdsap/test_cert_to_inputs.py | 39 +++++++++++++ 2 files changed, 96 insertions(+) diff --git a/datatypes/epc/domain/mapper.py b/datatypes/epc/domain/mapper.py index 978711c5..b9de5feb 100644 --- a/datatypes/epc/domain/mapper.py +++ b/datatypes/epc/domain/mapper.py @@ -1877,6 +1877,14 @@ class EpcPropertyDataMapper: # default 2 → over-counted shelter factor → -2.42 # ACH/month infiltration shortfall). sheltered_sides=_api_sheltered_sides(schema.built_form), + # RdSAP-Schema-21 `mechanical_ventilation` enum → §2 (24a..d) + # MV-kind dispatch. Without this an MEV / PIV-from-outside + # dwelling defaulted to NATURAL and under-stated its + # ventilation heat loss (+1.90 SAP over-rate on the n=20 + # MEV cohort). + mechanical_ventilation_kind=_api_mechanical_ventilation_kind( + schema.mechanical_ventilation + ), ), ) @@ -2780,6 +2788,55 @@ def _api_sheltered_sides(built_form: object) -> Optional[int]: return _API_BUILT_FORM_TO_SHELTERED_SIDES[built_form] +# GOV.UK API `mechanical_ventilation` integer (RdSAP-Schema-21 enum) → +# `MechanicalVentilationKind` enum name picking the SAP 10.2 §2 (24a..d) +# effective-air-change formula. Mirrors the Elmhurst-path +# `_ELMHURST_MV_TYPE_TO_KIND` so both source paths converge on the same +# cascade dispatch. +# 0 natural → None (NATURAL, 24d) +# 1 mechanical ventilation, no HR (MV) → MV (24b) +# 2 mechanical extract, decentralised (MEVdc)→ EXTRACT_OR_PIV_OUTSIDE (24c) +# 3 mechanical extract, centralised (MEV c) → EXTRACT_OR_PIV_OUTSIDE (24c) +# 5 positive input from loft → None — RdSAP 10 §2.6 treats +# loft-sourced PIV as natural +# (no added system air change) +# 6 positive input from outside → EXTRACT_OR_PIV_OUTSIDE (24c) +# Code 4 (MVHR, 24a) is DEFERRED: its (24a)m formula needs the lodged +# heat-recovery efficiency (`mvhr_efficiency_pct`, PCDB Table 326) which the +# API→cascade path does not yet plumb; mapping it to MVHR with a null +# efficiency would mis-model it as MV (no recovery), so it stays NATURAL +# until the efficiency is wired. The extract systems (2/3/6) carry no +# efficiency, so this slice closes the clean +1.90 SAP over-rate on the +# MEV/PIV-outside cohort (n=20, 5% within 0.5) — they were silently +# defaulting to NATURAL, under-stating ventilation heat loss. +_API_MECHANICAL_VENTILATION_TO_KIND: Final[dict[int, Optional[str]]] = { + 0: None, + 1: "MV", + 2: "EXTRACT_OR_PIV_OUTSIDE", + 3: "EXTRACT_OR_PIV_OUTSIDE", + 4: None, # MVHR — efficiency plumbing deferred; treat as natural + 5: None, + 6: "EXTRACT_OR_PIV_OUTSIDE", +} + + +def _api_mechanical_ventilation_kind(mechanical_ventilation: object) -> Optional[str]: + """Translate the API `mechanical_ventilation` integer to a + `MechanicalVentilationKind` enum name for the §2 cascade dispatch. + + Strict-coverage: a lodged integer outside the mapped set raises + `UnmappedApiCode` rather than silently defaulting to NATURAL (which + under-states ventilation heat loss for any mechanical system). + Non-int / None lodging stays as no-lodging (NATURAL).""" + if isinstance(mechanical_ventilation, str) and mechanical_ventilation.isdigit(): + mechanical_ventilation = int(mechanical_ventilation) + if not isinstance(mechanical_ventilation, int): + return None + if mechanical_ventilation not in _API_MECHANICAL_VENTILATION_TO_KIND: + raise UnmappedApiCode("mechanical_ventilation", mechanical_ventilation) + return _API_MECHANICAL_VENTILATION_TO_KIND[mechanical_ventilation] + + # GOV.UK API `glazing_type` integer → (u_value W/m²K, g_perpendicular, # frame_factor) lookup the cascade reads via `window_transmission_ # details` for per-window cascade fidelity. The cascade defaults to a diff --git a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py index 3c961919..b10fb12f 100644 --- a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py +++ b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py @@ -2715,6 +2715,45 @@ def test_api_type_1_gable_kind_maps_sheltered_and_connected_codes() -> None: assert _api_type_1_gable_kind(3) == "connected_wall" +def test_api_mechanical_ventilation_maps_extract_systems_to_cascade_kind() -> None: + # Arrange — RdSAP-Schema-21 `mechanical_ventilation` enum → the SAP + # 10.2 §2 (24a..d) MechanicalVentilationKind dispatch. The mapper + # previously dropped this field on the API path, so every mechanical + # system defaulted to NATURAL and under-stated its ventilation heat + # loss (the MEV-dc cohort, code 2, over-rated +1.90 SAP, n=20). + from datatypes.epc.domain.mapper import ( + _api_mechanical_ventilation_kind, # pyright: ignore[reportPrivateUsage] + ) + + # Act / Assert — extract / positive-input-from-outside systems carry + # the (24c) kind; MV-no-HR carries (24b); natural and PIV-from-loft + # stay NATURAL; MVHR (4) is deferred (efficiency not yet plumbed) and + # stays NATURAL rather than mis-modelling as MV. + assert _api_mechanical_ventilation_kind(0) is None # natural + assert _api_mechanical_ventilation_kind(1) == "MV" # MV, no HR + assert _api_mechanical_ventilation_kind(2) == "EXTRACT_OR_PIV_OUTSIDE" # MEV dc + assert _api_mechanical_ventilation_kind(3) == "EXTRACT_OR_PIV_OUTSIDE" # MEV c + assert _api_mechanical_ventilation_kind(4) is None # MVHR (deferred) + assert _api_mechanical_ventilation_kind(5) is None # PIV from loft + assert _api_mechanical_ventilation_kind(6) == "EXTRACT_OR_PIV_OUTSIDE" # PIV outside + assert _api_mechanical_ventilation_kind(None) is None + + +def test_api_mechanical_ventilation_unmapped_code_raises() -> None: + # Arrange — an out-of-range `mechanical_ventilation` integer is a + # spec-coverage gap: raise rather than silently default to NATURAL + # (which would under-state ventilation heat loss). Mirror of the + # `_api_sheltered_sides` / `_api_type_1_gable_kind` strict-raise. + from datatypes.epc.domain.mapper import ( + UnmappedApiCode, # pyright: ignore[reportPrivateUsage] + _api_mechanical_ventilation_kind, # pyright: ignore[reportPrivateUsage] + ) + + # Act / Assert + with pytest.raises(UnmappedApiCode): + _api_mechanical_ventilation_kind(7) + + def test_elmhurst_detailed_rir_keeps_roof_surfaces() -> None: # Arrange — a Detailed (§3.10) assessment DOES measure slope / flat # ceiling, so they must be retained (regression guard so the From dfba20babfad36c8c9550b53eec95cf4418924b7 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 8 Jun 2026 23:02:51 +0000 Subject: [PATCH 18/87] =?UTF-8?q?docs:=20session-8=20handover=20=E2=80=94?= =?UTF-8?q?=20API=20mechanical=5Fventilation=20enum=20=E2=86=92=20=C2=A72?= =?UTF-8?q?=20MV-kind=20dispatch?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 4.8 --- docs/HANDOVER_API_PROFILING.md | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/docs/HANDOVER_API_PROFILING.md b/docs/HANDOVER_API_PROFILING.md index bb842098..1dcb4d42 100644 --- a/docs/HANDOVER_API_PROFILING.md +++ b/docs/HANDOVER_API_PROFILING.md @@ -13,16 +13,33 @@ deproven approaches + the meter/shower data-fidelity findings), and the earlier `energy_rating_current`. Headline gauge: `PYTHONPATH=/workspaces/model python scripts/eval_api_sap_accuracy.py`. -| metric | session-4 (`faf29942`) | session-5 (`19235d11`) | session-6 (`a7761ea8`) | **session-7 (`3e05c95e`)** | +| metric | session-5 (`19235d11`) | session-6 (`a7761ea8`) | session-7 (`3e05c95e`) | **session-8 (`71b378b9`)** | |--------|------------------|------------------|------------------|------------------| -| **% \|err\| < 0.5** | 47.6% | 53.1% | 53.14% | **54.24%** | -| % \|err\| < 1.0 | 62.6% | 68.0% | 67.99% | **69.64%** | -| % \|err\| < 2.0 | 79.6% | ~81% | 81.85% | **83.50%** | -| mean \|err\| | 1.586 | 1.329 | 1.312 | **1.248** | -| median \|err\| | — | 0.467 | 0.467 | **0.457** | +| **% \|err\| < 0.5** | 53.1% | 53.14% | 54.24% | **55.01%** | +| % \|err\| < 1.0 | 68.0% | 67.99% | 69.64% | **70.08%** | +| % \|err\| < 2.0 | ~81% | 81.85% | 83.50% | **~83.6%** | +| mean \|err\| | 1.329 | 1.312 | 1.248 | **1.233** | +| median \|err\| | 0.467 | 0.467 | 0.457 | **0.448** | | computed / raises | 909 / 0 | 909 / 0 | 909 / 0 | **909 / 0** | | unsupported_schema | 100 (deferred) | 100 (deferred) | 100 (deferred) | 100 (deferred) | +### SESSION-8 — API `mechanical_ventilation` enum never mapped, `71b378b9` +Re-profiling after the sheltering fix surfaced `mechanical_ventilation=2` as a +clean systematic over-rate (n=20, signed +1.90, 5% within 0.5, every cert +positive). ROOT: `from_api_response` DROPPED the doc-level +`mechanical_ventilation` field, so `mechanical_ventilation_kind` was always None +and the §2 (24a..d) cascade defaulted to NATURAL — under-stating the air-change +rate for every mechanical system (only the Elmhurst path mapped it). New +`_api_mechanical_ventilation_kind` maps the RdSAP-Schema-21 enum → +MechanicalVentilationKind: 0→NATURAL, 1 MV→"MV", 2/3 MEV + 6 PIV-outside → +"EXTRACT_OR_PIV_OUTSIDE", 5 PIV-loft→NATURAL; **code 4 (MVHR) DEFERRED** (needs +the lodged HR efficiency, PCDB Table 326, the API path doesn't plumb — stays +NATURAL rather than mis-modelling as MV). Unmapped → `UnmappedApiCode`. Extract +cohort +1.90→+0.9 median (within-0.5 5%→35%), 20 improved / 3 regressed. The +**+0.9 residual is the MEV FAN ELECTRICITY** (§2.6.4 SFP, PCDB Table 322 +decentralised-MEV — 9 of 20 carry `mechanical_ventilation_index_number`): a +clean next slice. Goldens + regression green, pyright net-zero. + ### SESSION-7 — sheltered alternative walls (RdSAP Table 4 R=0.5), `3e05c95e` The headline-moving audit. User: 53% is poor enough to indicate a MAJOR error — audit again. The decisive diagnostic CHAIN (reusable): From e7af6fda6638cc01c42e26c3872102de4464a809 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 8 Jun 2026 23:16:34 +0000 Subject: [PATCH 19/87] fix(ventilation): map API mechanical_ventilation_index_number for MEV fan electricity MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Follow-up to the §2 MV-kind slice. Once MEV dwellings stopped under-stating their ventilation HEAT loss, a +0.9 SAP over-rate residual remained — the MEV FAN ELECTRICITY (§5 Table 4f line (230a), `SFPav × 1.22 × V`, PCDB Tables 322 decentralised-MEV + 329 in-use factors). `_mev_decentralised_kwh_per_yr_from_cert` already composes it, but reads `epc.mechanical_ventilation_index_number` + `epc.mechanical_vent_duct_type`, and the API builder (`from_rdsap_schema_21_0_1`) never set either — so `pcdf_id is None` short-circuited the fan energy to 0 on every API cert (the Summary/ Elmhurst path set them, so cert 000565 already billed it). Wire both schema fields through the 21.0.1 API construction (the corpus schema). Eval: the 9 MEV certs carrying a PCDB index closed +0.90 -> +0.13 signed (fan electricity now billed); headline within-0.5 55.01% -> 55.12%, mean|err| 1.233 -> 1.232, 909 computed / 0 raises. Only those 9 certs move (clean diff). The 11 index-less MEV certs still sit at +1.36 — they need the SAP Table 4h DEFAULT specific fan power (no PCDB record), a separate slice. New end-to-end test + fixture (cert 1300, Titon-class dMEV index 500777, Flexible duct): from_api_response preserves the index + duct type and (230a) resolves to a positive fan-energy contribution. Goldens + full calc/epc regression green; pyright net-zero. Co-Authored-By: Claude Opus 4.8 --- datatypes/epc/domain/mapper.py | 12 +++++++++ .../golden/1300-7634-0922-3203-3563.json | 1 + .../rdsap/test_golden_fixtures.py | 25 +++++++++++++++++++ 3 files changed, 38 insertions(+) create mode 100644 tests/domain/sap10_calculator/rdsap/fixtures/golden/1300-7634-0922-3203-3563.json diff --git a/datatypes/epc/domain/mapper.py b/datatypes/epc/domain/mapper.py index b9de5feb..0c7a0e76 100644 --- a/datatypes/epc/domain/mapper.py +++ b/datatypes/epc/domain/mapper.py @@ -1559,6 +1559,18 @@ class EpcPropertyDataMapper: open_chimneys_count=schema.open_chimneys_count, insulated_door_count=schema.insulated_door_count, draughtproofed_door_count=schema.draughtproofed_door_count, + # Mechanical ventilation PCDB plumbing — feeds the §5 Table 4f + # line (230a) decentralised-MEV fan electricity + # (`SFPav × 1.22 × V`, PCDB Tables 322/329). Without the index + # the cascade's `_mev_decentralised_kwh_per_yr_from_cert` + # short-circuits to 0, leaving the MEV fan running cost off the + # bill (the +0.9 SAP over-rate residual on the MEV cohort once + # the §2 ventilation heat-loss was fixed). Duct type selects the + # Table 329 in-use factor (1=Flexible / 2=Rigid). + mechanical_ventilation_index_number=( + schema.mechanical_ventilation_index_number + ), + mechanical_vent_duct_type=schema.mechanical_vent_duct_type, # Lighting led_fixed_lighting_bulbs_count=schema.led_fixed_lighting_bulbs_count, cfl_fixed_lighting_bulbs_count=schema.cfl_fixed_lighting_bulbs_count, diff --git a/tests/domain/sap10_calculator/rdsap/fixtures/golden/1300-7634-0922-3203-3563.json b/tests/domain/sap10_calculator/rdsap/fixtures/golden/1300-7634-0922-3203-3563.json new file mode 100644 index 00000000..3e4b64aa --- /dev/null +++ b/tests/domain/sap10_calculator/rdsap/fixtures/golden/1300-7634-0922-3203-3563.json @@ -0,0 +1 @@ +{"uprn": 100010371693, "roofs": [{"description": "Pitched, 400+ mm loft insulation", "energy_efficiency_rating": 5, "environmental_efficiency_rating": 5}], "walls": [{"description": "Cavity wall, filled cavity", "energy_efficiency_rating": 4, "environmental_efficiency_rating": 4}], "floors": [{"description": "Solid, no insulation (assumed)", "energy_efficiency_rating": 0, "environmental_efficiency_rating": 0}], "status": "entered", "tenure": 2, "window": {"description": "Fully double glazed", "energy_efficiency_rating": 2, "environmental_efficiency_rating": 2}, "addendum": {"addendum_numbers": [8]}, "lighting": {"description": "Good lighting efficiency", "energy_efficiency_rating": 4, "environmental_efficiency_rating": 4}, "postcode": "PR6 0AZ", "hot_water": {"description": "From main system", "energy_efficiency_rating": 4, "environmental_efficiency_rating": 4}, "post_town": "CHORLEY", "built_form": 2, "created_at": "2026-05-12 09:06:05", "door_count": 2, "region_code": 19, "report_type": 2, "sap_heating": {"number_baths": 0, "cylinder_size": 1, "shower_outlets": [{"shower_wwhrs": 1, "shower_outlet_type": 2}], "number_baths_wwhrs": 0, "water_heating_code": 901, "water_heating_fuel": 26, "secondary_fuel_type": 29, "main_heating_details": [{"has_fghrs": "N", "main_fuel_type": 26, "boiler_flue_type": 2, "fan_flue_present": "Y", "heat_emitter_type": 1, "ttzc_index_number": 200131, "emitter_temperature": 0, "main_heating_number": 1, "main_heating_control": 2112, "main_heating_category": 2, "main_heating_fraction": 1, "central_heating_pump_age": 0, "main_heating_data_source": 1, "main_heating_index_number": 19080}], "immersion_heating_type": "NA", "secondary_heating_type": 691, "has_fixed_air_conditioning": "false"}, "sap_version": 10.2, "sap_windows": [{"pvc_frame": "true", "glazing_gap": 12, "orientation": 3, "window_type": 1, "glazing_type": 3, "window_width": 0.4, "window_height": 1.3, "draught_proofed": "true", "window_location": 0, "window_wall_type": 1, "permanent_shutters_present": "N", "permanent_shutters_insulated": "N"}, {"pvc_frame": "true", "glazing_gap": 12, "orientation": 3, "window_type": 1, "glazing_type": 3, "window_width": 0.4, "window_height": 1.3, "draught_proofed": "true", "window_location": 0, "window_wall_type": 1, "permanent_shutters_present": "N", "permanent_shutters_insulated": "N"}, {"pvc_frame": "true", "glazing_gap": 12, "orientation": 3, "window_type": 1, "glazing_type": 3, "window_width": 1.75, "window_height": 1.3, "draught_proofed": "true", "window_location": 0, "window_wall_type": 1, "permanent_shutters_present": "N", "permanent_shutters_insulated": "N"}, {"pvc_frame": "true", "glazing_gap": 12, "orientation": 7, "window_type": 1, "glazing_type": 3, "window_width": 1.1, "window_height": 1.1, "draught_proofed": "true", "window_location": 0, "window_wall_type": 1, "permanent_shutters_present": "N", "permanent_shutters_insulated": "N"}, {"pvc_frame": "true", "glazing_gap": 12, "orientation": 7, "window_type": 1, "glazing_type": 3, "window_width": 1.2, "window_height": 1.2, "draught_proofed": "true", "window_location": 0, "window_wall_type": 1, "permanent_shutters_present": "N", "permanent_shutters_insulated": "N"}], "schema_type": "RdSAP-Schema-21.0.1", "uprn_source": "Energy Assessor", "country_code": "ENG", "main_heating": [{"description": "Boiler and radiators, mains gas", "energy_efficiency_rating": 4, "environmental_efficiency_rating": 4}], "air_tightness": {"description": "(not tested)", "energy_efficiency_rating": 0, "environmental_efficiency_rating": 0}, "dwelling_type": "Semi-detached bungalow", "language_code": 1, "pressure_test": 4, "property_type": 1, "address_line_1": "27 Epping Place", "assessment_type": "RdSAP", "completion_date": "2026-05-12", "inspection_date": "2026-05-07", "wet_rooms_count": 2, "extensions_count": 0, "measurement_type": 1, "total_floor_area": 39, "transaction_type": 5, "conservatory_type": 1, "heated_room_count": 2, "registration_date": "2026-05-12", "sap_energy_source": {"mains_gas": "Y", "meter_type": 2, "pv_connection": 2, "photovoltaic_supply": [[{"pitch": 3, "peak_power": 1.35, "orientation": 7, "overshading": 1}], [{"pitch": 3, "peak_power": 1.35, "orientation": 3, "overshading": 1}]], "wind_turbines_count": 0, "gas_smart_meter_present": "true", "is_dwelling_export_capable": "true", "wind_turbines_terrain_type": 2, "electricity_smart_meter_present": "true"}, "secondary_heating": {"description": "Room heaters, electric", "energy_efficiency_rating": 0, "environmental_efficiency_rating": 0}, "lzc_energy_sources": [11], "sap_building_parts": [{"identifier": "Main Dwelling", "wall_dry_lined": "N", "wall_thickness": 300, "floor_heat_loss": 7, "roof_construction": 4, "wall_construction": 4, "building_part_number": 1, "sap_floor_dimensions": [{"floor": 0, "room_height": {"value": 2.48, "quantity": "metres"}, "floor_insulation": 1, "total_floor_area": {"value": 39.2, "quantity": "square metres"}, "party_wall_length": {"value": 7, "quantity": "metres"}, "floor_construction": 1, "heat_loss_perimeter": {"value": 18.2, "quantity": "metres"}}], "wall_insulation_type": 2, "construction_age_band": "C", "party_wall_construction": 0, "wall_thickness_measured": "Y", "roof_insulation_location": 2, "roof_insulation_thickness": "400mm+", "wall_insulation_thickness": "NI", "floor_insulation_thickness": "NI"}], "solar_water_heating": "N", "habitable_room_count": 2, "heating_cost_current": {"value": 670, "currency": "GBP"}, "insulated_door_count": 0, "co2_emissions_current": 1.2, "energy_rating_average": 60, "energy_rating_current": 82, "lighting_cost_current": {"value": 29, "currency": "GBP"}, "main_heating_controls": [{"description": "Time and temperature zone control", "energy_efficiency_rating": 5, "environmental_efficiency_rating": 5}], "has_hot_water_cylinder": "false", "heating_cost_potential": {"value": 601, "currency": "GBP"}, "hot_water_cost_current": {"value": 203, "currency": "GBP"}, "mechanical_ventilation": 2, "percent_draughtproofed": 100, "suggested_improvements": [{"sequence": 1, "typical_saving": {"value": 68, "currency": "GBP"}, "indicative_cost": "\u00a35,000 - \u00a310,000", "improvement_type": "W2", "improvement_details": {"improvement_number": 58}, "improvement_category": 5, "energy_performance_rating": 84, "environmental_impact_rating": 84}], "co2_emissions_potential": 1.0, "energy_rating_potential": 84, "kitchen_duct_fans_count": 0, "kitchen_room_fans_count": 1, "kitchen_wall_fans_count": 0, "lighting_cost_potential": {"value": 29, "currency": "GBP"}, "schema_version_original": "21.0.1", "hot_water_cost_potential": {"value": 203, "currency": "GBP"}, "renewable_heat_incentive": {"water_heating": 1148.49, "space_heating_existing_dwelling": 5025.34}, "draughtproofed_door_count": 2, "mechanical_vent_duct_type": 1, "energy_consumption_current": 181, "has_fixed_air_conditioning": "false", "multiple_glazed_proportion": 100, "non_kitchen_duct_fans_count": 0, "non_kitchen_room_fans_count": 1, "non_kitchen_wall_fans_count": 0, "calculation_software_version": "5.02r0344", "energy_consumption_potential": 158, "environmental_impact_current": 81, "current_energy_efficiency_band": "B", "environmental_impact_potential": 84, "has_heated_separate_conservatory": "false", "potential_energy_efficiency_band": "B", "mechanical_ventilation_index_number": 500777, "co2_emissions_current_per_floor_area": 30, "low_energy_fixed_lighting_bulbs_count": 5, "incandescent_fixed_lighting_bulbs_count": 0, "is_mechanical_vent_approved_installer_scheme": "false"} \ No newline at end of file diff --git a/tests/domain/sap10_calculator/rdsap/test_golden_fixtures.py b/tests/domain/sap10_calculator/rdsap/test_golden_fixtures.py index 99dbad76..63e3b83d 100644 --- a/tests/domain/sap10_calculator/rdsap/test_golden_fixtures.py +++ b/tests/domain/sap10_calculator/rdsap/test_golden_fixtures.py @@ -983,6 +983,31 @@ def test_api_to_domain_mapper_preserves_main_heating_index_number( assert abs(inputs.main_heating_efficiency - expected_winter_eff) <= 1e-3 +def test_api_mapper_preserves_mechanical_ventilation_pcdb_index_for_mev_fan_energy() -> None: + # Arrange — cert 1300 is a decentralised-MEV dwelling + # (mechanical_ventilation=2) lodging a PCDB index (500777) + duct type + # (1=Flexible). The API path previously DROPPED + # `mechanical_ventilation_index_number`, so the §5 Table 4f line (230a) + # MEV fan electricity (`SFPav × 1.22 × V`, PCDB Tables 322/329) + # short-circuited to 0 in `_mev_decentralised_kwh_per_yr_from_cert` — + # leaving the fan running cost off the bill (+0.9 SAP over-rate residual + # on the MEV cohort once the §2 ventilation heat loss was fixed). + from domain.sap10_calculator.rdsap.cert_to_inputs import ( + _mev_decentralised_kwh_per_yr_from_cert, # pyright: ignore[reportPrivateUsage] + ) + + doc = _load_cert("1300-7634-0922-3203-3563") + + # Act + epc = EpcPropertyDataMapper.from_api_response(doc) + + # Assert — the PCDB pointer + duct type survive the API mapping, and the + # (230a) MEV fan electricity resolves to a positive contribution. + assert epc.mechanical_ventilation_index_number == 500777 + assert epc.mechanical_vent_duct_type == 1 + assert _mev_decentralised_kwh_per_yr_from_cert(epc) > 0.0 + + def test_0240_api_wall_type_4_windows_map_to_roof_windows() -> None: """Cert 0240 lodges 6 windows with `window_wall_type=4` — the RdSAP API code for a roof window ("Roof of Room" rooflight / inclined From 01ebc9ac1e309dc8f04c942553a71db7204373fe Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 8 Jun 2026 23:17:17 +0000 Subject: [PATCH 20/87] =?UTF-8?q?docs:=20session-8b=20handover=20=E2=80=94?= =?UTF-8?q?=20MEV=20fan=20electricity=20(PCDB=20index=20plumbed)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 4.8 --- docs/HANDOVER_API_PROFILING.md | 25 +++++++++++++++++++------ 1 file changed, 19 insertions(+), 6 deletions(-) diff --git a/docs/HANDOVER_API_PROFILING.md b/docs/HANDOVER_API_PROFILING.md index 1dcb4d42..7776af24 100644 --- a/docs/HANDOVER_API_PROFILING.md +++ b/docs/HANDOVER_API_PROFILING.md @@ -13,16 +13,29 @@ deproven approaches + the meter/shower data-fidelity findings), and the earlier `energy_rating_current`. Headline gauge: `PYTHONPATH=/workspaces/model python scripts/eval_api_sap_accuracy.py`. -| metric | session-5 (`19235d11`) | session-6 (`a7761ea8`) | session-7 (`3e05c95e`) | **session-8 (`71b378b9`)** | +| metric | session-6 (`a7761ea8`) | session-7 (`3e05c95e`) | session-8 (`71b378b9`) | **session-8b (`e7af6fda`)** | |--------|------------------|------------------|------------------|------------------| -| **% \|err\| < 0.5** | 53.1% | 53.14% | 54.24% | **55.01%** | -| % \|err\| < 1.0 | 68.0% | 67.99% | 69.64% | **70.08%** | -| % \|err\| < 2.0 | ~81% | 81.85% | 83.50% | **~83.6%** | -| mean \|err\| | 1.329 | 1.312 | 1.248 | **1.233** | -| median \|err\| | 0.467 | 0.467 | 0.457 | **0.448** | +| **% \|err\| < 0.5** | 53.14% | 54.24% | 55.01% | **55.12%** | +| % \|err\| < 1.0 | 67.99% | 69.64% | 70.08% | **70.08%** | +| % \|err\| < 2.0 | 81.85% | 83.50% | ~83.6% | **~83.6%** | +| mean \|err\| | 1.312 | 1.248 | 1.233 | **1.232** | +| median \|err\| | 0.467 | 0.457 | 0.448 | **0.446** | | computed / raises | 909 / 0 | 909 / 0 | 909 / 0 | **909 / 0** | | unsupported_schema | 100 (deferred) | 100 (deferred) | 100 (deferred) | 100 (deferred) | +### SESSION-8b — MEV fan electricity (PCDB index plumbed), `e7af6fda` +Follow-up to 8: the +0.9 residual on MEV after the §2 heat-loss fix was the FAN +electricity (§5 Table 4f (230a) `SFPav × 1.22 × V`, PCDB Tables 322/329). +`_mev_decentralised_kwh_per_yr_from_cert` already composes it but reads +`epc.mechanical_ventilation_index_number` + `mechanical_vent_duct_type`, which the +API builder never set → `pcdf_id is None` zeroed the fan energy on every API cert. +Wired both through the 21.0.1 construction. The 9 MEV certs with a PCDB index +closed +0.90 → +0.13; the **11 index-less MEV certs still sit at +1.36** — they +need the SAP Table 4h DEFAULT specific fan power (no PCDB record), a clean next +slice (verify the default SFP value against spec first). New e2e test + golden +fixture (cert 1300, dMEV index 500777). NB the 21.0.0 API builder didn't get the +8/8b ventilation edits (only 21.0.1, the corpus schema). + ### SESSION-8 — API `mechanical_ventilation` enum never mapped, `71b378b9` Re-profiling after the sheltering fix surfaced `mechanical_ventilation=2` as a clean systematic over-rate (n=20, signed +1.90, 5% within 0.5, every cert From e6dda705f4711d31b05578bc746ba36cb9967ec4 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Tue, 9 Jun 2026 09:00:54 +0000 Subject: [PATCH 21/87] fix(ventilation): apply Table 4g default SFP to index-less MEV fan electricity MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Completes the MEV fan-electricity thread. The PCDB-index slice closed the 9 MEV certs carrying a Table 322 record; the other 11 (mostly gas houses) lodge mechanical_ventilation=2 with NO PCDB index, so `_mev_decentralised_kwh_per_yr_from_cert` returned 0 and billed no fan running cost — a tight +2.2 SAP over-rate (signed +1.23, median +2.19). SAP 10.2 §2.6.3 / Table 4g note 1 (PDF p.176) prescribes a DEFAULT specific fan power of 0.8 W/(l/s) for an MEV system whose fans are not in the PCDB, used directly as SFPav in the §5 Table 4f (230a) formula (SFPav × 1.22 × V). Restructure the helper: when no Table 322 record resolves, fall back to the default for a mechanical-extract system (`mechanical_ventilation_kind == EXTRACT_OR_PIV_OUTSIDE`); natural / balanced (MVHR / MV) systems still contribute nothing. Index-less extract cohort closed +1.23 -> +0.18 signed (each gains ~1.1 SAP of fan electricity). This is a spec-correct fix that improves the aggregate but is a HEADLINE TRADE-OFF: within-2.0 83.6% -> 84.6%, within-1.0 70.08% -> 70.19%, mean|err| 1.232 -> 1.224, but within-0.5 55.12% -> 54.90% (-2) — the fan energy is only ~half each cert's over-rate, so the cohort lands at ~+1.0 (still outside 0.5) while two borderline certs with offsetting errors cross out. Applied uniformly per the determinism principle ([[feedback_software_no_special_handling]]): the unmasked residual (~+1.0 on gas-house MEV) is the next lead. 1 AAA test (default SFP 0.8 × 1.22 × V for index-less MEV, 0 for natural). Goldens + full calc/epc regression green (000565 MEV uses its resolvable PCDB record, unaffected); pyright net-zero. Co-Authored-By: Claude Opus 4.8 --- .../sap10_calculator/rdsap/cert_to_inputs.py | 31 +++++++++++-- .../rdsap/test_cert_to_inputs.py | 43 +++++++++++++++++++ 2 files changed, 70 insertions(+), 4 deletions(-) diff --git a/domain/sap10_calculator/rdsap/cert_to_inputs.py b/domain/sap10_calculator/rdsap/cert_to_inputs.py index 31709822..15f14149 100644 --- a/domain/sap10_calculator/rdsap/cert_to_inputs.py +++ b/domain/sap10_calculator/rdsap/cert_to_inputs.py @@ -566,6 +566,12 @@ _MV_DUCT_TYPE_RIGID: Final[int] = 2 # 5, 6 = through-wall (use no-duct IUF independent of duct type) _MEV_THROUGH_WALL_CONFIG_CODES: Final[frozenset[int]] = frozenset({5, 6}) +# SAP 10.2 Table 4g (PDF p.176) / §2.6.3 note 1 — default specific fan +# power (W per litre/sec) for an MEV system whose fan(s) are not in the +# PCDB. Used directly as the IUF-adjusted SFPav in the (230a) formula +# (SFPav × 1.22 × V) when no Table 322 record resolves. +_TABLE_4G_DEFAULT_MEV_SFP_W_PER_L_PER_S: Final[float] = 0.8 + def _mev_decentralised_kwh_per_yr_from_cert(epc: EpcPropertyData) -> float: """Compose the SAP 10.2 §5 Table 4f line (230a) MEV decentralised @@ -601,11 +607,28 @@ def _mev_decentralised_kwh_per_yr_from_cert(epc: EpcPropertyData) -> float: decentralised fixture; the rule above fits cert 000565 alone. """ pcdf_id = epc.mechanical_ventilation_index_number - if pcdf_id is None: - return 0.0 - record = decentralised_mev_record(pcdf_id) + record = decentralised_mev_record(pcdf_id) if pcdf_id is not None else None if record is None: - return 0.0 + # No PCDB Table 322 record (index absent, or lodged index not in + # the table). For a mechanical-EXTRACT system, SAP 10.2 §2.6.3 / + # Table 4g note 1 prescribes the default SFP (0.8 W/(l/s)) used + # directly as SFPav. Natural / balanced (MVHR / MV) systems + # contribute no (230a) decentralised-MEV fan electricity here — + # the gate mirrors `_has_balanced_mechanical_ventilation`'s + # MEV / PIV-from-outside vs balanced split. Closes the +2.2 SAP + # over-rate on the index-less MEV cohort (mostly gas houses), which + # previously billed zero fan electricity. + sv = epc.sap_ventilation + if ( + sv is None + or sv.mechanical_ventilation_kind + != MechanicalVentilationKind.EXTRACT_OR_PIV_OUTSIDE.name + ): + return 0.0 + return mev_decentralised_kwh_per_yr( + sfp_av_w_per_l_per_s=_TABLE_4G_DEFAULT_MEV_SFP_W_PER_L_PER_S, + dwelling_volume_m3=dimensions_from_cert(epc).volume_m3, + ) iuf_record = mv_in_use_factors_record(_MEV_DECENTRALISED_SYSTEM_TYPE) if iuf_record is None: return 0.0 diff --git a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py index b10fb12f..516a5e9e 100644 --- a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py +++ b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py @@ -1032,6 +1032,49 @@ def test_ventilation_from_cert_routes_mev_decentralised_to_extract_or_piv_outsid assert abs(w25 - expected) <= 1e-4 +def test_index_less_mev_uses_table_4g_default_sfp_for_fan_electricity() -> None: + # Arrange — an MEV system with NO PCDB record (index absent / not in + # Table 322). SAP 10.2 §2.6.3 / Table 4g note 1 prescribes a default + # specific fan power of 0.8 W/(l/s), used directly as the SFPav in the + # §5 Table 4f line (230a) `SFPav × 1.22 × V`. Without it the cascade + # billed ZERO fan electricity for index-less MEV (the +2.2 SAP + # over-rate on the index-less MEV cohort, mostly gas houses). A natural + # dwelling contributes nothing. + from domain.sap10_calculator.rdsap.cert_to_inputs import ( + _TABLE_4G_DEFAULT_MEV_SFP_W_PER_L_PER_S, # pyright: ignore[reportPrivateUsage] + _mev_decentralised_kwh_per_yr_from_cert, # pyright: ignore[reportPrivateUsage] + dimensions_from_cert, + ) + + base = _typical_semi_detached_epc() + mev_no_index = make_minimal_sap10_epc( + total_floor_area_m2=_TYPICAL_TFA_M2, habitable_rooms_count=4, + region_code="1", sap_building_parts=base.sap_building_parts, + sap_windows=base.sap_windows, sap_heating=base.sap_heating, + sap_ventilation=SapVentilation( + mechanical_ventilation_kind="EXTRACT_OR_PIV_OUTSIDE" + ), + ) + natural = make_minimal_sap10_epc( + total_floor_area_m2=_TYPICAL_TFA_M2, habitable_rooms_count=4, + region_code="1", sap_building_parts=base.sap_building_parts, + sap_windows=base.sap_windows, sap_heating=base.sap_heating, + sap_ventilation=SapVentilation(extract_fans_count=0), + ) + + # Act + mev_kwh = _mev_decentralised_kwh_per_yr_from_cert(mev_no_index) + natural_kwh = _mev_decentralised_kwh_per_yr_from_cert(natural) + + # Assert — default SFP 0.8 × 1.22 × V for the index-less MEV; zero for + # the naturally-ventilated dwelling. + volume = dimensions_from_cert(mev_no_index).volume_m3 + expected = _TABLE_4G_DEFAULT_MEV_SFP_W_PER_L_PER_S * 1.22 * volume + assert abs(mev_kwh - expected) <= 1e-6 + assert mev_kwh > 0.0 + assert natural_kwh == 0.0 + + def test_open_chimneys_raise_infiltration_ach() -> None: # Arrange — Direction check: chimneys add Table 2.1 volume to the # infiltration calc, so an otherwise identical dwelling with 2 open From da094feb6287382c69e0e9c42dce3f9ad86e9f64 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Tue, 9 Jun 2026 09:01:37 +0000 Subject: [PATCH 22/87] =?UTF-8?q?docs:=20session-8c=20handover=20=E2=80=94?= =?UTF-8?q?=20Table=204g=20default=20SFP=20for=20index-less=20MEV=20(withi?= =?UTF-8?q?n-0.5=20trade-off)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 4.8 --- docs/HANDOVER_API_PROFILING.md | 25 +++++++++++++++++++------ 1 file changed, 19 insertions(+), 6 deletions(-) diff --git a/docs/HANDOVER_API_PROFILING.md b/docs/HANDOVER_API_PROFILING.md index 7776af24..c31f2915 100644 --- a/docs/HANDOVER_API_PROFILING.md +++ b/docs/HANDOVER_API_PROFILING.md @@ -13,16 +13,29 @@ deproven approaches + the meter/shower data-fidelity findings), and the earlier `energy_rating_current`. Headline gauge: `PYTHONPATH=/workspaces/model python scripts/eval_api_sap_accuracy.py`. -| metric | session-6 (`a7761ea8`) | session-7 (`3e05c95e`) | session-8 (`71b378b9`) | **session-8b (`e7af6fda`)** | +| metric | session-7 (`3e05c95e`) | session-8 (`71b378b9`) | session-8b (`e7af6fda`) | **session-8c (`e6dda705`)** | |--------|------------------|------------------|------------------|------------------| -| **% \|err\| < 0.5** | 53.14% | 54.24% | 55.01% | **55.12%** | -| % \|err\| < 1.0 | 67.99% | 69.64% | 70.08% | **70.08%** | -| % \|err\| < 2.0 | 81.85% | 83.50% | ~83.6% | **~83.6%** | -| mean \|err\| | 1.312 | 1.248 | 1.233 | **1.232** | -| median \|err\| | 0.467 | 0.457 | 0.448 | **0.446** | +| **% \|err\| < 0.5** | 54.24% | 55.01% | 55.12% | **54.90%** ⚠ | +| % \|err\| < 1.0 | 69.64% | 70.08% | 70.08% | **70.19%** | +| % \|err\| < 2.0 | 83.50% | ~83.6% | ~83.6% | **84.60%** | +| mean \|err\| | 1.248 | 1.233 | 1.232 | **1.224** | +| median \|err\| | 0.457 | 0.448 | 0.446 | **0.448** | | computed / raises | 909 / 0 | 909 / 0 | 909 / 0 | **909 / 0** | | unsupported_schema | 100 (deferred) | 100 (deferred) | 100 (deferred) | 100 (deferred) | +### SESSION-8c — Table 4g default SFP for index-less MEV (HEADLINE TRADE-OFF), `e6dda705` +The 11 index-less MEV certs (mostly GAS houses, `mechanical_ventilation=2`, no PCDB +index) billed ZERO fan electricity → +2.2 over-rate (signed +1.23, median +2.19). +SAP 10.2 §2.6.3 / Table 4g note 1 (p.176): default SFP **0.8 W/(l/s)** used directly +as SFPav in (230a). `_mev_decentralised_kwh_per_yr_from_cert` now falls back to it +for a mechanical-EXTRACT system when no Table 322 record resolves. Cohort closed ++1.23 → +0.18 signed. **TRADE-OFF: within-0.5 55.12% → 54.90% (−2)** — the fan energy +is only ~half each cert's over-rate, so the cohort lands at ~+1.0 (still outside 0.5) +while 2 borderline offsetting-error certs cross out; within-1.0/within-2.0/mean|err| +all improve. Spec-correct, applied uniformly per the determinism principle. **NEXT: +the unmasked ~+1.0 residual on gas-house MEV — the OTHER half (not electric-flat +fabric, since these are gas houses).** Goldens green, pyright net-zero. + ### SESSION-8b — MEV fan electricity (PCDB index plumbed), `e7af6fda` Follow-up to 8: the +0.9 residual on MEV after the §2 heat-loss fix was the FAN electricity (§5 Table 4f (230a) `SFPav × 1.22 × V`, PCDB Tables 322/329). From ddb9fdbec583e1057cd3cacca66053f74b494357 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Tue, 9 Jun 2026 09:24:40 +0000 Subject: [PATCH 23/87] =?UTF-8?q?docs:=20session-9=20sweep=20=E2=80=94=20s?= =?UTF-8?q?ix=20API-error=20candidates=20ruled=20out=20(no=20shipped=20fix?= =?UTF-8?q?)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Profile-driven re-sweep at HEAD da094feb. Every biased/error-carrying bucket chased field-by-field resolved to proxy / already-deproven / already-fixed: roof code 5/8 (same u_roof as code 4), per-bp age mapping (correct), '(same dwelling above)' roofs (5 certs, 4 fine), index-less MEV gas (centred by e6dda705 to signed +0.09), wit=4 cavity -0.25 (tail-driven), community whc=903 HW (= deproven meter_type=3). The +32 outlier 2958 is per-cert (twin 3420 is +0.18). Residual is a broad per-cert fabric+HW tail. Co-Authored-By: Claude Opus 4.8 --- docs/HANDOVER_API_PROFILING.md | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/docs/HANDOVER_API_PROFILING.md b/docs/HANDOVER_API_PROFILING.md index c31f2915..930bd71b 100644 --- a/docs/HANDOVER_API_PROFILING.md +++ b/docs/HANDOVER_API_PROFILING.md @@ -381,6 +381,39 @@ bug, not noise): on unsupported schema 19.1.0; type 4 already accurate). `shower_wwhrs` 1/2/3/4 = none / inst- WWHRS-1 / inst-WWHRS-2 / storage. Low headline value — not worth pursuing. +### SESSION-9 — profile sweep, six candidates ruled out (no shipped fix) +Re-ran `profile_api_error.py --min-n 10` at HEAD `da094feb` and chased every +biased/error-carrying bucket field-by-field. All resolved to proxy / already-deproven / +already-fixed — **no clean systematic bug found at this resolution**: +- **roof_construction code 5/8 (vaulted/sloping) under-rate** (gas: `5`−0.67, `4,5`−0.59, + `4,8`−0.85): a PROXY. `u_roof` returns the SAME value for code 4 and code 5 at a given + age/thickness (band G ND → 0.40 both; verified in `rdsap_uvalues.u_roof`). Code-5 certs + correlate with electric HW / flats / multi-part dwellings — those are the cause, not the roof. +- **age_band dropped on the roof path:** NOT a bug. Doc-level `construction_age_band` is None for + ALL 909 certs (age lives per-building-part); the mapper reads the bp-level band correctly + (`heat_transmission.py:735 part.construction_age_band`; cert 2270 → bp0 age=G mapped fine). +- **roof description "(same dwelling above)"** (a heated dwelling above → should be zero-loss + internal element): only 5 certs carry it and 4 are already within 0.5. Cert 0700 (−6.37) is a + messy 4-building-part data-fidelity cert (mixed Flat-no-insulation + Roof-room parts), not a + "same dwelling above" bug. Not systematic. +- **index-less MEV gas residual (the post-e6dda705 "next lead"):** RESOLVED by e6dda705. The 17 + index-less-MEV gas certs are now signed +0.09 / median +0.43 / 41% within-0.5 — centred scatter, + not a systematic over-rate. The ~6 certs at +1.0..+1.9 are per-cert, no common cause. +- **wit=4 (as-built cavity) gas −0.25 over 478 certs:** tail-driven, not a uniform Table-6 shift. + Splitting by age band, the worst bands (B −0.54, F −0.60, G −1.00) still hold 70–77% within-0.5 + — a few big-negative outliers drag the mean; correcting a Table-6 value would over-shift the + majority that are already accurate. +- **community-main + whc=903 electric-immersion HW under-cost (−6.3, n=3):** = the deproven + **meter_type=3** data-fidelity issue. All three (0380/2270/2673) carry meter_type=3; the HW + electricity tariff ambiguity (lodged HW ≈17.3 p/kWh vs our standard ≈22.36 p) is exactly the + Unknown-meter artifact already on this list. +- **The biggest +32 outlier 2958** (fuel=0 / sapcode=699): per-cert, NOT a class bug — cert 3420 + has an identical heating profile and sits at +0.18. 2958's error is fabric/geometry-specific. +Method note: `decompose_api_cost_error.py` cluster table = heat:high 311 (47% within) / +heat:low 206 / hw:low 173 / hw:high 125 / balanced 94 — i.e. the residual is a broad per-cert +fabric+HW tail, not one lever. The clean systematic wins (sheltered walls, MEV, fuel collisions) +are harvested; remaining headway is per-cert worksheet grind or the 100-cert schema big-ticket. + ## THE 100 unsupported_schema CERTS (deferred — bigger ticket) SAP-Schema-19.1.0 (and other pre-21). The user is planning a separate big piece: map old schemas → new + **predict missing fields from similar-looking properties** (needs an EPC-prediction From 7878a96900367144d02b1c1b31b7f1340e4caa3c Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Tue, 9 Jun 2026 10:05:57 +0000 Subject: [PATCH 24/87] fix(fuel): strict-raise on unmapped Table-12 factor fuel codes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Tier-1 finding of the silent-fallback audit. The fuel-type helpers fed the SAP 10.2 Table 12/32 cost/CO2/PE lookups via a silent `API_FUEL_TO_TABLE_12.get(fuel, fuel)` passthrough at 5 sites (_heat_network_factor_fuel_code, HW CO2/PE, _secondary_fuel_code, PV). A fuel code in NEITHER the API enum map NOR the Table-12 numbering passed straight through to the mains-gas default baked into unit_price_p_per_kwh / co2_factor_kg_per_kwh / primary_energy_factor (table_12.py:233/274/287, table_32.py:190) — silently mis-pricing a novel/colliding fuel as grid gas. This is the class that mis-priced cert 8536's community biomass as electricity (-17 SAP) before a7761ea8. New _table_12_factor_fuel_code mirrors .get(fuel, fuel) EXACTLY for every recognised input (union of the CO2/PE/price/monthly table keys + API_FUEL_TO_TABLE_12 values) and raises UnmappedSapCode only when the resolved code is recognised by no table — surfacing the gap loudly per the strict-raise principle (reference_unmapped_sap_code). Verified behaviour- preserving: 0/909 corpus certs hit the new raise; eval unchanged at 54.9% within-0.5 / 909 computed / 0 raises. Co-Authored-By: Claude Opus 4.8 --- .../sap10_calculator/rdsap/cert_to_inputs.py | 54 +++++++++++++++---- .../rdsap/test_cert_to_inputs.py | 37 +++++++++++++ 2 files changed, 82 insertions(+), 9 deletions(-) diff --git a/domain/sap10_calculator/rdsap/cert_to_inputs.py b/domain/sap10_calculator/rdsap/cert_to_inputs.py index 15f14149..4c0941ac 100644 --- a/domain/sap10_calculator/rdsap/cert_to_inputs.py +++ b/domain/sap10_calculator/rdsap/cert_to_inputs.py @@ -87,7 +87,10 @@ from domain.sap10_calculator.tables.pcdb.postcode_weather import ( from domain.sap10_calculator.tables.table_12 import ( API_FUEL_TO_TABLE_12, CO2_KG_PER_KWH, + CO2_KG_PER_KWH_MONTHLY, + PE_FACTOR_MONTHLY, PRIMARY_ENERGY_FACTOR, + UNIT_PRICE_P_PER_KWH, _DEFAULT_CO2_KG_PER_KWH, # pyright: ignore[reportPrivateUsage] _DEFAULT_PEF, # pyright: ignore[reportPrivateUsage] co2_monthly_factors_kg_per_kwh, @@ -2040,6 +2043,42 @@ def _main_fuel_code(main: Optional[MainHeatingDetail]) -> Optional[int]: raise MissingMainFuelType(fuel, main.sap_main_heating_code) +# Fuel codes the Table 12 / Table 32 factor & price lookups recognise as a +# DIRECT key (vs falling through to their mains-gas default). The union of +# every per-fuel column the cost/CO2/PE cascade consumes, plus the values +# `API_FUEL_TO_TABLE_12` translates to (all valid Table-12 codes). A code in +# this set — or translatable into it via the API enum map — is priced/ +# factored correctly; a code in NEITHER would silently default to mains gas. +_RECOGNISED_TABLE_12_FUEL_CODES: Final[frozenset[int]] = frozenset( + set(CO2_KG_PER_KWH) + | set(PRIMARY_ENERGY_FACTOR) + | set(UNIT_PRICE_P_PER_KWH) + | set(CO2_KG_PER_KWH_MONTHLY) + | set(PE_FACTOR_MONTHLY) + | set(API_FUEL_TO_TABLE_12.values()) +) + + +def _table_12_factor_fuel_code(fuel: int) -> int: + """`API_FUEL_TO_TABLE_12.get(fuel, fuel)` with a STRICT tail. + + Returns the Table-12 factor code for the cost / CO2 / PE lookups, with + behaviour identical to the prior silent passthrough for every recognised + input. The one difference: when the resolved code is neither translatable + via the API enum map NOR already a recognised Table-12/32 fuel code, it + raises `UnmappedSapCode` instead of passing the unknown code through to + the mains-gas default baked into `unit_price_p_per_kwh` / + `co2_factor_kg_per_kwh` / `primary_energy_factor` (the silent fuel- + collision class — cert 8536's community biomass mis-priced as grid + electricity was this pattern). Mirror of the strict-raise principle + ([[reference-unmapped-sap-code]]). + """ + code = API_FUEL_TO_TABLE_12.get(fuel, fuel) + if code in _RECOGNISED_TABLE_12_FUEL_CODES: + return code + raise UnmappedSapCode("table_12_factor_fuel", fuel) + + def _heat_network_factor_fuel_code( main: Optional[MainHeatingDetail], ) -> Optional[int]: @@ -2068,7 +2107,7 @@ def _heat_network_factor_fuel_code( fuel = _main_fuel_code(main) if fuel is None or not _is_heat_network_main(main): return fuel - return API_FUEL_TO_TABLE_12.get(fuel, fuel) + return _table_12_factor_fuel_code(fuel) def _fuel_cost_gbp_per_kwh( @@ -3627,7 +3666,7 @@ def _hot_water_co2_factor_kg_per_kwh( return _DEFAULT_CO2_KG_PER_KWH table_12_code = ( fuel if fuel in CO2_KG_PER_KWH - else API_FUEL_TO_TABLE_12.get(fuel, fuel) + else _table_12_factor_fuel_code(fuel) ) if tariff is not Tariff.STANDARD: return co2_factor_kg_per_kwh(table_12_code) @@ -3691,7 +3730,7 @@ def _hot_water_primary_factor( return _DEFAULT_PEF table_12_code = ( fuel if fuel in PRIMARY_ENERGY_FACTOR - else API_FUEL_TO_TABLE_12.get(fuel, fuel) + else _table_12_factor_fuel_code(fuel) ) if tariff is not Tariff.STANDARD: return primary_energy_factor(table_12_code) @@ -3725,7 +3764,7 @@ def _secondary_fuel_code(epc: EpcPropertyData) -> int: return _STANDARD_ELECTRICITY_FUEL_CODE if code in CO2_KG_PER_KWH: return code - return API_FUEL_TO_TABLE_12.get(code, code) + return _table_12_factor_fuel_code(code) def _secondary_heating_co2_factor_kg_per_kwh( @@ -7098,15 +7137,12 @@ def cert_to_inputs( secondary_fuel_monthly_kwh=energy_requirements_result.secondary_fuel_monthly_kwh, hot_water_monthly_kwh=hot_water_monthly_kwh_for_pv, main_fuel_code_table_12=( - API_FUEL_TO_TABLE_12.get(main_fuel, main_fuel) + _table_12_factor_fuel_code(main_fuel) if main_fuel is not None else None ), secondary_fuel_code_table_12=_secondary_fuel_code(epc), water_heating_fuel_code_table_12=( - API_FUEL_TO_TABLE_12.get( - epc.sap_heating.water_heating_fuel, - epc.sap_heating.water_heating_fuel, - ) + _table_12_factor_fuel_code(epc.sap_heating.water_heating_fuel) if epc.sap_heating.water_heating_fuel is not None else None ), # SAP 10.2 Appendix M1 §3a — exclude the low-rate portion of an diff --git a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py index 516a5e9e..584a36be 100644 --- a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py +++ b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py @@ -56,6 +56,7 @@ from domain.sap10_calculator.rdsap.cert_to_inputs import ( _heat_network_factor_fuel_code, # pyright: ignore[reportPrivateUsage] _heat_network_standing_charge_gbp, # pyright: ignore[reportPrivateUsage] _main_fuel_code, # pyright: ignore[reportPrivateUsage] + _table_12_factor_fuel_code, # pyright: ignore[reportPrivateUsage] _is_electric_main, # pyright: ignore[reportPrivateUsage] _is_heat_network_electric_main, # pyright: ignore[reportPrivateUsage] _is_electric_water, # pyright: ignore[reportPrivateUsage] @@ -1663,6 +1664,42 @@ def test_heat_network_unmapped_community_collision_fuel_raises() -> None: _heat_network_community_fuel_code(31, main) +def test_table_12_factor_fuel_unmapped_code_raises() -> None: + # Arrange — a fuel code that is neither translatable via + # API_FUEL_TO_TABLE_12 nor already a recognised Table-12/32 fuel code. + # 998 is a deliberately out-of-range sentinel for "novel/unknown fuel". + unmapped_fuel: Final[int] = 998 + + # Act / Assert — the gap surfaces loudly instead of passing the code + # through to the silent mains-gas default in unit_price_p_per_kwh / + # co2_factor_kg_per_kwh / primary_energy_factor (the cert-8536 + # community-collision class). Strict-raise per [[reference-unmapped-sap-code]]. + with pytest.raises(UnmappedSapCode) as excinfo: + _table_12_factor_fuel_code(unmapped_fuel) + + # Assert — the raised error names the field and the offending value. + assert excinfo.value.field == "table_12_factor_fuel" + assert excinfo.value.value == unmapped_fuel + + +def test_table_12_factor_fuel_recognised_codes_preserve_get_semantics() -> None: + # Arrange — recognised inputs must behave EXACTLY like the prior + # `API_FUEL_TO_TABLE_12.get(fuel, fuel)` passthrough: a gov-API enum is + # translated (26 mains-gas-not-community -> Table-12 code 1); a code + # already in the Table-12 numbering passes through unchanged (51 = heat- + # network mains gas). + api_enum_mains_gas: Final[int] = 26 + table_12_heat_network_gas: Final[int] = 51 + + # Act + translated: int = _table_12_factor_fuel_code(api_enum_mains_gas) + passthrough: int = _table_12_factor_fuel_code(table_12_heat_network_gas) + + # Assert — identical to the old .get(fuel, fuel) results, no behaviour drift. + assert translated == 1 + assert passthrough == 51 + + def test_responsiveness_solid_fuel_sap_code_160_returns_0p50_per_table_4a() -> None: # Arrange — SAP 10.2 Table 4a (PDF p.169, "Heating systems"): # From c8f075314248de2227befef6880786d5f02606c6 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Tue, 9 Jun 2026 10:06:54 +0000 Subject: [PATCH 25/87] =?UTF-8?q?docs:=20session-9=20cont.=20=E2=80=94=20s?= =?UTF-8?q?ilent-fallback=20audit=20+=20Tier-1=20strict-raise=20shipped?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Record the 4-agent audit, the shipped Tier-1 fuel-code strict-raise (7878a969), and the un-actioned Tier 2/3 candidates (glazing codes 4-12, window orientation, age-band swallows) for a future robustness slice. Co-Authored-By: Claude Opus 4.8 --- docs/HANDOVER_API_PROFILING.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/docs/HANDOVER_API_PROFILING.md b/docs/HANDOVER_API_PROFILING.md index 930bd71b..426db63c 100644 --- a/docs/HANDOVER_API_PROFILING.md +++ b/docs/HANDOVER_API_PROFILING.md @@ -414,6 +414,29 @@ heat:low 206 / hw:low 173 / hw:high 125 / balanced 94 — i.e. the residual is a fabric+HW tail, not one lever. The clean systematic wins (sheltered walls, MEV, fuel collisions) are harvested; remaining headway is per-cert worksheet grind or the 100-cert schema big-ticket. +### SESSION-9 (cont.) — silent-fallback audit + Tier-1 strict-raise SHIPPED, `7878a969` +User pivoted from accuracy-chasing to a robustness audit: "where do we map codes but NOT raise +on an unmapped code (silent fallback)?" Four parallel Explore agents swept the API mapper, +`cert_to_inputs`, the U-value/table lookups, and the worksheet dispatch. Findings triaged into +tiers (full list in the session report I wrote inline). **Tier 1 (shipped):** the fuel-type +helpers fed Table 12/32 cost/CO2/PE via a silent `API_FUEL_TO_TABLE_12.get(fuel, fuel)` +passthrough at 5 sites → an unmapped/colliding fuel hit the **mains-gas default** in +`unit_price_p_per_kwh`/`co2_factor`/`primary_energy_factor` (the cert-8536 −17-SAP class). New +`_table_12_factor_fuel_code` is byte-identical to `.get(fuel,fuel)` for every recognised input +and raises `UnmappedSapCode("table_12_factor_fuel", …)` only when the resolved code is in NO +table. **Verified 0/909 corpus raises; eval unchanged 54.9%/909/0** — pure future-proofing. +2 AAA tests, goldens + gate green, pyright net-zero (44=44). +**Tiers NOT actioned (user chose Tier-1 only — candidates for a future robustness slice):** +- Tier 2: `_api_glazing_transmission` ([mapper.py:2925]) maps only glazing codes [1,2,3,13,14]; + 4–12/15+ silently default U≈2.5 (the `reference_unmapped_api_code` "pending" gap). Window + orientation drop ([solar_gains.py:391]) silently drops a window from solar gains on an unmapped + orientation. Both would make currently-computed certs RAISE → need the codes MAPPED first, not + just a guard added (coverage tradeoff). +- Tier 3 (low impact, bands are schema-bounded A–M): `.get(band, DEFAULT)` swallows a non-None + unrecognised age band in `u_door` (→3.0), `u_basement_wall` (→0.7), `u_basement_floor` (→0.50), + `u_roof` (→0.4). The `age_band is None` branches above each are justified absent-data defaults; + only a typo/unknown band silently passing is the gap. + ## THE 100 unsupported_schema CERTS (deferred — bigger ticket) SAP-Schema-19.1.0 (and other pre-21). The user is planning a separate big piece: map old schemas → new + **predict missing fields from similar-looking properties** (needs an EPC-prediction From a04329770d35e5d87aaef7379cf09bf0dcd1a06f Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Tue, 9 Jun 2026 10:23:25 +0000 Subject: [PATCH 26/87] fix(glazing): map single/secondary/triple glazing per RdSAP 10 Table 24 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The API glazing-transmission table mapped only the double-glazing codes [1,2,3,13,14]; single (5/15), secondary (4/11/12) and triple (6/8/9/10) glazing codes returned None from _api_glazing_transmission, so the cascade silently routed them to the u_window all-None default U=2.5 instead of their RdSAP 10 Table 24 (spec p.50) value. Single glazing (U=4.8) was the worst: modelled at half its true heat loss → systematic over-rate (cert 0370-2933, 7 single-glazed windows, +17 SAP). Extended _API_GLAZING_TYPE_TO_TRANSMISSION + the gap-keyed override table with the Table 24 (U, g, frame-factor) rows for every RdSAP-21 glazing code (single 4.8/g0.85; secondary normal-E 2.9 / low-E 2.2 /g0.85; triple pre-2002 2.4/2.1/2.0 by gap, 2002-2022 2.0, all g0.68/0.72; known-data codes 7/8 alias their family default). 94 corpus certs carry an unmapped glazing code (code 5 = 79); they sat at 32% within-0.5 vs 54.9% baseline. Eval: within-0.5 54.90% -> 56.66% (net +16 certs: 22 in, 6 offsetting-error out), within-1.0 70.2 -> 71.9%, mean|err| 1.224 -> 1.203, 909 computed / 0 raises. Spec-applied uniformly per the determinism principle. 7 AAA tests, goldens + gate green, pyright net-zero (38=38). Co-Authored-By: Claude Opus 4.8 --- datatypes/epc/domain/mapper.py | 51 ++++++++++- .../domain/tests/test_from_rdsap_schema.py | 87 +++++++++++++++++++ 2 files changed, 135 insertions(+), 3 deletions(-) diff --git a/datatypes/epc/domain/mapper.py b/datatypes/epc/domain/mapper.py index 0c7a0e76..1f55d4e3 100644 --- a/datatypes/epc/domain/mapper.py +++ b/datatypes/epc/domain/mapper.py @@ -2865,9 +2865,10 @@ def _api_mechanical_ventilation_kind(mechanical_ventilation: object) -> Optional # "Fully double glazed" with a worksheet-resolved U=2.7. Per Table 24 # row 2 (DG pre-2002, gap 16+, PVC/wooden) the spec answer is U=2.7, # so GOV.UK API code 1 is a schema sibling of code 3 (both alias the -# "DG pre-2002 / unknown install date" row). The wider SAP10.2 -# glazing-type enum (4-12, 15+) is not yet mapped — incremental -# coverage as new fixtures surface them. +# "DG pre-2002 / unknown install date" row). The full RdSAP-21 +# glazing-type enum (single / secondary / triple, codes 4-12 + 15) is +# now mapped from Table 24 below — single-glazed windows previously fell +# through to the cascade default U=2.5 instead of their spec 4.8. # # Spec source: RdSAP 10 Table 24 "Window characteristics" page 49 — # DG pre-2002 spec U varies by gap (6mm=3.1, 12mm=2.8, 16+=2.7); the @@ -2889,6 +2890,34 @@ _API_GLAZING_TYPE_TO_TRANSMISSION: Dict[int, tuple[float, float, float]] = { # product family. Cert 0380 lodges code # 14 on all windows; worksheet uses U=1.4 # = post-curtain 1.3258.) + # + # SINGLE / SECONDARY / TRIPLE glazing — RdSAP 10 Table 24 (spec p.50). + # Previously unmapped (`_api_glazing_transmission` returned None) so the + # cascade silently routed e.g. a single-glazed window to the u_window + # all-None default 2.5 instead of its true 4.8 — over-rating single- + # glazed dwellings (cert 0370-2933, 7 single windows, +17 SAP). Codes + # per datatypes/epc/domain/epc_codes.csv `glazed_type` (RdSAP-Schema-21). + 4: (2.9, 0.85, 0.70), # Secondary glazing, unknown data → Table 24 + # Secondary "Normal emissivity" default (2.9). + 5: (4.8, 0.85, 0.70), # Single glazing — Table 24 "Single / Any + # period" (PVC/wooden 4.8, g 0.85). + 6: (2.1, 0.68, 0.70), # Triple glazed, unknown install date — Table + # 24 Triple pre-2002 12mm-gap default (2.1). + 7: (2.8, 0.76, 0.70), # Double glazed, known data — no measured U on + # the reduced-data path → double pre-2002 / + # unknown-date family default (2.8), as code 3. + 8: (2.1, 0.68, 0.70), # Triple glazed, known data → triple unknown- + # date family default (2.1), as code 6. + 9: (2.0, 0.72, 0.70), # Triple glazed, 2002-2022 — Table 24 "Double + # or triple, 2002+ (pre-2022), any gap" (2.0). + 10: (2.1, 0.68, 0.70), # Triple glazed, pre-2002 — Table 24 Triple + # pre-2002 12mm-gap default (2.1). + 11: (2.9, 0.85, 0.70), # Secondary glazing, normal emissivity — + # Table 24 Secondary "Normal emissivity" (2.9). + 12: (2.2, 0.85, 0.70), # Secondary glazing, low emissivity — Table 24 + # Secondary "Low emissivity" (2.2). + 15: (4.8, 0.85, 0.70), # Single glazing, known data → Single row + # (4.8) when no measured U is lodged, as code 5. } @@ -2908,6 +2937,22 @@ _API_GLAZING_TYPE_GAP_TO_TRANSMISSION: Dict[ (3, 6): (3.1, 0.76, 0.70), (3, 12): (2.8, 0.76, 0.70), (3, "16+"): (2.7, 0.76, 0.70), + # Double glazed, known data (code 7) — aliases the double pre-2002 / + # unknown-date Table 24 row (same as codes 1/3) when no measured U. + (7, 6): (3.1, 0.76, 0.70), + (7, 12): (2.8, 0.76, 0.70), + (7, "16+"): (2.7, 0.76, 0.70), + # Triple glazed pre-2002 / unknown / known-data (codes 6/8/10) — + # Table 24 Triple pre-2002 row varies by gap (6mm=2.4, 12mm=2.1, 16+=2.0). + (6, 6): (2.4, 0.68, 0.70), + (6, 12): (2.1, 0.68, 0.70), + (6, "16+"): (2.0, 0.68, 0.70), + (8, 6): (2.4, 0.68, 0.70), + (8, 12): (2.1, 0.68, 0.70), + (8, "16+"): (2.0, 0.68, 0.70), + (10, 6): (2.4, 0.68, 0.70), + (10, 12): (2.1, 0.68, 0.70), + (10, "16+"): (2.0, 0.68, 0.70), } diff --git a/datatypes/epc/domain/tests/test_from_rdsap_schema.py b/datatypes/epc/domain/tests/test_from_rdsap_schema.py index 04d9ff64..8cadcc6b 100644 --- a/datatypes/epc/domain/tests/test_from_rdsap_schema.py +++ b/datatypes/epc/domain/tests/test_from_rdsap_schema.py @@ -1101,3 +1101,90 @@ class TestApiRoofConstructionCode: # Assert assert result == "Pitched, sloping ceiling" + + +class TestApiGlazingTransmissionTable24: + """`_api_glazing_transmission` must resolve the SINGLE / SECONDARY / + TRIPLE glazing RdSAP-21 enum codes to their RdSAP 10 Table 24 (spec + page 50) (U, g, frame-factor) — not leave them unmapped (None), which + silently routed single-glazed windows to the cascade default U=2.5 + instead of their true 4.8, over-rating single-glazed dwellings (cert + 0370-2933, 7 single-glazed windows, +17 SAP).""" + + def test_single_glazing_code_5_is_table_24_u_4p8(self) -> None: + # Arrange — RdSAP 21 glazing_type 5 = "single glazing"; Table 24 + # row "Single / Any period" → U 4.8 (PVC/wooden), g 0.85. + from datatypes.epc.domain.mapper import _api_glazing_transmission # pyright: ignore[reportPrivateUsage] + + # Act + result = _api_glazing_transmission(5, None) + + # Assert + assert result == (4.8, 0.85, 0.70) + + def test_single_glazing_known_data_code_15_is_table_24_u_4p8(self) -> None: + # Arrange — code 15 = "single glazing, known data"; same Table 24 + # Single row when no measured U is lodged on the reduced-data path. + from datatypes.epc.domain.mapper import _api_glazing_transmission # pyright: ignore[reportPrivateUsage] + + # Act + result = _api_glazing_transmission(15, None) + + # Assert + assert result == (4.8, 0.85, 0.70) + + def test_secondary_glazing_normal_emissivity_code_11_is_u_2p9(self) -> None: + # Arrange — code 11 = "secondary glazing, normal emissivity"; + # Table 24 Secondary "Normal emissivity" row → U 2.9, g 0.85. + from datatypes.epc.domain.mapper import _api_glazing_transmission # pyright: ignore[reportPrivateUsage] + + # Act + result = _api_glazing_transmission(11, None) + + # Assert + assert result == (2.9, 0.85, 0.70) + + def test_secondary_glazing_low_emissivity_code_12_is_u_2p2(self) -> None: + # Arrange — code 12 = "secondary glazing, low emissivity"; Table 24 + # Secondary "Low emissivity" row → U 2.2, g 0.85. + from datatypes.epc.domain.mapper import _api_glazing_transmission # pyright: ignore[reportPrivateUsage] + + # Act + result = _api_glazing_transmission(12, None) + + # Assert + assert result == (2.2, 0.85, 0.70) + + def test_triple_glazing_2002_to_2022_code_9_is_u_2p0(self) -> None: + # Arrange — code 9 = "triple glazing, installed 2002-2022"; Table 24 + # "Double or triple, 2002+ (pre-2022), any gap" → U 2.0, g 0.72. + from datatypes.epc.domain.mapper import _api_glazing_transmission # pyright: ignore[reportPrivateUsage] + + # Act + result = _api_glazing_transmission(9, None) + + # Assert + assert result == (2.0, 0.72, 0.70) + + def test_triple_glazing_pre_2002_code_10_default_gap_is_u_2p1(self) -> None: + # Arrange — code 10 = "triple glazing, pre-2002"; Table 24 Triple + # pre-2002 12mm-gap default → U 2.1, g 0.68. + from datatypes.epc.domain.mapper import _api_glazing_transmission # pyright: ignore[reportPrivateUsage] + + # Act + result = _api_glazing_transmission(10, None) + + # Assert + assert result == (2.1, 0.68, 0.70) + + def test_double_glazing_2002_plus_code_2_unchanged(self) -> None: + # Arrange — regression guard: the already-mapped double-glazing + # 2002+ entry (U 2.0, g 0.72) is untouched by the single/secondary/ + # triple extension. + from datatypes.epc.domain.mapper import _api_glazing_transmission # pyright: ignore[reportPrivateUsage] + + # Act + result = _api_glazing_transmission(2, None) + + # Assert + assert result == (2.0, 0.72, 0.70) From 8e1e746a3e95e6db015e6178e1f676b039e3db7e Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Tue, 9 Jun 2026 10:24:21 +0000 Subject: [PATCH 27/87] =?UTF-8?q?docs:=20session-9=20cont.2=20=E2=80=94=20?= =?UTF-8?q?glazing=20Table-24=20win=20(54.9->56.7%)=20+=20ranked=20known?= =?UTF-8?q?=20bugs?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 4.8 --- docs/HANDOVER_API_PROFILING.md | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/docs/HANDOVER_API_PROFILING.md b/docs/HANDOVER_API_PROFILING.md index 426db63c..7edf26fa 100644 --- a/docs/HANDOVER_API_PROFILING.md +++ b/docs/HANDOVER_API_PROFILING.md @@ -437,6 +437,36 @@ table. **Verified 0/909 corpus raises; eval unchanged 54.9%/909/0** — pure fut `u_roof` (→0.4). The `age_band is None` branches above each are justified absent-data defaults; only a typo/unknown band silently passing is the gap. +### SESSION-9 (cont. 2) — glazing single/secondary/triple per RdSAP 10 Table 24, `a0432977` (BIGGEST WIN) +Chasing Tier-2 turned up the session's biggest lever. `_API_GLAZING_TYPE_TO_TRANSMISSION` mapped +only the DOUBLE-glazing codes [1,2,3,13,14]; single (5/15), secondary (4/11/12), triple +(6/8/9/10) returned None → the cascade routed them to the `u_window` all-None default **U=2.5** +instead of their Table 24 value. **Single glazing (U=4.8) modelled at half its heat loss** was +the killer — 79 corpus certs carry code 5; the 94 certs with any unmapped glazing code sat at +**32% within-0.5 vs 54.9% baseline**. Extended the transmission + gap-override tables with the +RdSAP 10 Table 24 (spec p.50) (U, g, FF) rows for every RdSAP-21 glazing code. **Eval 54.90% → +56.66% within-0.5** (net +16: 22 in, 6 offsetting-error out), within-1.0 70.2→71.9, mean|err| +1.224→1.203, 909/0. 7 AAA tests, goldens + gate green, pyright net-zero. **Method that found it: +profile by `sap_windows[].glazing_type`, split known vs unmapped codes, decode against +`epc_codes.csv` `glazed_type`.** + +### OPEN — KNOWN bugs still on the board (ranked) +1. **Glazing SOLAR/LIGHT g mis-keyed** (the natural follow-on): `_G_PERPENDICULAR_BY_GLAZING_TYPE` + (solar_gains.py) + `_G_LIGHT_BY_GLAZING_CODE` (internal_gains.py) are keyed on the SAP-10.2 + Table-6b cascade ordering (1=single…) but `_api_cascade_glazing_type` only translates {1:2} — + every OTHER API code passes through UNTRANSLATED, so API code 5 (single) reads the cascade-5 + slot (secondary/DG-low-E g) for solar+daylight gains. Smaller than the U effect (gains are + second-order) but a real systematic mis-map. Fix = complete `_api_cascade_glazing_type` → + Table-6b, OR feed the transmission-tuple g instead. Verify magnitude first. +2. **Window orientation silently dropped** (solar_gains.py:391 `if orientation is None: continue`) + — an unmapped orientation code loses the whole window's solar gain. HIGH-confidence data-loss. +3. **Tier-2 glazing strict-raise** — now that codes 1-15 are mapped, make `_api_glazing_transmission` + raise `UnmappedApiCode` on a present-but-unmapped (>15) code (zero current regression). +4. Tier-3 age-band swallows (low impact). MVHR (mech_vent=4) deferred. 21.0.0 builder missing the + 8/8b/8c ventilation + (now) glazing edits — extend if a 21.0.0 cert surfaces. +5. Per-cert: cert 0370-2933 still +15 after glazing (7 single windows but the over-rate is + non-window — separate fabric/heating cause). The 100 unsupported-schema certs (big ticket). + ## THE 100 unsupported_schema CERTS (deferred — bigger ticket) SAP-Schema-19.1.0 (and other pre-21). The user is planning a separate big piece: map old schemas → new + **predict missing fields from similar-looking properties** (needs an EPC-prediction From 49fb6c1b8e09f8792977302170f5de03f26c748c Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Tue, 9 Jun 2026 10:42:16 +0000 Subject: [PATCH 28/87] fix(glazing): remap divergent RdSAP-21 glazing codes 4/5 to cascade g slots MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The g-value tables (_G_PERPENDICULAR_BY_GLAZING_TYPE solar g⊥, _G_LIGHT_BY_GLAZING_CODE daylight g_L) are keyed on the SAP 10.2 Table 6b cascade enum, but _api_cascade_glazing_type only translated code 1. Codes 4 and 5 sit in the 1-6 range where RdSAP-21 and the cascade enum disagree (RdSAP-21 4=secondary/5=single vs cascade 4=double-low-E/5=secondary), so an API single-glazed window read the cascade-5 secondary g (0.76/0.80) instead of single (0.85/0.90), and a secondary window read cascade-4 double-low-E (0.63). Added the {4:5, 5:1} remap entries the existing design comment already anticipated ("only divergent codes need a remap"). Correctness fix: solar/daylight gains are second-order, so eval is unchanged (56.66% within-0.5, 0 certs flip) — the dominant single-glazing error was the U-value, closed in a0432977's Table 24 transmission map. This closes the keying inconsistency to prevent future drift. 4 AAA tests, goldens + gate green, pyright net-zero (38=38). Co-Authored-By: Claude Opus 4.8 --- datatypes/epc/domain/mapper.py | 10 ++++ .../domain/tests/test_from_rdsap_schema.py | 51 +++++++++++++++++++ 2 files changed, 61 insertions(+) diff --git a/datatypes/epc/domain/mapper.py b/datatypes/epc/domain/mapper.py index 1f55d4e3..72e10f44 100644 --- a/datatypes/epc/domain/mapper.py +++ b/datatypes/epc/domain/mapper.py @@ -2988,6 +2988,16 @@ def _api_glazing_transmission( # remap — incremental coverage as new fixtures surface them. _API_TO_SAP10_CASCADE_GLAZING_CODE: Dict[int, int] = { 1: 2, # RdSAP 21 DG pre-2002 → cascade DG (g_L=0.80, not single 0.90) + # RdSAP-21 codes 4 and 5 DIVERGE from the cascade enum in the 1-6 range + # (cascade 4 = double low-E soft-coat, 5 = secondary). Without these the + # g-tables read the wrong slot: a single-glazed window (RdSAP-21 5) took + # the cascade-5 secondary g (g⊥ 0.76 / g_L 0.80) for solar + daylight + # gains instead of single (0.85 / 0.90), and a secondary-glazed window + # (RdSAP-21 4) took cascade-4 double-low-E (0.63). Correctness fix + # (gains are second-order — negligible SAP impact; the dominant single- + # glazing error was the U-value, closed in the Table 24 transmission map). + 4: 5, # RdSAP 21 secondary glazing → cascade secondary slot (0.76/0.80) + 5: 1, # RdSAP 21 single glazing → cascade single slot (0.85/0.90) } diff --git a/datatypes/epc/domain/tests/test_from_rdsap_schema.py b/datatypes/epc/domain/tests/test_from_rdsap_schema.py index 8cadcc6b..b882a4c0 100644 --- a/datatypes/epc/domain/tests/test_from_rdsap_schema.py +++ b/datatypes/epc/domain/tests/test_from_rdsap_schema.py @@ -1188,3 +1188,54 @@ class TestApiGlazingTransmissionTable24: # Assert assert result == (2.0, 0.72, 0.70) + + +class TestApiCascadeGlazingCodeDivergentRemap: + """`_api_cascade_glazing_type` must translate the RdSAP-21 glazing codes + that DIVERGE from the SAP 10.2 Table 6b cascade enum the g-value tables + (`_G_PERPENDICULAR_BY_GLAZING_TYPE` / `_G_LIGHT_BY_GLAZING_CODE`) are + keyed on. Only code 1 was ever remapped; codes 4 and 5 sit in the 1-6 + range where the two enums disagree — RdSAP-21 4=secondary / 5=single, + cascade 4=double-low-E / 5=secondary — so an API single-glazed window + (5) read the cascade-5 (secondary) g slot for solar + daylight gains.""" + + def test_single_glazing_code_5_remaps_to_cascade_single_slot_1(self) -> None: + # Arrange — RdSAP-21 code 5 = single glazing; cascade single slot + # is 1 (g⊥ 0.85, g_L 0.90), not cascade 5 (secondary, 0.76/0.80). + from datatypes.epc.domain.mapper import _api_cascade_glazing_type # pyright: ignore[reportPrivateUsage] + + # Act + result = _api_cascade_glazing_type(5) + + # Assert + assert result == 1 + + def test_secondary_glazing_code_4_remaps_to_cascade_secondary_slot_5(self) -> None: + # Arrange — RdSAP-21 code 4 = secondary glazing; cascade secondary + # slot is 5 (g⊥ 0.76, g_L 0.80), not cascade 4 (double low-E 0.63). + from datatypes.epc.domain.mapper import _api_cascade_glazing_type # pyright: ignore[reportPrivateUsage] + + # Act + result = _api_cascade_glazing_type(4) + + # Assert + assert result == 5 + + def test_double_pre_2002_code_1_remap_unchanged(self) -> None: + # Arrange — regression guard: the existing code-1 remap (→2) stands. + from datatypes.epc.domain.mapper import _api_cascade_glazing_type # pyright: ignore[reportPrivateUsage] + + # Act + result = _api_cascade_glazing_type(1) + + # Assert + assert result == 2 + + def test_rdsap21_native_codes_pass_through(self) -> None: + # Arrange — codes 9-15 already coincide with the g-table's RdSAP-21 + # extension slots, so they must pass through untranslated. + from datatypes.epc.domain.mapper import _api_cascade_glazing_type # pyright: ignore[reportPrivateUsage] + + # Act / Assert + assert _api_cascade_glazing_type(14) == 14 + assert _api_cascade_glazing_type(9) == 9 From 32bbb92be3f414b08a6fd52e47296bd80431fe1d Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Tue, 9 Jun 2026 11:46:26 +0000 Subject: [PATCH 29/87] =?UTF-8?q?docs:=20session-9=20cont.3=20=E2=80=94=20?= =?UTF-8?q?glazing=20g=20remap=20+=20post-glazing=20re-profile=20(solid=20?= =?UTF-8?q?brick=20verified=20spec-faithful)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 4.8 --- docs/HANDOVER_API_PROFILING.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/docs/HANDOVER_API_PROFILING.md b/docs/HANDOVER_API_PROFILING.md index 7edf26fa..fd7e0195 100644 --- a/docs/HANDOVER_API_PROFILING.md +++ b/docs/HANDOVER_API_PROFILING.md @@ -467,6 +467,27 @@ profile by `sap_windows[].glazing_type`, split known vs unmapped codes, decode a 5. Per-cert: cert 0370-2933 still +15 after glazing (7 single windows but the over-rate is non-window — separate fabric/heating cause). The 100 unsupported-schema certs (big ticket). +### SESSION-9 (cont. 3) — glazing g remap (correctness, 0 impact) + post-glazing re-profile +- **Glazing g remap `49fb6c1b`:** completed the design's divergent-code remap — + `_API_TO_SAP10_CASCADE_GLAZING_CODE` gained {4:5, 5:1} so API single (5) / secondary (4) read + the right cascade g-slot (single 0.85/0.90, not secondary 0.76/0.80). CORRECTNESS ONLY: solar/ + daylight gains are second-order → 0 certs flip, eval unchanged 56.66%. The single-glazing + *U-value* (a0432977) was the whole accuracy effect. +- **Re-profiled at 56.7%.** Remaining biased buckets ruled out as deproven / spec-faithful / + tail-driven (verified, do NOT re-chase): + - `wall_construction=3` solid brick gas (n=197, signed −0.52, survives gas-split): **spec-faithful, + NOT a bug.** `u_wall` applies RdSAP §5.7 Table 13 thickness (≤200→2.5, 200-280→1.7, 280-420→1.4, + >420→1.1); the API plumbs `wall_thickness` and the mapper passes it; wit=4 "insulated (assumed)" + correctly takes the as-built U (no §5.8 reduction). Direction is also wrong for a thickness gap + (using thickness LOWERS U → over-rate). Residual = old solid-brick houses outperform as-built. + - `main_control=2113` (n=26, −0.76 but 77% within-0.5): tail-driven by messy multi-part certs + (0700/9092), not uniform. `wall_construction=5` (timber, n=35): tail-driven (7921 −23, 0370 +15). + - worst remaining under-rater **7921 (−23)**: roof code-8 sloping-ceiling "Pitched, insulated" + thick=None → uninsulated 2.3 (roof 241 W/K). On the DEPROVEN list (= data-fidelity, we ≡ Elmhurst + at uninsulated). Don't chase. + - NOT yet run down (genuine open candidates): `main_heat_cat=4` heat pumps (n=20, −0.78), + `water_fuel=20` community HW over-rate (n=40, +0.54, distinct direction — all sapcode 301). + ## THE 100 unsupported_schema CERTS (deferred — bigger ticket) SAP-Schema-19.1.0 (and other pre-21). The user is planning a separate big piece: map old schemas → new + **predict missing fields from similar-looking properties** (needs an EPC-prediction From a7990edb8c14d2655aea2ef82f3df50409b883a3 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Tue, 9 Jun 2026 13:21:13 +0000 Subject: [PATCH 30/87] robustness: strict-raise on unmapped glazing + heating/HW efficiency codes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Forcing-function guards so a lodged-but-unmapped code surfaces loudly instead of silently taking a wrong-but-plausible default (the class that hid single glazing as U=2.5 until this session). Four silent fallbacks converted to raise on PRESENT-but-unmapped codes, while keeping the legitimate ABSENT (None) defaults: - _api_glazing_transmission: unmapped glazing_type -> UnmappedApiCode (was None -> u_window all-None default 2.5). - _api_cascade_glazing_type: unmapped glazing_type -> UnmappedApiCode (was pass-through -> wrong g-value slot). - seasonal_efficiency: a lodged code/category resolving in neither _SPACE_EFF_BY_CODE nor the category/room-heater fallbacks -> UnmappedSapCode (was blind 0.80 gas-boiler default, which 'catastrophically misrates heat pumps and storage' per the table comment). Data-free calls keep 0.80. - water_heating_efficiency: WHC in no SAP 10.2 Table 4a HW row -> UnmappedSapCode (was blind 0.78). Absent code keeps 0.78. Zero current-corpus impact (909 computed / 0 raises, 56.66% within-0.5 unchanged) — the code/efficiency tables are complete for today's data, so these are guards for the ongoing audit + future data refreshes. Verified the WHC table already covers 908 (multi-point gas) and 950 (HW heat network), so those are NOT unmapped-code bugs. 8 AAA tests, goldens + gate green, pyright net-zero. Co-Authored-By: Claude Opus 4.8 --- datatypes/epc/domain/mapper.py | 23 +++++++--- domain/sap10_ml/sap_efficiencies.py | 17 ++++++- .../sap10_ml/tests/test_sap_efficiencies.py | 45 +++++++++++++++++++ 3 files changed, 77 insertions(+), 8 deletions(-) diff --git a/datatypes/epc/domain/mapper.py b/datatypes/epc/domain/mapper.py index 72e10f44..9c2587b9 100644 --- a/datatypes/epc/domain/mapper.py +++ b/datatypes/epc/domain/mapper.py @@ -2959,15 +2959,20 @@ _API_GLAZING_TYPE_GAP_TO_TRANSMISSION: Dict[ def _api_glazing_transmission( glazing_type: Optional[int], glazing_gap: object, ) -> Optional[tuple[float, float, float]]: - """Resolve (U, g, frame_factor) for an API window. Per-gap override - takes precedence over the type-only default; returns None when the - glazing_type isn't yet in the lookup.""" + """Resolve (U, g, frame_factor) for an API window from RdSAP 10 Table 24. + Per-gap override takes precedence over the type-only default. Returns None + only when `glazing_type` is absent (None → cascade default). A glazing_type + PRESENT but unmapped raises `UnmappedApiCode` rather than silently routing + the window to the u_window all-None default U=2.5 — the forcing function + that surfaced single-glazing (code 5 → 4.8) instead of letting it hide.""" if glazing_type is None: return None gap_key = (glazing_type, glazing_gap) if gap_key in _API_GLAZING_TYPE_GAP_TO_TRANSMISSION: return _API_GLAZING_TYPE_GAP_TO_TRANSMISSION[gap_key] - return _API_GLAZING_TYPE_TO_TRANSMISSION.get(glazing_type) + if glazing_type in _API_GLAZING_TYPE_TO_TRANSMISSION: + return _API_GLAZING_TYPE_TO_TRANSMISSION[glazing_type] + raise UnmappedApiCode("glazing_type", glazing_type) # GOV.UK RdSAP 21 `glazing_type` integer → SAP 10.2 Table 6b cascade @@ -3003,8 +3008,14 @@ _API_TO_SAP10_CASCADE_GLAZING_CODE: Dict[int, int] = { def _api_cascade_glazing_type(api_glazing_type: int) -> int: """Canonicalise an API-lodged RdSAP 21 glazing-type code to the SAP - 10.2 Table 6b cascade enum that `_G_LIGHT_BY_GLAZING_CODE` reads. - Pass-through for codes already coincident with the cascade table.""" + 10.2 Table 6b cascade enum the g-value tables (`_G_LIGHT_BY_GLAZING_CODE` + / `_G_PERPENDICULAR_BY_GLAZING_TYPE`) read. Divergent codes are remapped; + coincident codes pass through. An unknown glazing code raises + `UnmappedApiCode` (mirrors `_api_glazing_transmission`) so a new code + surfaces rather than silently reading a wrong g-value slot. The known + set is the RdSAP-21 glazing enum the transmission table already covers.""" + if api_glazing_type not in _API_GLAZING_TYPE_TO_TRANSMISSION: + raise UnmappedApiCode("glazing_type (cascade g)", api_glazing_type) return _API_TO_SAP10_CASCADE_GLAZING_CODE.get(api_glazing_type, api_glazing_type) diff --git a/domain/sap10_ml/sap_efficiencies.py b/domain/sap10_ml/sap_efficiencies.py index d62db300..59ce819c 100644 --- a/domain/sap10_ml/sap_efficiencies.py +++ b/domain/sap10_ml/sap_efficiencies.py @@ -16,6 +16,8 @@ from __future__ import annotations from typing import Final, Optional +from domain.sap10_calculator.exceptions import UnmappedSapCode + # --------------------------------------------------------------------------- # Table 4a + Table 4b — space-heating seasonal efficiency by code @@ -148,6 +150,15 @@ def seasonal_efficiency( eff = _CATEGORY_FALLBACK_EFF.get(main_heating_category) if eff is not None: return eff + # Reached when neither code nor category resolved. If SOMETHING was + # lodged but unmapped, surface it (the blind 0.80 gas-boiler default + # catastrophically misrates heat pumps / storage). Only a genuinely + # data-free call (both absent) keeps the 0.80 "no data" default. + if sap_main_heating_code is not None or main_heating_category is not None: + raise UnmappedSapCode( + "seasonal_efficiency (code/category)", + (sap_main_heating_code, main_heating_category), + ) return 0.80 @@ -159,13 +170,15 @@ def water_heating_efficiency( Codes 901/914 ("from main / from second main") inherit the main code's seasonal efficiency. Code 902 ("from secondary") falls back to typical. - Unknown -> 0.78 (gas-combi typical). + Absent (None) -> 0.78 (gas-combi typical). A code PRESENT but in no SAP + 10.2 Table 4a HW row raises `UnmappedSapCode` rather than silently taking + the 0.78 default — the forcing function that surfaces a new HW code. """ if water_heating_code is None: return 0.78 eff = _WATER_EFF_BY_CODE.get(water_heating_code) if eff is None: - return 0.78 + raise UnmappedSapCode("water_heating_code (efficiency)", water_heating_code) if eff == 0.0: # sentinel for "inherit" return seasonal_efficiency(main_heating_code) return eff diff --git a/domain/sap10_ml/tests/test_sap_efficiencies.py b/domain/sap10_ml/tests/test_sap_efficiencies.py index 76f513b9..435fae6c 100644 --- a/domain/sap10_ml/tests/test_sap_efficiencies.py +++ b/domain/sap10_ml/tests/test_sap_efficiencies.py @@ -275,3 +275,48 @@ def test_fuel_unit_price_recognises_api_code_29_electricity_not_community() -> N def test_fuel_unit_price_recognises_api_code_27_lpg_not_community() -> None: # Arrange / Act — gov API code 27 = LPG not community -> bulk LPG 7.60 p/kWh. assert fuel_unit_price_p_per_kwh(fuel_code=27) == pytest.approx(7.60, abs=0.01) + + +# ----- Robustness: strict-raise on a lodged-but-unmapped code ----- + + +def test_seasonal_efficiency_raises_on_present_unmapped_code_and_category() -> None: + # Arrange — a main-heating SAP code AND category that resolve in NEITHER + # _SPACE_EFF_BY_CODE nor the category/room-heater fallbacks. The blind + # 0.80 gas-boiler default "catastrophically misrates heat pumps and + # storage" (per the table comment), so a lodged-but-unmapped pairing must + # surface as UnmappedSapCode, not silently rate as a gas boiler. + from domain.sap10_calculator.exceptions import UnmappedSapCode + + # Act / Assert + with pytest.raises(UnmappedSapCode): + seasonal_efficiency(sap_main_heating_code=8888, main_heating_category=99) + + +def test_seasonal_efficiency_no_data_still_defaults_to_gas_boiler() -> None: + # Arrange — when NOTHING is lodged (code and category both absent), the + # 0.80 typical-gas default is the correct "no data" fallback, not a raise. + # Act + result = seasonal_efficiency(sap_main_heating_code=None, main_heating_category=None) + + # Assert + assert result == 0.80 + + +def test_water_heating_efficiency_raises_on_present_unmapped_code() -> None: + # Arrange — a water-heating code that exists in NO SAP 10.2 Table 4a HW + # row must surface rather than silently take the 0.78 gas-combi default. + from domain.sap10_calculator.exceptions import UnmappedSapCode + + # Act / Assert + with pytest.raises(UnmappedSapCode): + water_heating_efficiency(water_heating_code=9999, main_heating_code=None) + + +def test_water_heating_efficiency_absent_code_still_defaults() -> None: + # Arrange — no water-heating code lodged (None) keeps the typical default. + # Act + result = water_heating_efficiency(water_heating_code=None, main_heating_code=None) + + # Assert + assert result == 0.78 From 872bc585f7bfef9daed4ef7d831990ab08589d53 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Tue, 9 Jun 2026 13:40:17 +0000 Subject: [PATCH 31/87] fix(hot-water): apply Table 12c distribution loss to HW-only heat networks (whc 950/951/952) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The heat-network HW distribution-loss override fired only when the MAIN was a heat network AND whc inherited from main ({901,902,914}). Water-heating-only heat networks (SAP 10.2 Table 4a HW codes 950 boilers / 951 CHP / 952 heat pump) were missed entirely: their Table 4a plant efficiency applied with NO distribution loss, so the HW fuel was under-counted by the Table 12c DLF (1.33-1.48x) → under-cost → over-rate. RdSAP 10 §10 (spec p.36): a water-heating-only heat network is calculated 'for plant efficiency, distribution loss and pumping energy - see Table 12c'. Added a whc-gated branch (independent of the main) applying water_eff = plant_eff / DLF — the per-kWh-generated cost model (q_generated = q_useful x DLF). Fires on the WHC alone so a HW-only heat network with a non-network main (cert 9093, whc 950 + warm-air main 502) is covered. The 3 corpus whc=950 certs all improve in |err|: 2153 +2.62->-0.48 (now within 0.5), 7220 +1.27->-0.97, 9093 +6.04->+3.60 (residual is its warm-air main, a separate cause). within-0.5 56.66->56.79%, within-1.0 71.9->72.2%, mean|err| down; only those 3 certs change. New AAA test pins the DLF scaling fires on the WHC independent of the main. Goldens + gate green, pyright net-zero. Co-Authored-By: Claude Opus 4.8 --- .../sap10_calculator/rdsap/cert_to_inputs.py | 17 +++++++ .../rdsap/test_cert_to_inputs.py | 50 +++++++++++++++++++ 2 files changed, 67 insertions(+) diff --git a/domain/sap10_calculator/rdsap/cert_to_inputs.py b/domain/sap10_calculator/rdsap/cert_to_inputs.py index 4c0941ac..395626d8 100644 --- a/domain/sap10_calculator/rdsap/cert_to_inputs.py +++ b/domain/sap10_calculator/rdsap/cert_to_inputs.py @@ -3092,6 +3092,15 @@ def _pumps_fans_fuel_cost_gbp_per_kwh( # the SAME cascade the main heating uses, including the main_heating_ # category fallback (e.g. heat pumps return 2.30 via category 4). _WATER_INHERIT_FROM_MAIN_CODES: Final[frozenset[int]] = frozenset({901, 902, 914}) +# Hot-water-only heat-network codes — SAP 10.2 Table 4a HW section (PDF +# p.167): 950 boilers / 951 CHP / 952 heat pump. The DHW is supplied by a +# heat network independent of the space-heating system, so RdSAP 10 §10 +# (spec p.36 "water heating only ... for plant efficiency, distribution loss +# and pumping energy — see Table 12c") requires the Table 12c distribution +# loss factor applied on top of the Table 4a plant efficiency. Distinct from +# the inherit-from-main codes: the DLF fires on the WHC regardless of whether +# the *main* is a heat network (e.g. cert 9093, whc 950 + warm-air main 502). +_WATER_HEAT_NETWORK_ONLY_CODES: Final[frozenset[int]] = frozenset({950, 951, 952}) # Water-heating code 901 = "From main heating system" — used by the # SAP 10.2 Appendix D §D2.1 (2) Equation D1 gate, which only applies # when "the boiler provides both space and water heating". @@ -6776,6 +6785,14 @@ def cert_to_inputs( # space heating so the delivered HW kWh reflects q_useful × DLF # = q_generated, matching the per-kWh-generated unit price. water_eff = 1.0 / _heat_network_dlf(primary_age) + elif epc.sap_heating.water_heating_code in _WATER_HEAT_NETWORK_ONLY_CODES: + # HW-only heat network (whc 950/951/952): the Table 4a plant + # efficiency is already in `water_eff`; apply the Table 12c + # distribution loss on top per RdSAP 10 §10 (spec p.36 "water + # heating only ... distribution loss"). q_generated = q_useful × + # DLF, so delivered-per-fuel efficiency = plant_eff / DLF. Fires + # on the WHC alone — the HW network is independent of the main. + water_eff = water_eff / _heat_network_dlf(primary_age) is_instantaneous = epc.sap_heating.water_heating_code in _INSTANTANEOUS_WATER_CODES # §9a Table 11 secondary fraction — pulled forward of §4 so the # post-§8 Equation D1 cascade can derive Q_space = (98c)m × (204) diff --git a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py index 584a36be..a0a777d0 100644 --- a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py +++ b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py @@ -459,6 +459,56 @@ def test_heat_network_main_with_hw_from_main_dlf_scales_hot_water_kwh() -> None: assert hn_hw / gas_hw == pytest.approx(1.41 * 0.80 / 1.0, abs=0.02) +def test_hot_water_only_heat_network_whc_950_applies_table12c_dlf() -> None: + # Arrange — water_heating_code 950 = "hot-water-only heat network + # (boilers)" (SAP 10.2 Table 4a HW section, plant eff 0.80). RdSAP 10 + # §10 (spec p.36) requires the Table 12c distribution loss applied on + # top of the plant efficiency for water-heating-only heat networks, so + # the delivered HW fuel = q_useful × DLF / 0.80. The DLF must fire on + # the WHC ALONE — independent of the main, which here is an ordinary + # gas boiler (NOT a heat network), mirroring cert 9093 (whc 950 + a + # warm-air main). Compare against a non-heat-network baseline at the + # same 0.80 water efficiency (whc 901 from the same gas main): the 950 + # HW fuel must exceed it by exactly the DLF (age E → 1.41). + part = make_building_part(construction_age_band="E") + gas_main = MainHeatingDetail( + has_fghrs=False, + main_fuel_type=26, + heat_emitter_type=1, + emitter_temperature=1, + main_heating_control=2106, + main_heating_category=2, + ) + hw_network_epc = make_minimal_sap10_epc( + total_floor_area_m2=_TYPICAL_TFA_M2, + habitable_rooms_count=4, + country_code="ENG", + sap_building_parts=[part], + sap_heating=make_sap_heating( + main_heating_details=[gas_main], + water_heating_code=950, # hot-water-only heat network + ), + ) + # Baseline: HW from the same gas main (eff 0.80), no heat network → no DLF. + baseline_epc = make_minimal_sap10_epc( + total_floor_area_m2=_TYPICAL_TFA_M2, + habitable_rooms_count=4, + country_code="ENG", + sap_building_parts=[part], + sap_heating=make_sap_heating( + main_heating_details=[gas_main], + water_heating_code=901, + ), + ) + + # Act + hw_network: float = cert_to_inputs(hw_network_epc).hot_water_kwh_per_yr + baseline: float = cert_to_inputs(baseline_epc).hot_water_kwh_per_yr + + # Assert — the HW-only-heat-network fuel is scaled up by the age-E DLF. + assert abs(hw_network / baseline - 1.41) <= 0.02 + + def test_gas_boiler_main_efficiency_unchanged_by_dlf_override() -> None: # Arrange — regression check: the DLF override only fires for heat- # network main heating. A standard gas boiler (cat=2, code=102) must From 590cb97ef6f1e601a495f15c297f94024bb5bcfb Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Tue, 9 Jun 2026 14:54:08 +0000 Subject: [PATCH 32/87] docs: session-9 close-out + session-10 handover (summary-report-based audit) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Session 9 ran five independent data-driven audits (profiler, dropped-field scan, CO2/PE reconciliation, cross-provider LIG parity, HW-demand reconciliation) — all converged on diffuse remaining gap — and shipped glazing Table-24 (+16 certs) + HW-only heat-network DLF, taking 54.90% -> 56.8% within-0.5. The data-driven seam is exhausted; session 10 switches to worksheet-level ground truth via the summary-report-based per-cert audit. New agent prompt at HANDOVER_SUMMARY_AUDIT.md with method, starter candidate certs, ruled-out list, and conventions. Co-Authored-By: Claude Opus 4.8 --- docs/HANDOVER_API_PROFILING.md | 6 ++ docs/HANDOVER_SUMMARY_AUDIT.md | 145 +++++++++++++++++++++++++++++++++ 2 files changed, 151 insertions(+) create mode 100644 docs/HANDOVER_SUMMARY_AUDIT.md diff --git a/docs/HANDOVER_API_PROFILING.md b/docs/HANDOVER_API_PROFILING.md index fd7e0195..0b37765b 100644 --- a/docs/HANDOVER_API_PROFILING.md +++ b/docs/HANDOVER_API_PROFILING.md @@ -1,5 +1,11 @@ # Handover — API SAP accuracy (session 3): raises cleared, now profile-driven +> **➡️ SESSION 10 STARTS HERE: `docs/HANDOVER_SUMMARY_AUDIT.md`.** HEAD `872bc585`, **56.8% +> within-0.5** (909 computed / 0 raises). Session 9 ran FIVE data-driven audit angles — all +> converged on "remaining gap is diffuse" — and shipped the glazing Table-24 win (+16 certs) + +> HW-only heat-network DLF. The data-driven seam is mined out; **session 10 switches to the +> summary-report-based per-cert worksheet audit.** Read that doc first. + **Branch:** `feature/per-cert-mapper-validation` (long-lived working branch — **NEVER PR to main**; the user pushes/PRs when ready). **HEAD `a8e5563a`+** (the profiler commit), local-only ahead of origin. diff --git a/docs/HANDOVER_SUMMARY_AUDIT.md b/docs/HANDOVER_SUMMARY_AUDIT.md new file mode 100644 index 00000000..20d44be1 --- /dev/null +++ b/docs/HANDOVER_SUMMARY_AUDIT.md @@ -0,0 +1,145 @@ +# Handover — API SAP accuracy, SESSION 10: summary-report-based per-cert audit + +You're continuing API→SAP accuracy work on branch **`feature/per-cert-mapper-validation`** in +`/workspaces/model`, **HEAD `872bc585`**. This is a **long-lived working branch — NEVER PR to +main**; the user pushes/PRs when ready. 31 commits ahead of `origin`, unpushed. + +## THE GOAL (measurable, unchanged) +100% of API records with a lodged SAP compute **within 0.5 SAP** of the API's +`energy_rating_current`. Headline gauge: +``` +PYTHONPATH=/workspaces/model python scripts/eval_api_sap_accuracy.py +``` +**Current: 56.8% within-0.5** (within-1.0 72.2%, within-2.0 84.8%, mean|err| 1.197, median 0.438, +signed −0.229, **909 computed / 0 raises**, 100 unsupported_schema). Writes `_results.csv` to the +cache. Re-profile with `scripts/profile_api_error.py --min-n 12`; component decomposition with +`scripts/decompose_api_cost_error.py`. ~1009 cached API JSONs at `/tmp/epc_2026_sample` +(`EPC_SAMPLE_CACHE` overrides). + +## ⚠️ THE PIVOT — why this session is DIFFERENT +The previous session (9) ran **FIVE independent data-driven audit angles**, all of which converged +on the same conclusion — the clean systematic levers are harvested and the remaining gap is +**diffuse** (data-fidelity matching the reference software, data-composition, per-cert scatter): +1. **Error-bucket profiler** → scatter, no clean residual bias. +2. **Dropped-field scan** (raw-JSON field present but mapped-None) → every field plumbed. +3. **CO2/PE reconciliation** vs lodged → systematic +15% but it's a **factor-basis** difference + (our SAP 10.2 Table 12 vs the lodged EPC's published basis), NOT demand (cost-SAP matches), + NOT scope (our CO2/PE correctly exclude appliances/cooking per spec line 326). **Off-goal** — + CO2/PE don't feed the cost-based SAP rating. +4. **Cross-provider parity** (LIG-21.0 vs 21.0.1, same builder): LIG under-rates −0.59, cleanest in + cavity (LIG −0.45 vs standard −0.01) — but the wall U computes CORRECTLY; the cause is diffuse + (composition: more solid-brick/system-built/non-PCDB mains, all under-rating in BOTH datasets; + plus per-cert scatter). No recoverable LIG-specific mapping bug. +5. **HW-demand reconciliation** vs lodged HW cost → median residual ≈ £0, well-calibrated. The + high-HW/m² certs are small flats (SAP HW floor effect) and are ACCURATE. + +**The data-driven seam is mined out.** The user (correctly — they drove the glazing find) wants to +switch to **worksheet-level ground truth**: the **summary-report-based audit**. Do NOT re-run the +five angles above expecting a new clean bug; pursue per-cert worksheet pins instead. + +## THE METHOD — summary-report-based audit (this session's loop) +For a chosen cert, the **user generates two Elmhurst worksheets from the cert's OWN API JSON** +(`/tmp/epc_2026_sample/.json`): the **P960** (full SAP worksheet, line refs `(1a)..(486)`) +and the **Summary**. Your loop: +1. **Describe the cert field-by-field FIRST** (so the user can reproduce it in Elmhurst): dwelling + type, TFA, age band, every building part (wall/roof/floor construction + insulation + thickness), + windows, the heating system (sap_main_heating_code, category, control, emitter, fuel, PCDB index), + water heating (whc, fuel, cylinder), ventilation, PV. Use the mapper to dump the *mapped* + `EpcPropertyData` so the description matches what we actually compute on. +2. **Pin the cascade to the worksheet line refs at abs=1e-4** — `heat_transmission_section_from_cert` + for §3 (26)..(37), the water-heating/§4, §9a/§10a etc. Localise the divergence to a specific + line ref → extractor / mapper / calculator gap. +3. **VALIDATE BEHAVIOUR against the LODGED SAP, not blindly against the user's repro.** The user's + Elmhurst repros are APPROXIMATE (they often pick a slightly-wrong system / inputs). Confirm the + repro's continuous SAP ≈ the lodged `energy_rating_current` BEFORE trusting its line refs; if the + repro diverges from lodged, the repro is the problem, not our cascade. (See + `reference_elmhurst_only_test_pattern` + the `_elmhurst_worksheet_000565` prototype for the + mapper-driven cascade-fixture shape.) +4. One confirmed cause = one TDD slice = one commit (conventions below). + +### Starter candidate certs (clean gas, single-building-part, schema 21.0.1 — NOT electric-fabric +### tail, NOT LIG, NOT deproven). |err| in 0.7–6, good worksheet targets: +``` +8700-1771-0622-8501-3963 -5.77 gas cat2 whc=903 (electric immersion HW on a gas main — odd) +2135-2729-0509-0142-6226 +5.29 sapcode 119 +4700-6865-0122-1501-3963 -5.19 detached gas boiler +0700-6754-0922-3505-3963 +4.35 whc=911 (gas boiler/circulator for water only) +9093-3060-2207-6506-0204 +3.60 sapcode 502 cat9 (WARM AIR main) + whc=950 — see SESSION-9 below +0330-2817-5590-2096-7831 -2.95 gas cat2 mid-terrace +``` +The full list (81 clean candidates) regenerates from the profiler/`_results.csv`. The user may pick +their own certs from domain knowledge — let them drive selection; your job is the field-by-field +description + the line-ref pin. + +## WHAT SHIPPED IN SESSION 9 (don't redo) — 54.90% → 56.8% +- **`a0432977` glazing single/secondary/triple per RdSAP 10 Table 24 (THE BIG WIN, +16 certs).** + `_API_GLAZING_TYPE_TO_TRANSMISSION` only mapped double-glazing [1,2,3,13,14]; single (5/15, U 4.8), + secondary (4/11/12), triple (6/8/9/10) returned None → silent u_window default U=2.5. Single glazing + at half its real heat loss was the killer. Method that found it: profile `sap_windows[].glazing_type`, + decode vs `epc_codes.csv` `glazed_type`. +- **`872bc585` HW-only heat-network DLF (whc 950/951/952).** The Table 12c distribution loss fired + only for `_is_heat_network_main AND whc∈{901,902,914}`; HW-only heat networks missed it entirely. + Added a whc-gated branch `water_eff = plant_eff / DLF` (RdSAP §10, spec p.36). All 3 corpus whc=950 + certs improved in |err|; cert **9093 still +3.60** — its residual is the **warm-air main (sapcode + 502, cat 9)**, a SEPARATE cause and a good worksheet candidate. +- **`7878a969` fuel strict-raise** at the Table-12 factor boundary (the cert-8536 collision class). +- **`49fb6c1b` glazing g remap** (codes 4/5 → correct cascade g-slots) — correctness, 0 SAP impact. +- **`a7990edb` ROBUSTNESS GUARDS** (forcing functions): `_api_glazing_transmission` + + `_api_cascade_glazing_type` raise `UnmappedApiCode` on present-but-unmapped glazing; + `seasonal_efficiency` + `water_heating_efficiency` raise `UnmappedSapCode` on present-but-unmapped + codes (was the blind 0.80/0.78 default). **0 current-corpus impact (tables complete) — these are + guards.** KEY for this session: **if a worksheet-audit cert RAISES, that's the guard surfacing a + real gap — map the code.** Also re-verify: efficiency table already covers WHC 908 (multi-point + gas) / 950 (HW heat network) — those are NOT unmapped bugs. + +## RULED OUT — do NOT re-chase (verified this session + DEPROVEN list in HANDOVER_API_PROFILING.md) +- **The 100 `unsupported_schema` certs are full-SAP NEW BUILDS** (`assessment_type="SAP"`, mean + rating 86, transaction_type 6 = new dwelling). Structurally different (sap_walls/sap_roofs/ + sap_openings with measured U-values, DER, construction_year). **Out of scope for a retrofit + product — do NOT build a parallel pipeline.** They're already excluded from the 56.8%. +- **Solid brick** (gas, −0.52): spec-faithful — `u_wall` applies RdSAP §5.7 Table 13 thickness; + direction wrong for a thickness gap. Data-fidelity (old houses outperform as-built). +- **Roof code-8 sloping-ceiling "insulated"-no-thickness** (cert 7921 −23): data-fidelity, we ≡ + Elmhurst at uninsulated. **meter_type=3** (Unknown meter): data-fidelity. Orientation code-9 drop: + the East/West "fix" HURTS the gauge; conservatory-only spec rule; leave it. +- LIG-21.0 divergence, CO2/PE +15%, HW-demand over-estimate: all diffuse / off-goal (see THE PIVOT). + +## CONVENTIONS (non-negotiable) +One cause = one slice = one commit; **spec citation (page+line) in the message** (the user +explicitly asks us to confirm against the SAP 10.2 / RdSAP 10 PDFs in `domain/sap10_calculator/ +docs/specs/` before claiming a fix — see `feedback_spec_citation_in_commits`); AAA test headers +(`# Arrange / # Act / # Assert`); **`abs(x-y)<=tol` not `pytest.approx`** (strict-pyright); +private-symbol test imports single-line with `# pyright: ignore[reportPrivateUsage]`; **SAP 10.2 +only** (ignore the 10.3 PDF); no tolerance-widening / xfail; RdSAP is deterministic — every fix is a +spec rule, apply uniformly even when it unmasks offsetting errors, **but flag any within-0.5 +regression to the user**; **pyright strict net-zero** (baseline-compare via `git stash`; avoid +`**dict` unpacking into `make_minimal_sap10_epc` — explodes pyright); **stage files BY NAME** (the +tree carries unrelated `scripts/` + `sap worksheets/` changes — never `git add -A`); end commit +messages with `Co-Authored-By: Claude Opus 4.8 `. + +**Regression gate** after any calc/mapper change (goldens esp. 6035 + 000565 are the gate): +``` +PYTHONPATH=/workspaces/model python -m pytest tests/domain/sap10_calculator/ \ + domain/sap10_ml/tests/ datatypes/epc/ backend/documents_parser/tests/ -q +``` +**IGNORE these pre-existing fails** (not yours): `test_total_floor_area`, the 2 stone-wall U tests +in `test_rdsap_uvalues.py`, the flaky `test_other_client_error_propagates` (passes in isolation). + +## ARCHITECTURE (quick map) +API path = `EpcPropertyDataMapper.from_api_response(doc)` → `from_rdsap_schema_21_0_1` → +`cert_to_inputs(epc, prices=SAP_10_2_SPEC_PRICES)` → `calculate_sap_from_inputs(...)`. Fabric U via +`domain/sap10_ml/rdsap_uvalues.py` (`u_wall/u_roof/u_floor/u_window`) feeding +`worksheet/heat_transmission.py` (per-BP loop). HW in `cert_to_inputs` §4 + `worksheet/water_heating.py`. +Efficiency in `domain/sap10_ml/sap_efficiencies.py` (`seasonal_efficiency` / `water_heating_efficiency`, +now strict-raising). Fuel cost/CO2/PE: `tables/table_12.py` + `tables/table_32.py`. SAP equation: +`worksheet/rating.py` (ECF = 0.42·cost/(TFA+45)). The §3 breakdown helper for pins: +`cert_to_inputs.heat_transmission_section_from_cert(epc)` → `HeatTransmission` (every (26)..(37) line +ref). **KEY INSIGHT: the gov-API JSON is the published OUTPUT of RdSAP software, not its input — +route fields Elmhurst doesn't consume to the spec default.** + +## READ ALSO +- `docs/HANDOVER_API_PROFILING.md` — the full SESSION-3..9 log + the load-bearing **DEPROVEN** list. +- Auto-memories: `project_per_cert_mapper_validation_state`, `reference_unmapped_sap_code`, + `reference_unmapped_api_code`, `reference_fuel_code_collision`, `feedback_software_no_special_handling`, + `feedback_spec_citation_in_commits`, `feedback_worksheet_not_api_reference`, + `reference_elmhurst_only_test_pattern`. From 10335268122a187ce1325098724c8ba461bb782b Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Tue, 9 Jun 2026 16:41:00 +0000 Subject: [PATCH 33/87] =?UTF-8?q?fix(elmhurst-extractor):=20read=20Main=20?= =?UTF-8?q?Property=20age=20band=20from=20=C2=A73.0=20Date=20Built=20block?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Elmhurst Summary §3.0 "Date Built" lodges the per-building-part age bands; the Main row reads "Main Property" / "C 1930-1949". But "Main Property" ALSO heads the §4.0 Dimensions table, so the global `_str_val("Main Property")` collides with it: when pdftotext renders "3.0 Date Built:" glued onto its "Main Property" row token on one layout line (as the recommendation worksheets do), the first standalone "Main Property" match is the §4 dimensions header — returning its next token "Floor" as the "age band". That garbage age propagated to `u_roof`: for a "Pitched, sloping ceiling" (PS) roof with no lodged insulation thickness, `u_roof` returns the spec uninsulated U=2.3 for the correct age C but U=0.4 for the unparseable "Floor" — collapsing the roof heat-loss term and inflating SAP by ~14 points on the affected cert. Scope the read to the Date-Built block (between "3.0 Date Built" and "4.0 Dimensions") and take the first age row — a line beginning with a single A-M band letter + space ("C 1930-1949", "A before 1900", "J 2003-2006"). Building-part name rows never start that way, and the Main row precedes any extension / room-in-roof rows. Regression: full sap10_calculator + documents_parser suite green bar the 3 pre-existing unrelated fails (2 stone-wall U tests, test_total_floor_ area); the multi-bp / "A before 1900" fixtures (000516, 001431_case*, 6035) keep their age bands. Co-Authored-By: Claude Opus 4.8 --- .../documents_parser/elmhurst_extractor.py | 21 +++++++++++++++++- .../fixtures/Summary_001431_topfloor_flat.pdf | Bin 0 -> 77066 bytes .../tests/test_summary_pdf_mapper_chain.py | 18 +++++++++++++++ 3 files changed, 38 insertions(+), 1 deletion(-) create mode 100644 backend/documents_parser/tests/fixtures/Summary_001431_topfloor_flat.pdf diff --git a/backend/documents_parser/elmhurst_extractor.py b/backend/documents_parser/elmhurst_extractor.py index 01b50deb..e2a5f1e7 100644 --- a/backend/documents_parser/elmhurst_extractor.py +++ b/backend/documents_parser/elmhurst_extractor.py @@ -1727,6 +1727,25 @@ class ElmhurstSiteNotesExtractor: )) return arrays + def _extract_main_age_band(self) -> str: + """Read the Main Property age band from the §3.0 Date Built block. + + "Main Property" also heads the §4 Dimensions table, so a global + `_str_val("Main Property")` collides with it: when the layout + glues "3.0 Date Built:" onto the "Main Property" row token (the + recommendation worksheets do), the first standalone "Main + Property" match is the dimensions header — yielding its next + token ("Floor") instead of the age band. Scope the read to the + Date-Built block and take the first age row — a line beginning + with a single A-M band letter + space (e.g. "C 1930-1949", + "A before 1900", "J 2003-2006"). Building-part name rows + ("Main Property", "1st Extension", "Main Prop. Room(s) in + Roof") never start that way, and the Main row precedes any + extension / room-in-roof rows.""" + block = self._between("3.0 Date Built", "4.0 Dimensions") + m = re.search(r"^([A-M] .+)$", block, re.MULTILINE) + return " ".join(m.group(1).split()) if m else "" + def extract(self) -> ElmhurstSiteNotes: emissions_raw = self._next_val("Emissions (t/year)") co2 = float(emissions_raw.split()[0]) if emissions_raw else 0.0 @@ -1744,7 +1763,7 @@ class ElmhurstSiteNotesExtractor: number_of_storeys=self._int_val("Storeys"), habitable_rooms=self._int_val("Habitable Rooms"), heated_habitable_rooms=self._int_val("Heated Habitable Rooms"), - construction_age_band=self._str_val("Main Property"), + construction_age_band=self._extract_main_age_band(), dimensions=self._extract_dimensions(), has_conservatory=self._bool_val("Is there a conservatory?"), walls=self._extract_walls(), diff --git a/backend/documents_parser/tests/fixtures/Summary_001431_topfloor_flat.pdf b/backend/documents_parser/tests/fixtures/Summary_001431_topfloor_flat.pdf new file mode 100644 index 0000000000000000000000000000000000000000..e61f3466a02fe2e390858a38b4a9f1ee04e34c77 GIT binary patch literal 77066 zcmeF)1ymeCqA=7MTDs+ys?s=AtwqEHkSr)OqhLu4jpBDK~t}w;~t4PG$ z=b&5+lyQ9Da|v`rw?JRB-3ucc=<3*W>HhFhP*}JE)qsdl&*6zMZqOSfqO9jcTTL3z zzD!P-6;3o}f;BdjR3~@cJfEIvm?$d!cB!gnM3mf{>-QgC?JHgKy!Wmf=f=vc z*7fA5&4gh=iNc!<@9R7u?9q}9`vlyCR-c6N4u8B*3dRNpH|&4oCR7_}21nOAoTc_l zbYhKnq0A2-k6<*cm2)c9k|)P z3Yj~;%A_^lXkOge@eHkW=o!YHlJFdwn@pO~v!@HF2YoSIx&=-&DD z>$ty@MbnKc-}6L5f9`AZ4S2{KVto`oF}b3cb!^%^5GA1Yv%~2nb6$?&M@g?^6)B3V zgx%S94L2VmJMR@$DD9uyYn5SERWYIUQ8+||Z|0Vw2Emm0wX=}$)0awW&i-`dix0iF zp9r^ zq-dem=fI^^b7S%8#mM{vZ)m5Lh$Z>F!d_&j>EO->yAA z8f%j_FgK_y70qRSX-?U@;; zUlbwfw=vTzZf_k3WtL24nA~TM>#t4QGk7a`iRR|`U0ANq8{SRXfDS!U9rBA2J!3jG zXau|OuG?@nBp$RJ+aF5KOG+5OS*vq=$vcC^E}^sT}7#ffWP(^tlDO14Y*Qw*YoMGhQCfh-=H zqaD!pNyymJE`igk50!Ik(OA%ak{kJ}RJtZp=88Idn>aRW#`TC0MW&N$Kj~h22-k5YdlrpV`=|NBEdk=VnS^8lteo zeLNHM5A4msUu0&kJsiqk)QykrXs}f^;)kw~FwLA@5OX{Cg2i09*j67r1)5Y0tc%MV zu8%yt=C(dOoW0!>jjp4jzp`v+NqS?%JE!jg|RIZGC8@C+5T|d z&~wtKg9Gn4<3v>dAu0H{v?T^)-%n>@c;V|#ARVZMvp^x9O>ytAFDO_P3+it!Py?|V z6{|i=040gu@*I0T*jM$IeW{p!fEj)NkfB5F9Z_i#@Aom3gfx1A1?9pnlq`!H{K3pX*&}i zqu{g9dpMptjIH4uscOgGujYIiwfJF#me8O#*zy~P&aKhj-T zXNBMADi0|c8MVEO%tAV{&4LqC0Otn)9;JqzpdzU+C<&HsTz8q zy#u=zoc4*T$l&y2mVJ*fZNtL6`I=7=8}*}4=O*7+8TaPk6ww!t(uROBe~`|Ju?2e) zohV>tCPRhH8ebw__VDv?;XJxLu;1 z+g;Rn+)qD3{|a2}lD5qs8VnND?H5t@I%8cI|3!4S;041L-A7W{k3U_=l$Vl-0Lj;3 z*lm7+MDC|%gtdJvO_D)oP3JSnTYJ_QM5}o2PMHs9wE0fF`=Pp&3E?6rNQghsaEP*I>& zFwa3<^B_p`bi6KuzM;+H>?ZR}CK7ase8XkpmEdg4B3L!VS}dMp@w?c_vFf7AyJWJu z%J(G124X!@(Z*z-Jy&QfKBDUF>=Dh4Iw&7-Pj!cUGXy)FZ`6gSgE}-6%EsHw#Ka2! zN?kyNPU+C#9sC!2|KcMv=88Vtq0;xSXJo}Wxp5dMATjQ{rN^t2-3ja@X>zvY7_s#y zebIgiS>EYnE}jOGxWm2EX9KCCfeqKi*|QQP5nV~I>%U67oP&i7SW~19eYC6VGiTXW zpp9sz-F;~?CcnGQo<{jFX&*UcaX=2&-Jp$b+${b{k(T=G+r1x{4l2TGEz`FpzXx7Y zjI(E+)BnQG98N~Lo$pTkS=o1;!Jo-t6aex;4Q;|VXe&aJySzgpNzoixst`bikR(G} zZ-r(Jj+tj<)|%nzvs+quG0h8uOK#o>qi$)Hg z?M*?tF0GHLP3IcO^5`0^6G1IqYI~n`m+825SBwHw$lB{f$6AWePEK^ioWc$k9OIRD zmyg=NL_^y_Xd`%d-?=ZiyT2E&P|%&<25+=^b~|jmIbN!%R-9pD5Px^w=dagn5Y6I4 zyT_mzcr$d!^6WTd!3!l>f#03X!tMTL$+Jva$@HH2T zEUgO+nt{UnJq@2okJ#2(JxZ|9F8bF!izEDMLmI}X<>opU&hihNUCn1xj$%KLr2|Xl zUwf+ChYk)%2gay$OMlrRoIuq{cEO44ra9GtEL|{@!^l=hddN9(ePGzc{Lp?n^D``F z5PIVF5Mz;W1Jj4-`)(R_jue-?JtXK)=!Q@%biYF#LxVE?tKNj9Ol4GAuj44lQBInp zn+|sGcaZ>g{5DZr#&TXq#-=~)zCEJMrs@NmzFlC0zV`S2{?X|Rg=!{Dgxgn>b%h45 z=W7JjAmX0mXGf8B;*IE?Y&H!_Ee9t-$j(7>mpLMfRTeirFV~;{%0E6*Qg#1IL)Blj z;dq{1kP+c!kFO&wF!q+}_lrwZ=cY@6(0)HJ^QwuyouajP)C&)uGeS>IvgpY+_N%N+ zg!e9;88tGmd&S}1>)6BXllD{!($4G~#>(}%?)o#h^w2o2zJ8yw#$zkeNZ?0fu=%?E zEH7fzwpMTwPiI6b>Isi9qjZz*zDsAe$(+&Q~~11IW4jd1ECw8 z>jl9~$I{V(=Zs55bm?n3WbT%dg!6du&I^s%XP=b<>& znaoX}z}L<46j{3-rCb58i@fXAhrP^ie^?~8PWm;455&pi#wQXrXy-kLgAL=quxpgW zOG%M~M>gZBS4dR*#m@2YGa;~kWSTyY6DcX7AZ&{;Oz%kD!MaWJJ-4PN6^qAv&!Vyo z-C2RnC+gKN)nV;Fa_X`?OSfoPf}thBMCSPQy)gM8mX)IX<+F z6@DRQK(B=b-sds}lmQ9O!q_CTSavZaL6{XHC781!!KbtG^l1FAqkFyaFyrB z5vX*6-1^PsI9Rw6ZAD?LG!ELnPIPR&e%@M_~?UHzyLF8q3Q?(^WK?cMFb zh~PbuL*eDpM4CqsS+{)(GRkc<?DZBl>y&aFAZO}trHm({u!XYbyfLF!O>>VNTQJsF`${c~UW3CY2TV&H#!Ct&5 zG{{K!Nvtx(6hEYx;{3)0%r9Q>`JGTJk`?c^&?Z&GjfsQf`x;zq=i7((Q!jkxIZnv) zu)HuT;r%hcbM)htc@;e zXSzBd6FY%+l!3Y!v!hZ;cr}ZcoW_{MJ;L{Nhm~EMR&=8iYjxrfNoMqD#!%IHqM1PZ z*hpkNMgUj6EXp_o+NV~CFYqziGT$M4JxcZ_)7v>ug*SI`azSvk@^YB=Tl^*)xol3P ze{NEZIyjosf+IKIX0%XNx7YJ~u&>2z4RqT}gri3}6w^)!-C|+;k zSg;|xyz7G2lc*LH44Oh#iLSg}_YzRtm8HnRS#?Cj$&GX~i;&8WkuG*1#aBL2wJo6Q z%Y7}#Ubw#(I24)Ri$H$nbg8^G(B^O8>R9J6EJvrwhZ$46`yLA-d&7l|DV71bb9FRW za{c^dri7m|&B9+8q+$oRigS)ohdtZ0Sd;`mx}rEl!^+(KWA)on$R5qK#5td*bT1BA zc*Tw+hE>FqpL}EH=iHlgFUKZ9FUf?@d!G^sM@sDZu|9(mZoqFLJ$H<$&2wuw=9wuj z6Z#+U@}k69ARd$WCN-Yqh3Q~ENb*kMExPNj?~iE57{T4_4&DBqo({rTv8!o`(xLg5^ty6aV$Fs2k&$gjH(qBn$zO>P;`t|BZqGCZL z$&-btVn^L>PgFO!Ae>h|YHF!bJzS%f5NQYZZ1q|6F-v@AX&a#KK`hYhcF_9QU zDA~mqF-3)JN^Eunv%oFQYcUVV#Gu?U69uHrsR0pwJ$^cZ?0wpi6Rj!p)8x@i3Dn8H zjv{<(pDkJSiTM*mDXKG8Dc;U8GRmAr_vlEaawHxfN>o@N%65*$aV~b6O*w+A^V<$(HJI7U1JP{ z>7P--&&X>LIx`|+{S|Gcoz7LD|3NErsA#>ZvGC0-_Z_r7Sc#t|`RQZ(HPMeTdo!-_ zJpYV8$U3}Gk@2^vSMWU5eM1{>-YH@W^@t3r>x!14dut|0iV+;P8W>@LrNM5f107n> z6X>VnYkeV-Gr(i_c+0}(C(;{qOzvwXk~@qq{8?`~GAPj{b})9jf=cNJk^amNVe@M` zeuiAcE}9+W$>43`%C;wQ_6K~&2#zU|f*2*+TfT6xbiEQAi^1LRv8ay{o>Gh=pp!a8 zJ^%63*-6l^b%uMKoYrl46M{q+eZ9yZha5{%IacpBTuaE=zuR0v5Pq<~e<0Ax>nQsj z9s!$Kammik<_NxowandKY8xihPlR-y4vD-?-CA23#DrYT;^ZX3rxnwTY~zwyFR}5!_NQx7Jo6xPhlhAIlU1n6C^6=@$^O zc)pIT&n$rdO9v-te&yk*Bx?N=MCcIrUxtDIGT6=buMJN_N2>q1;c4c7Fg(r4#?JJg zhNs~jUQNYMWLVC`$yK;Fa0fe2HS)-(8AQfTEA4Sv#mrOmC(x@0G+rXAl{vr1u172s zF6S)sX~cT-p#=^ThDWIEbMAC~S>j8Cr%#bzjNyi|UEcbMpxzHWu-ZSkU*5q*e|`pw zh)u1SCFt|_8*P5U$$6TgMXWs9NQh_zTz50-d}>H^YdqQ*h-j#4iK5S+iqP**k{pIx@e%)-LLN4|Q7JHK^RC%GKg zB9J^`&e1edIewm97F1qV#>BMN5!}qy{h-+)cadOM3nBTpl#YPMY3fk-vMd5?R z(-fd3b}+h7-@KSvWi4Gx&3-!7iSrG~w~ex_rsf3|*IPKPmFXiDmHys!V#gX`qf8Vf zD*^iQQ?~WVv#}9*P(oWq>)F|lX@pW;6BiRl;SQ727v^{7Gg0l=E?AtXl_n= zbjwdAwk@_yPbYPN7wiK9Ho2@nYkvu6s~oFyh>S85Kpt`R1AG37k(i_fQU=B?(DqPqOQJ$Vi%FImUQ1hq5Uck+(s=MN7(%@cC z9XFtiwH5Uqu8pSV3_JVX z-Cfq?)>Z*>WxjRE8KNr(&VzmWWqi${_N2kanL6l2C!nUX zGMmjIPtL@V?9YGXlQX0~nmrS6oY%i15eZb&)F`72WqdgTZIq|_4 zd`CW=r(Sc2c-i5!_{;@XNnQOz_OHwOx4aZ6?B{fe3`>W;O%l*@BPeYBJsdV*877tB zRWMA1EdhaBryWvN=*X!}fM*-(X@x|aCjTe5#IwfDGmTbX3THD-5bsWlTUYHh>>G*6 z@3R3n)9swq`?xJdEuZX%BIIAUzrL(5PWXk3C9;==aarzQ20WZ;8vxLE978mn!SK)v20TeKi??vG?_Br_hKe>SEe;!>7{iY2>UzRF05$*%tKuL&3{>{eBfx6! zyvjE3=A<^c5t?XIIh~Xmj}y zPrx;XrwadWMne4V?e>PC(JL9cN>*dbl43rsg*{UzjU9r5mVg%09wP(E&k9$2ISNGk zgd=fBY-f&VVJKluch|X)#Nl7yF3R${y4l&WXU1RCKxU_IsA2lXsUd;+MlAzX3*K*-xli#e=Qd-_tfjYPGh!Ls<%h|J=k`qLrQ>TiNk7c z;0(EEa(v-A(i3TScefSnUUmdhZDwq^SIivhfz$P3uuqLMGBmZkwosFQ_7_Ldhf>H{saih}Cl zjz9J(8hd0@mZkL|6O*I8)fBQKzS>@2cG(!)2&p((ZOh&pcw6;zyq@HPI+&6xCk=Gz zp{H-$Zey^I(B9`=QR1P+T^N=2fw7M5UMc8U)*bRuxzab7iIu}gduLCD=`KGfr@gCh zwsnZuE1XNXiw^yH_sY2uUi*;!A6+MjwaiyE-V%E`mQW^~tnOkXE;G1N6f zMb)bHnCWRLlPL{UR#wIvf?p}B7-Q*er9bT(=)jg>fPwwu)AQof$TGVA^L7I6z?Ry} ziwj|4$q1G2-*h`(ZX;9Vc{jJbda9eXtFof1{Q7INYsz~PldkJ%`}5O2sY)Y=S>~mw z>o3lP;onr?C6$sE_$yL1RUGxefiUFo?>pSw+%ffvJ#;UKr%~wak=m$xpUb_<7#a!; z#QVzhoHYo6KQ}L5I^q+^2x0?Z31-b@Q6=t?qgR=>9eX2=M z4_-D=GH}S-!88a9*+@x)v1*>W!_w1pSFnQ(eLzmAik^mfc`V6fABQrL{5M3ka@JE| zsxm>&`|*7ZW~LS!oG38m0?fvD4zKmk(~QT*Gk<&4Eba9#B_9}{?sArfzjUp2)gVZj zQJ*0%X9CCgNj3!g2c_~@U|nFbL z&!qO^Gn>8gce+G~WQ0XBh?Ges)8oZk%sC}(OCnyF3URX5gH4S$O+Krr>+U-vD%5u*o@$$yLdh$%YBUtwuw5k4S@lIcU=sDVez9k*qSAMyi`Fu% z+h^`ah;1(+k#9wK!oCI$A$=aiNA5)?S`IgRGDL~C3->Rqmpjzz)5!aQc8 zqod^*azis{2nqZAqx{Ft?*3~Yj0WLPib^W72|i(70Ih0@m#N8#dHg>USijC`W(K9^A%E5$)f*MU&SXekZKDzMhH!?L_Y->SbDIBDQ;mAV-Z2QW!Bz&Kr z9lluofufx*vg@l*$z^)P?cjxsuZpPK%(4Qbd;}rRuz1-Oz zPfv@Nv;y;3--1Ca1d!Y3geFd&p1agP++6}b4z>*t?r&V|&z=0nI3|OMsK+7f*ao!hLwtnhYrbGkea;jNxg=q zMvu$HoC8nweF*HXv3KU1cDL5=+om@siA$9at3r!8Hu*KP>kY?VOjV%K&b35zMoRwIlHHA!;3)|QuBAhEN~Y9p zQ`6tBuA+U0EUl#t^YbAmb|a}_+WXzWgD*7&(_{0(cf~2Gq2_yosoXQ%_+iLL-D3l{dxhgqo4ibtVg?GnC6^;u zJX=if@PUJaS=DuNc8tYomDKmJ-{YDVOL+Qrj4Ov2AD2u!YPOIykv!g1jF*|7J$wX- zYz%&PXGf4jQo)YDP~g7a{EMRb1rK>@K0>yekJES-Hljogfp{8da>>ls$n{Sg`YUvnxfs3qGkf2Qm*}*7Wl}k=dieGwl)QE3C zKHjcON=kIjKu?o5Gxk9Vq6DX%uFc?p*TCM3K^(ccw;rZJQ^+x7XKxz|Qg#W98n91kEKk)Xx-4% z86ft1AWMxt6@_l+U?))dfD`hmFC+vRP9y~+>N`O`t*W6U{C1)>wib`0RH}F&5S?{0 z5SE5rW_lyE+;j%P`rAiaT_G;&z3l@ZiTV-~QITiU7qzwA z6VH41yLb9#)m}1vY2<5EUKJ`hj7Z+z_PO`c!lE~^)m1v~Jgs`GFz%))g!@ z4XcXgr>3MeP+I7dOZAT89l^~^8p{?P!LBXI0oEjJ^Ig%z@g*VlOS(3&=c<#3oBZ(= zV*orR^cVe13kwYzV&0S?6&935ZN-^vp44fI3r|bXfAij3IQf}~$ItcgB3xKM;LNzl zInpIPO(&NF&h>NIk01U_=nU2(W@H$;`np0-i@w=qxkikxo6p^%iEl$1O^?rSXI2YJZNG^2u$P*6|`R~O+_3n+9!s$R-qLtlwGMu(V(BfL;ZuVld4x8A4e zWtJKjV!D?f4GiA!(W59yi-8y7!Q8L`GlrYHOKfxxI6bGOjTY7MZyIwSByL_-r#HzU z@_afuu?tj@NC+vU{YqaUJXp1@6Pdwto3LRUkV~HbQM<3X%9etf*JkVR5QK>na}v@U zPJd#%a;La|?epA-3j1V-@9X1ZO>u5#`!y}hfq{WD2WRJ_cY*$G6RN82dx?n&FwdW- zdw6;4EcqgiR!h#$e7==jvqO5)`{^T*ZoS6+a3_CBchr}p#`^iZ~Nc8=; zm$Z|{&e_F3uzkMl&>|JSN$ecZzpCQZPr6&xDQQ-sUbuu7iYPA!3%Bu9JV$5epg@1j zUF=)WyDj-4$vldR*haC{-h^a){Ej92@85$Fg5@tF803e*#9*$@$yrLj=r6gXPIBRp z>d5aoSp3n1dwa(;jkmEaW2>v3Jw20?^+*DAeSLfjd}odh!4zIFFwcY}ZC(s^O@<%r zZ2wv@Ib2?qr3wt8q@*N&O)lPPucfZ4TJtmTrCx}VuiS^liv04I3d27q!Wki1}0lovC#C=)Xlz|f`E5-_WXN=N8UN@s@dvB zf64H2F-emBwT#r-A(X_9q-g3M*CXA+kl z&qw_-O(#j`rkuC+qn1T+LLD4&fWxofzg=bQLp3gD6V7$XzKFvGCWnQEgoOC`)O-G# zC^oZ1DfxW#f?Qgq>yG6bMe3j9lZB9_kF(y2%fIdK zpQd^`)T5^b2Ih8j3?>z8smZ&mBo}J;HxnbhyUNeAP3fpUXBF26}Q1gdTE|wnCtRkvrx6b$j-yaRRS)omZkezPK{}D&pbPt*ChsVl=Wug}8PYFP+&trbo60n|aS-$ujeIfONw8nUM9fJf9jU&dhev=8>>XCs|a@B0D z#K6R?SsR9T#hZ&2?-=OIKNc^;u#1Fos6OSy1L51hLF2XAMu<{RPKMqt(!AMKK*-I{ z%}A_3YZqxZ>dHr1Cn9;TSqP`4*I(BOVq@VU&!9*gz&_9gTdmaEi5By~O40D2pP z0C_^fcz9LA6_C6VWja~aDVA&g?t*-z2L08lVWD`*t+bK7=wPBzaVD|dotE3l1lcv7 zf_XCmsC3VTWhL})^Q=o6iU~uIuw#a`)RaOtN!QFsp0w1i zxqcHbng5n{F^%cmJxSt)f7_J1c9(S;Euh|H|Jlnyd0c-)cspX~^GLmW9gpM_A=GjI zBe|1l-6hiIcpdyci4@T2`%yAHU$WjFX`c&mn;`Tj+`-CxYNF(BCLTnm&8&^q27Hp4UNPjG2apvL*w}ts~9bQNb&tI*1jn`xDmi6N) znB8$iVbm20Bb^}aS(;{89auTwIpcu>5NJC)d+~J>4WyAB+~elEh*+LLa`ov%Qyi#_ zwdACPuKE@y8R$+AhE}qk%G1!kjca!Kr1M0!-~>^yynV8F03IupnlU-8W07*qU6=1; z4IgvsK(Eh=#;y{nF-R9|Ying_Gw#LrYt6e-eYf-7md6>iU$a=j!^=`z+X#7&tHQjv z46Rw;AXtS8DOU&<%)M!Lb14*C>Y;$GHbAB*>ciTER1$eZ$$?3ZSzN2XVbX zbSz#HezSA3f#efXP44;;+WM#r>QPX>p;kRHZ$ue=OLUr&>9KV&vt*(}?8JSM`^);- zm>;Jm;lS&kdq%^v&mib)ujl7fXkZ}I7FD+(T{5y)T?Bl5eDbIfCu&=_wWnjTE3Jm2 zg($BaE%{!4n)h8HS396K0O5RR|1v#X{+p`XchS55Bz5ZTqSw^%c<+pyLU6I+)U3t~ z_58LlLRCwVe~pXDSGo9Xf6#Hc?JJHTXeDSaL-#u>7dr5_qs`*Ume}iGy`a*0!A5E{ zTQk2`T)f^Py5(B?G@aInxIOb`AUU=9A@p}J6`M{l4VgB>R~XiDRyEv&zQnYCeWV66 zGYd8jHUiQ^{+J+4FuNUBeix;m^yR*+f=o4==F~9hQ(+vbC4NzD%HL{l&}g$fprd7o zNQg?YvC7IyADa$Pw$>9_xuetC-CQG^vd$B%k~$MYKSzdsuPRfrElX;_b)mqt7Nl)t za1o(iiI<$;&$lV~-12iBTd%iLY$n7dK0X@ZU2w-P%Q8%NclCMDM<=v{`C7j!bsbP* zVk(kT-Ww`orm=ojO^6x+zRK;#=KcCgB1_GttDT*1>EV{2GBaa%pWoe$&7nenpy<5N z;jQF#kuhgA5OlZ*cXHQ>1?+=KV76nB!14ACb_5IgTMxz2Yf^LsA7gJFZ!L?xyq?fL z>vi47HZ4lAXTy%JufMtHOi+1L~wjg-yc&y)_$%&3pFwEfhoD1O=CJE%KF{R>A? zRXwjT-bO~4h~*h(M!k( z6-zW;AOdfHe;ang4>T-kyuP_%%*65$TulG={g1Qh%iG(A!sy_$z#k`D+uLw_7A3{D zgU|u+MdFMjbI}4Wr`?25R&A!NB#?-RNEck-tHn-{(7M89PT|W=Y%9GSG77==p!tA> z7l^!(#;D%irG?R%RWN?FD>V;UAIrXEIQTsS^DrJ5y)A$ZS;g}1x0b1|sAwxSDmp2` zMMu~$+<1{zq+Z` zxh2V~jwGM^P4tUC1;IQ)p*I%}hK7b6)-!^cBGR>E+N7P(?$Nq;I`r87Zn$weB>AjMehII1IWL7BpLVsPoA9V z|Mw4XAF0uQhb?0H2M=!nTLjpm|4+FQV2c1-1lS_L76G;hutk6^0&EdrivU{$*do9d z0k#ORMSv{=Y!P6K09ypuqW?eIB94D;c>15VMXdi|cp9)pfGq-S5nzh|TLjo5z!m|v z2(U$fEdp#2V2c1-1lS_L76G;hutk6^0&EdrivU{$*do9d0k#ORMSv{=Y!P6K{*Sjs zk5e4|ci1AffAH`Yutk6^0&Ed5ZxJwWkv%YP5ioBNFmDksZxJwW5ioBNFmDksZ_$7G z3;&OAfq9F7d5eH~i-38HfO(66d5eH~i-38H{zuPS#PzQYPyf@li2WZ7PXo3Hutk6^ z0&EdrivU{$*do9d0k#ORMSv{=Y!P6K09ypuBES{_wg|9AfGq-S5nzh|TLjo5z!m|v z2(U$fEdp%O|M9kn`(Jx_%gV+mVr^k zJ`La^02cwc2*5=EE&^~7fQtZJ1mGe77Xi2kz(oKq0&o$4ivU~%;35DQ0k{ajMF1`W za1nru09*v%A^;ZwxCp>S|HtDZX6AqG>FqzQi@5&5(_26n0lEm#MSv~>bP=G709^zI zbP=G709^#=B0v`bx(LukfG+Z?_?f*UBtELsPh6QJuy)K9;tS{^KobP=G709^#=B0v`bx(LukfGz@b5ul3zT?FVN zKocmR<{lEVGu5?tD{O6Hz$k8^Yiv)-%JS!+u&}j@COsP`Hz_?QCmSg<8ygcTGZPbs zHb1|im4UUM86q>=->1N+?5OA9YGcT#V&`b6^0@IQ5pgC)VO@K}$3z+bLxN&f`ql=f zR>q7VQ!61Wd(*$Y7dN%DcMvhrwR_BstnOd7nORt%Aq^Z%>@`_9xkw)uR(5t$c6N?G z3l|qRDF+89DKj@SDJv`Mzb(*x>|AXB78bhiujl>sod4DOZ{eYFJjU~9fu_gI#LWID zvBv~CSy=uokJ}tj)x`hzpv5KVhN!|D5-iaDO`wdd}lEHyhjE z;(ml zd3`(&y8W2P$Ms(p?LXz=-^&gu^JCFc`uB3c{`ckJ-$VbGlJ-v*GLFBG{Qs$tG5?oB z#`w4L^#8JaadH6V>%YHzLCZ4q0_9-(vwpn59{;NC|K7jH()^#UkiT9L&{Q5v`QtH8 zR!&k@&c9qa&{Y4rkDZ-`l#Pk$?@Rq-G5-&xo)h{OS`G5@GXAOO6m^Xa85MQy9;bDC z{KXk1Mma+RQ{Bf}^zpN>bF(wBa6*&jVq<22)>K^D&?(}q9H61?NtwC+gjX`Ow|2DC zH?)UF`B+mvuAp`9Pvb!?gZPekZOjJh-Ibl>t(c$(t?-?J-=qUPN{mk{W4&;M* zeB@txaKHIKQ`o9%!Mk*z0^!CU%28;E|^T;!*7!Lzq;%Q{Ni9J~n-el39 z11U^!PR;$LBHIhphg|qrqltaS=i(XgY@b&}F$vz3!QC&@Y|tSsOUqVka1juqwDyct zc?229F-Zl+KE->x%nNt9%7vcR)_ojQ-|11h7o&xZrN8;DPxjdM>T=nkWx1)*i>aWGdq*`D`rGed$(SZ~%7viYCM6GBRGh}B zeWlF8C~1BtxT@|Y8Rrt~6Yhj;fx&$>_mQINBPX6s^()K#6ALfJ?FzG_S7N{PHl!Ne z_+R8WB0!oq{Oa#sb*?7txahWyhRl05k>Q#ZCaqTWvDyl^6L|J|1Ti)iSlR=M)@wibWVyLWG;=iavxBC9(Ic{Jsi`W;L0o}aOBEmFq z0}X6OfHJzO2O{ss#Am#&i9VceBZe){Gw1~=BPw{k+pFA zao~q2DO{Ggj#X*IU}+b=WpuuoHRO`!r6H-ux);%Oo>ZLi3^2lO$9E6N}7nng9KhYBw zP3GEi^Ep?-_(c^{yw_|Q{;6&|zJoo*Z;_2!upV`+yyVbew!d#jl^=f`)%qJcIz?%e z!9%_ufp_VoA3pzTAAK@kyJG;1lE1{u-N5G-%8n z!DrPTx&1L=qs7EY`6%YrbNe6G5Ds^-x;uM!4LR{#Z}5|*#@g~@!;nxCpOUOp}>3A zz4`W*%(b+)#nJ~4!wM;BNwWYmV)GQ)1nGIXrjol)xkx?vz6WK%(XTy>D3pv*Aq&wccm^LAzZS~c!lZ1p3zqbs*=g!`}h+09MaazIa9M7Ij zymESmuGxe|+sv2h(_reD+@gci_?aBa? z%UBW7bBl5Xq1%t|-clx6oogEwgAefQZ?oQgIP6*dP@}~t076sGBes*X3QvAL^RA|& zjCr)ebi=B|Ioe6hO-j3uT89xYip6<6I!7$w@IB-38mdQLMK)g>wE)i^vui!ZEQVpP`ZrM<({ZoGL_(j|x5143aS>pjqPPV{<5@_`b~*n~C-{ z{H^`LN1KvPX8Z&zdXEf`4~-oozd}X4uZk|!5lca2)N`ME;)Mm6Om9z~y_=eGn55I! z>JjWyY?C@nvoyEw_KtH9Q1WkT?D zwQN4$p_9pWqfqoGE$b#NnY~xEp+8KP=c&~n@h>^X%iCQpp$rM*WfizIVqg66ok!); z?wIB0dG)>Gg1e{)hEPE%y|DJ{0X}NB7JYq7j>Xq~Ub4gov)D-qX|>J)aCZ8A7VIr; zF04;=5aub-YJIpNzsoceJ*ap?MNUN(K_RCdjqliMZqzwaCM303_}+Hgl-Fo4vH4q; zQS+k>XKM#_t4{Q{f4~2QIxO?#_9CH8VNT(trgqeGO0A0BhZ{J~``2H8dsLN?T7Eq@ zajo`G_5jE9TV&K;j5NM)!$b1EV@Wn(D)y$YK$PF6vI9ly3>H&=Nu>MU)=2p&TfmDv zF%{k{)@eg zSe6>}hO@ybHqEJXaVPhS=or$_S0Of1eIYlI*No1B>gz2mu!`KSpG8hnt(jPtkOkL= zr9^{&V~u^n5|T>mBb6HSADAK$u(vCudPu(Imbze!W{P-5+~4E%eh)m~sLChrPil*O z=*8Gc=i3hXY4H1cGEo6byb6mz)D{!kJvc(W?Nsj5eC!@9Rod`#H^aZaRPDYzPQk>D zx!LeX!nGmxbGIdYA-}S+nmIn`mafXUZeROz;+?sj_Wka?cWlwkCCYpKo+Fv2HFzvr z!Cn&e_df-QgecuV_Pkq$sDoqbr3DJVx!UTIH8Cd=UEFU-p$Jc-w$+B$a=pfzu|MeU zY(u@QE#~Yz9{DKl-#|I#`o5s^1(^2T4+Pp;osCmS-Z%3TMKAjE$zgbd4QM|=oQ5Rp z{7buu5RE#-lRCBuli15+C?uS``HBJE?+&*H5BbRfO95wvX1+1L?$vVx@>HAXwWomW8}M^*H8b(=cAyrXMg zu2*|;0AzOT^E1G!uU~=%3W>3rBO`1B^i~;Z3NbRiphcyhgRQy zvLQh_C89`hYdZC!TCQxTkeJhO8V(Is1Z_-mh}IPPd=;BFL3Y@sc;z2YvVjCwQuC1CA70FX@$bpo?Bu|)(jIYbJ!@ZCGd=l z=xK0qs1$)DPG1j_d6qCxS#j>-cStchi#JSu4zocUW7^-e zQ@EtHemT0>F+HMa&>lx5TIK#L%p&P*TYta;a$@(T8Z1WAmK)+PJ12GErh~Y9m#KZ& z{tKhoTdi;aR!SM^q_F3nMVRZI`WF8*^G8F{i~2%ZO}f>95Lru`%@^(gHeCv&X7Bk% z9XYHWHee9k=u}HcC$f(Wtqr&-=~JD7 zv0|Sq>;Jz>xz3;{mu4*}5+vvBA|QEL!UnQLSqV$d5?pd-K_urWQ4j%%0(!_fD`^Rm zK|r!BNz@eqiIOk+Ro(N6_tw32zNvcubj{4$JvB2=_tRZd+82?DdRDBXz0W@EK7l+acRQ`m)*GlVKZKoIR*St<_gAssk-&vCSY|vxOd`ns@uc< zc*y$#^G;zM!AUn_4oD$AhJ;@{L}Fm(ItE{}${{3|^5A5D`fY33@>W@Am9%4HSeRH<)kn6^Zw#A2PLb`vXBnKk zpVVGPNbIr_MFL@+T%N;4BD`$OaLh8`d8cu*zZ7~Q`W>vR@toIYX&@2bVvfUSx(t{# zC(lk#ah+miiN4!_T&52YRbj2b7{yJAVP|xwOo?U|U|cP$uqW>_h7hcjF@-S*G7}GD zm*DZ?NHOonsM3jqC-E#}6tlAJn#-~)stns>D&-J%Ee<`%u6FR+ipn4>0JpXI@$1!_S4#BohXyOL;2aEyTyUTZF$P`JR1@ zJCSgOeF7=}=Hfey7my*10~a ziBOVQt&xnxqZ0h9Zwv8P;*0+}kHFLX#_JeZ?l5BeR=PYNp-(@;&3R7$!;dr16Uig5&>_mI~t72E9(O__#7 z#NfC+{(OI!GqdfK;0cbPoV&cY<&&GfK1pCTYiGT|_v3Fy!6&tYdI?7-A;AOBS*#^e?@->UfX2L^pt>;-fX%+Rm`@G)x#_qzHQ=@fXv?F4OoFSt)%L1K<9*PRdgH=3R5d8|!S9j6JH&g8-kIPH^IFhlP-g^a() zd}TWK^>D;aN02^=^ay$ba@#R>gO$q*XSa6R(Tay8Cpg5%a)uVQyfa4W7GvMd)EQY@ z_Qj<4gbsR(MKYK8Y>31fp)5=J%CF+Qdvs0VNf0SyXkr#Pj*}upSr?GT=hJkY;Jes^ zSuHlGfY9uQ-6!wced#>WeEnlbW~9)rqSV?}LCDQOW1u@doXlz=NJBc&PX4S+-%4#5 z`f%aQtm<{?(NXYx?IUjE`(VBwk+`u!8WrZAa&FHP*ky7y_09vOVO4v7Iw8WBo|pPh zoREJJ8vn!z5e59@NwfsvV8Aby<2Qohf@l1H5EP;!LjUH3)EUm9E(i+$DgAX7o7oFC zq5W7@B1`!h(uQ)XZo=oNSNVdX;Gj+{T{+Xl97W@Ku%*7Tb58a=F~5P%4((o8La^rq zpSsN$1a$(U-nh{#vGLbT=>s!M-i_oV^0#UU=i6__wC~I@wRHA$0$UiwNUsIB{#bC2 zr$qa4-u=2#KG(obO2!V^4;{sWkPfl#s3@qeQw(b2?(?H9MGff_!^;m%%|MGl{z>+w ztt#?hF;ntbmX5(??3{OsRaHP(iC>uqZEAFXoAHVCbAfF92eiH0zRT?{fJO=HEjD#j z<8otU!We7*z^zz{are@uhHLiI z7@#Nxaet^KZA?CLh;}#R$Y7o<@(I$imMB`O9us|VZReD1(JK(avzk3lI-N|0&^5JB z+2;!?q9EWg_t%%aDaNJ8)Wl(ZO*CtedxG?4ZuVOVN-%CUPMatWS%}Cvv2*S0XV^f2 z^wlz*>mT*7sx&VNGht>O%lrCCNP@}wAsvQ<-WyD{1yr}ZpBow^$2b?5_P-HjBeid7 zM7n-f8HB^m2BHN8CWe~H?R36z9J@DyT#YX7&@rc#Q(6m z7Mfysg`4Nzb*HC0yy}Eq&btK-b6uTa9@_y5Y^4=Zo{es1hI+H){L{YmYS4nJ_?3{? zdei}JUxpTXy6JXqr9D3=Vrpuga?AH|8jUll=<}NwL87z1t*L9u6Hre%B^jtZ%S-~H zKYkzyAE%JYe6^TtN1@!sP4$G&i|m^sr#8x`9RwjJqQ+-g*>*N6or;JXz;!esO)ZvW z^g2qrefW5yL4x95ILno4nMfxNBl&M(YQ7pG!^tvmy0~zeUR^CK>YiM}sFF8@>mK7W zPUaAfA3)QEfipf?$JU*vLxTsE)3*nXO=t_Au^`hD7~m?q;u@H*dCpGoT7o>GHEn#) z`NEaQ`geD|H&O%cn|-RW8tJMEZxpyH-lyx4+-*momSzva&^_n)s~hOR#%U@$$!~2`PJi zrTj9ya|X1#@RbK)?FWt8zchH}k#?~d(K9F`ex zL@ve6XKZ%R%2)ESJCeMgyH^t=`JzS;*ko4|$@eiE_St$Ywb^b14~hAnw^A~z#q~N^ ze@lKWem3uFhYxIlilN$cKMlyVlW)>?j`DzUn6Gu|Z^Mf2R6PpoC|Q@=jeCBQu9>`| zS)<)+P!d+f+P1d1nNqd+ZDGlbc?}TQ!9P@=0=$>@QOiZ0;Z4v1{Mj_!v~g1vqH6Z! zaNH-0pY!CwV{zAW;x*1U+Q#G5h~isS0_5{uM`y$f`ui61s+%7t{SuXWaH;1MRJ>$X za2iwJMxF&;VLP)3`ybHfQu^^fL7#sy;Qs`DAb_8d_yzi0WCec((!ayc&x!x97#ITj zcX~WdXU;j}4!J+3BJHi-y}s&pU_i0u`z9UD=RFFVx_cI(Z}%Z4nO1Nt0&C_;fHU_n z6Ob5WF`O=DV{;G;Cmo_eA?~!x#@!N(zbjO5_`NAikn?z$)nJ}ewy=OphvN~IZpO6r z4Bib~4XwKK{6##r&vey?DwBIWJ4?}Sh+^2ajqa!NWJ1f!>A^;tC%O?rOk4^{)2$2N z3GrpZEQb{qedW0A5j+a-aDnAg*u zVa>^{o4r7B{m2)_NA*?Eef2i*nZJ}6R%oQGxn9?xF8SD7V@V(>G@tL~{7zj$%ceuE zp#)p+)E6S{d=c0SHKVxzFy=O4CrLCWaaDXkT1{IG=~16OgNj1RR;G6mYul*V&oj~* zRD)b!E(|%9D8#XS|J|COZYw6|_dz*w==``2qrHxZC0Ydz$zUBVY_s&qKw09bFIM z>0|cBLKd8C+fCq?tAyj^Ix0vC;O*9sEywxdbNo#C7x({Qk%TYhRsWMC^os!cM=TNy z@avt@i!&q!5c=f@{(gr3&d>huIVQ2+QcPv3L#R0z@!L%wBIbMBJzHCSxgG9>-WV`T}$kOpGy+$(@IO{4g0iV&$3oz zzc$vu;PS3EJmnpNGVZ7GL0XgthUoe93i9ftNlZoq(2K2fGXAEQ2nL~Hnly-u>CWgb zvC`I6*jRjJdc%-nNWs7UXpLWQ#8*a^!J1F`R5Oik@D>z)ZhYUq3y+&`S3*VSA;Mi# zw-zWaYhUqMA6WtIndb_C(PJLhUDqp#cr>}Sh0PzmQ*S(s+V1_*Im09QwB=KaBB?Ul zq+nz-GE7dV!tnGc!F6SjsWEjm`U^{K*bZ_XFZC(4guC{Nyiul-`iI~*)R{CRL4SRY zs}X9F>qX|;PtZ1G(^f*rZVulTj~?sj0~<9cXcHdBizp1pBxQgYm`&hgj`v86YqClg zRBawiO$$2*PwF^(wlko8OTWcx2M1WHAfxW5>geHGefc6THRn9_)MFN<>A=&7%giyV zMIzSC6eizfmg{+M+`d_5%5(M8H>IQybiR3Jzq>|3UZpd44sn9;TzV>J5p<$>60M*& zYLs3=q8XXma%Fc`OkZH!TddABe}X9^??o0LJ1)KrFuPEcjRAvsF20GSZAsA>_tX*U zc0f%pDr_8EMg=g6^pnOQiOtP$Ufto{r}|K+RjX6{WsY{1wT+Sl8%n|zxUyuTNZc>=Zy&%!fx51wi2Zh)Q-g%*UPKW#M zygfwUiajMlG4RB|6KUKbP?84kl==XA!~Xq~*;nuYk6(HEbwTOKmk|efr8I*~ezIDn zN`q$y?u|~Wa}9D>Nf`XJu)0@v4Dlmr!@{6Xiqe38u^e`32A%F3GPXUPZ==)W5P z0tSnK0oH)OWf#!eqgbmvFmR#ArbIJk-#OHn8?p7^0$7% z5HXR9IP*7|h%i+6vM!<^5z))Kh(d%e$^w2L3l@QjUIehe&3BQ?7r8ti1S$l+JQe~K z24A)V0);~Vl))gEZG*tTf9MQ>LBuZW3 N assert abs(ht.fabric_heat_loss_w_per_k - 285.9847) <= 1e-4 +def test_summary_001431_topfloor_extracts_main_property_age_band() -> None: + # Arrange — the gas-boiler-upgrade recommendation "after" Summary + # renders "3.0 Date Built:" glued to its "Main Property" row header + # on one layout line, so the FIRST standalone "Main Property" token + # is the §4 dimensions-table header (followed by "Floor"). The + # extractor must read the age band from the Date-Built block, not the + # first global "Main Property" match — the worksheet lodges age band + # C (1930-1949). + pages = _summary_pdf_to_textract_style_pages(_SUMMARY_001431_TOPFLOOR_PDF) + + # Act + survey = ElmhurstSiteNotesExtractor(pages).extract() + + # Assert + assert survey.construction_age_band == "C 1930-1949" + + def test_summary_000474_mapper_produces_three_building_parts() -> None: # Arrange — cert U985-0001-000474 is a mid-terrace with 3 building # parts (Main + 2 extensions) per the hand-built worksheet fixture From b473f6a1ecfe3bd408ce2879d4cf5a565168c9e7 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Wed, 10 Jun 2026 08:18:51 +0000 Subject: [PATCH 34/87] fix(elmhurst-mapper): classify top-floor flat from roof type, not room-in-roof MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `_elmhurst_dwelling_type` derived a flat's roof exposure from `room_in_roof is not None`, so a top-floor flat whose roof is a plain external "PS Pitched, sloping ceiling" (no room-in-roof) fell through to "Mid-floor flat". The cascade's `_dwelling_exposure` then treats a mid-floor flat's roof as a party ceiling (RdSAP 10 §5 / §3 — party surfaces carry no heat loss) and drops the entire roof term: cert 001431's 105 m² roof at U=2.3 = 241.68 W/K (30) vanished, collapsing (33) fabric heat loss 320.06 → 78.38 and over-rating SAP by ~5 points (on top of the age-band roof-U bug — see prior commit). Read the roof TYPE instead — the dual of the floor's "Another dwelling below" signal. A flat's roof is a party ceiling only when its Elmhurst code is S / A / NR (Same/Another dwelling or Non-residential space above); F / PN / PA / PS are exposed external roofs, so the dwelling is on the top storey. `has_exposed_roof = room_in_roof present OR _elmhurst_roof_is_exposed(roof)` — which is exactly what the function's own docstring already described as the intent ("RR present or external roof"), now implemented. With both upstream fixes the full chain (Summary PDF → extractor → mapper → cert_to_inputs → calculator) reproduces the worksheet's §11a unrounded SAP 56.3649 at abs < 1e-4, with (30)/(33)/(37) matching to the decimal. Only flat fixture reclassified; 000784 (top-floor, RR) and 000910 (ground-floor) unchanged. Regression suite green bar the 3 pre-existing unrelated fails. Co-Authored-By: Claude Opus 4.8 --- .../tests/test_summary_pdf_mapper_chain.py | 39 ++++++++++++++++++ datatypes/epc/domain/mapper.py | 40 ++++++++++++++++--- 2 files changed, 73 insertions(+), 6 deletions(-) diff --git a/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py b/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py index 406729ad..c05f1437 100644 --- a/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py +++ b/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py @@ -180,6 +180,45 @@ def test_summary_001431_topfloor_extracts_main_property_age_band() -> None: assert survey.construction_age_band == "C 1930-1949" +def test_summary_001431_topfloor_flat_classified_as_top_floor() -> None: + # Arrange — the recommendation "after" Summary lodges §6.0 "Position + # of flat in block of flats: Top Floor": floor "A Another dwelling + # below" (party) + roof "PS Pitched, sloping ceiling" (an exposed + # external roof, NOT a room-in-roof). The mapper must classify it + # Top-floor (roof exposed) — not Mid-floor — so the cascade charges + # the roof heat-loss term. + pages = _summary_pdf_to_textract_style_pages(_SUMMARY_001431_TOPFLOOR_PDF) + site_notes = ElmhurstSiteNotesExtractor(pages).extract() + + # Act + epc = EpcPropertyDataMapper.from_elmhurst_site_notes(site_notes) + + # Assert + assert epc.dwelling_type == "Top-floor flat" + + +def test_summary_001431_topfloor_full_chain_sap_matches_worksheet_pdf() -> None: + # Arrange — gas-boiler-upgrade-with-cylinder recommendation "after" + # worksheet (P960-0001-001431). Top-floor flat, PS sloping roof at + # U=2.3 (age C, uninsulated) → (30) roof 241.68 W/K, (33) fabric + # 320.06, (37) HLC 348.76. Worksheet §11a lodges unrounded SAP + # 56.3649. Exercises both upstream fixes: the Date-Built age band + # (roof U 2.3 not 0.4) and the top-floor flat classification (roof + # not dropped). + pages = _summary_pdf_to_textract_style_pages(_SUMMARY_001431_TOPFLOOR_PDF) + site_notes = ElmhurstSiteNotesExtractor(pages).extract() + epc = EpcPropertyDataMapper.from_elmhurst_site_notes(site_notes) + + # Act + result = calculate_sap_from_inputs( + cert_to_inputs(epc, prices=SAP_10_2_SPEC_PRICES) + ) + + # Assert + worksheet_unrounded_sap = 56.3649 + assert abs(result.sap_score_continuous - worksheet_unrounded_sap) < 1e-4 + + def test_summary_000474_mapper_produces_three_building_parts() -> None: # Arrange — cert U985-0001-000474 is a mid-terrace with 3 building # parts (Main + 2 extensions) per the hand-built worksheet fixture diff --git a/datatypes/epc/domain/mapper.py b/datatypes/epc/domain/mapper.py index 9c2587b9..01512ddf 100644 --- a/datatypes/epc/domain/mapper.py +++ b/datatypes/epc/domain/mapper.py @@ -296,6 +296,7 @@ class EpcPropertyDataMapper: built_form=built_form, property_type=property_type, floor=survey.floor, + roof=survey.roof, room_in_roof=survey.room_in_roof, ) @@ -2311,26 +2312,53 @@ _ELMHURST_INSULATION_CODE_TO_SAP10: Dict[str, int] = { } +# Elmhurst roof codes that denote a party ceiling (another/same dwelling +# or non-residential space ABOVE), so the flat's roof is NOT a heat-loss +# surface: S (Same dwelling above), A (Another dwelling above), NR +# (Non-residential space above). Every other roof code (F / PN / PA / PS) +# is an exposed external roof — the dwelling is on the top storey. +_ELMHURST_PARTY_ROOF_CODES: frozenset[str] = frozenset({"S", "A", "NR"}) + + +def _elmhurst_roof_is_exposed(roof: Optional[ElmhurstRoofDetails]) -> bool: + """Whether a flat's roof is an exposed (heat-loss) external roof. + + The dual of the floor's "Another dwelling below" signal: the roof is + a party ceiling only when its Elmhurst code is S / A / NR (a dwelling + or non-residential space above). A plain external roof — including a + "PS Pitched, sloping ceiling" with no room-in-roof — is exposed, so + the flat sits on the top storey.""" + if roof is None: + return False + return _leading_code(roof.roof_type) not in _ELMHURST_PARTY_ROOF_CODES + + def _elmhurst_dwelling_type( *, built_form: str, property_type: str, floor: Optional[ElmhurstFloorDetails], + roof: Optional[ElmhurstRoofDetails], room_in_roof: Optional[ElmhurstRoomInRoof], ) -> str: """Compose `EpcPropertyData.dwelling_type` from the Elmhurst Summary's - property-type + attachment + floor-location + RR presence. + property-type + attachment + floor-location + roof-type + RR presence. For HOUSES: returns `f"{built_form} {property_type.lower()}"` — the historical contract ("Mid-Terrace house", "Detached house"). For FLATS: derives the floor-position prefix ("Top-floor", - "Mid-floor", "Ground-floor") from `floor.location` + RR presence: - - floor lodges "dwelling below" → roof exposed (RR present or - external roof) → Top-floor; roof party (no RR/external) → - Mid-floor; + "Mid-floor", "Ground-floor") from `floor.location` + roof exposure: + - floor lodges "dwelling below" → roof exposed (RR present OR an + external roof type, per `_elmhurst_roof_is_exposed`) → Top-floor; + roof party (dwelling above, no RR) → Mid-floor; - floor not over another dwelling → Ground-floor. + Reading the roof TYPE (not just room-in-roof presence) is the dual of + reading the floor location: a top-floor flat can have a plain external + sloping ceiling and no room-in-roof, which the RR-only test wrongly + routed to Mid-floor (dropping the roof heat-loss term). + The cascade's `_dwelling_exposure` (cert_to_inputs.py) is prefix- matched on the lowercase result; correct flat-prefix detection is the gate for floor / roof party-surface routing (RdSAP 10 §5). @@ -2339,7 +2367,7 @@ def _elmhurst_dwelling_type( return f"{built_form} {property_type.lower()}".strip() floor_loc = (floor.location if floor is not None else "") or "" has_dwelling_below = "dwelling below" in floor_loc.lower() - has_exposed_roof = room_in_roof is not None + has_exposed_roof = room_in_roof is not None or _elmhurst_roof_is_exposed(roof) if has_dwelling_below and has_exposed_roof: position = "Top-floor" elif has_dwelling_below: From 90de1fc976bee9754dc20305dd0062852f3e1c43 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Wed, 10 Jun 2026 08:48:15 +0000 Subject: [PATCH 35/87] fix(elmhurst-mapper): map "Bottled gas" main fuel to bottled LPG, not mains gas MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An LPG-boiler dwelling on the Summary → from_elmhurst_site_notes path mapped to main_fuel_type=26 (mains gas), making it indistinguishable from a mains-gas boiler downstream — wrong Table 12/32 cost / CO2 / PE (bottled LPG is ~10.30 p/kWh vs mains gas 3.48), and it defeats any "non-gas → gas only with a mains-gas connection" gate (an LPG dwelling looks already-gas). Root cause: the recommendation worksheets lodge the boiler carrier as §15.0 "Water Heating Fuel Type: Bottled gas" (§14.0 carries only SAP code 115, a Table 4b gas-family row, + "Main gas: Yes" in §14.2 — a mains-gas CONNECTION, not the heating fuel). "Bottled gas" was absent from `_ELMHURST_MAIN_FUEL_TO_SAP10`, so the §15.0 fuel resolved to None and `_elmhurst_gas_boiler_main_fuel` fell through priority-1 to the mains-gas meter flag → 26. Map "Bottled gas" → 3 (bottled LPG MAIN heating): code 3 routes via `API_FUEL_TO_TABLE_32`/`API_FUEL_TO_TABLE_12` → Table-code 3 (10.30 / 9.46 p/kWh). NOT the legacy "LPG bottled": 5 entry — API code 5 = anthracite, and `canonical_fuel_code` resolves the same-valued Table-32 code 5 to anthracite (3.64 p/kWh), so a 5 here mis-prices the dwelling as cheap solid fuel (verified: a 5 mapping moved SAP the WRONG way, 42.33 → 45.11; code 3 moves it to -6.40 vs the worksheet's -6.6499). Also add 3 to `_GAS_LPG_MAIN_FUEL_CODES` so the §15.0-lodged bottled-LPG water fuel is adopted as the boiler's space-heating carrier (priority 1) instead of the meter flag. Effect: main_fuel_type=3 (bottled LPG) and water_heating_fuel=3 (was None). Mains-gas certs still → 26 (full regression suite green bar the 3 pre-existing unrelated fails); the MissingMainFuelType tripwire still fires for genuinely-undeterminable carriers. Spec: SAP 10.2 Table 12 / RdSAP 10 Table 32 (PDF p.95) — bottled LPG main heating fuel code 3. Co-Authored-By: Claude Opus 4.8 --- .../fixtures/Summary_001431_lpg_boiler.pdf | Bin 0 -> 78774 bytes .../tests/test_summary_pdf_mapper_chain.py | 22 ++++++++++++++++++ datatypes/epc/domain/mapper.py | 18 +++++++++++++- 3 files changed, 39 insertions(+), 1 deletion(-) create mode 100644 backend/documents_parser/tests/fixtures/Summary_001431_lpg_boiler.pdf diff --git a/backend/documents_parser/tests/fixtures/Summary_001431_lpg_boiler.pdf b/backend/documents_parser/tests/fixtures/Summary_001431_lpg_boiler.pdf new file mode 100644 index 0000000000000000000000000000000000000000..2b7a72872eff91ad1e1c95760ba210dddb1b713e GIT binary patch literal 78774 zcmeF)1ymeQqA2_b5+p!^B?M3K;2zxFWr7Xv4ue~O;O;I79%O(7cXto&7Th)HH~d%b z-QDlo@9nlz!{6End!D)RCgS{cBCAV0kOPeK0(LN8?PU5^ zuy`2gMNKUrhIaI#7P=5aVMBdu14DXgLn~ug#H?&g9K5{t5IaL%OXQD!ds;~L%3{%X zSkolBgvr+xY zmx*z+g7Jn7u=={9%EXSV$0MdQ5x=K=u*F2$FiTegD&|ZMjpH`_5(k%vg!p zx{egB2|qL-L2!dkq1GMB8YN!8hsTL;^+^!-;Kwt?AS`fD{oXfDeAWIYa8wQCG^I~0 z`<~Y8=YD&<5b-C59P}9jMXlOdp8!lL`t^XOMjJyHk5x<1#InuO`pgWX%Wozx|Bbd4 z=Yp zM|~Yk8m<(%9>-$(vw2O|;6X2_^-ULLm2gDuWjn>y75wB|H0!!@$DXW(FABVAN7*p zKCJ&QTtzLkOa3yYY-;So3DVw0W;5^D)b85!! z8;MW+ZPfIV(@Pslo+*_PDig?dmDjj6jk}zaU~c}p6Vv5s{f9{#(1CjjB)16JBf3MK ziofgjsug=(>|WEM?Y{V|xS0N%wHn)(+^i4>vb2{LpU|^2d3*UI(1&odC-310ilOYI zr~UN5sqiGBa5!qq*ktZkKQuPb54YJDzt#IZJ9f!w%%cw@XSqN;c|kD0z=rL>m&rwS zxDDDm4jx_H!E;>krf_O091Yk@bR~V0LeprKS^pNJsZllGB`Er1(Q81 z@bgUzuDPEF-;9Tzw-taDy-ug$ih-J%)_WMJ6#M{ABidI z?U9hXZ*LC%A~k*G4k>$9J2twl&QjU%I%JuMVfyr(kkhFLEaJk!vU2ai*Qjh@T~t+=4g<8AB~0KxsMy3q`xNiJelZMvOCBgKYw8isIMts6~t^* zr1B&llqh_|b>w+(U)fXorF`lhZbaceU7OS^yuu{-qZBt((JwxQMVjXX3MwBbl>)P# zGJR`Zw3^6!PlWUQr-eZ!F8wR3Ui36;Gfx)oZ`_~Djb}T5ILFOtOs5!MdqGH2x5srN z5i>Em*&ahiSFsOOv|{d7vc8O1{4hd~uh$!B{*6uJ+U4Z*w1Z=8fHEJ^xdha#I4+$6 zYGw}`xH?d+aGGK7LO_Pf#%X?^`!1dWDyq>M8!^=0C{80Vv0V+3lCN)zQZdB(c)We$ z8-+u)uE+K0j8jdyq9fg-xUis`2$H6ihlhx`J3nEJeJ@Ao4wkr zwyaosL5a?_oObW8r?3(?(W3jUo)ucMv(7J7Wzoo!c}~;xr0$ceT7o|8Qalx#galnV z(rQ?iFDrk;1U^LmCX#74_ABnCda|PJj{bdBEQ4aMupNJu2Lvp@H>r(|chHYeo2zh@ z8FrVgG^l7~)Jn5-xn6j6VB}O0pq@54g@4-d!=812m5G%~%r{ z1b)-g>B=P5uf>9N2ac)r7L-gXzgRTg6)0 zU4;!teYC^0Z@@*)saw1uK_GtJJ|QK~Q|2|%Uj(=Fo^UKty+jp#ucrzaa*`8}pt;&~ zJ5A3}NPSg}Ft?5*iPA}|X}kxxYfgIusO8Vx$a4{lHa>`U-B*>6JO^8Ltx_RPU;f~; zeEY&qf?j?a9bSmHQ8{}}qUKRL67=rCF$PSNFSJ#`xbv)bAzYTN2XU;Wl229n`|qQ% zD+FW~M=j9NhRP_HNkX)`xb`@!Kq=hb*Ac%`a4gtftE}tifwtXD{dPvW;hR7b{2vaS zPx=abD~T9AbgC81h+g|Ln7)2>Je|$9=O*Z7baK5rq!-PjVn>Ssjb1QIL?>B?l;>;a z&#_Tf-}6&F9;;2Kt#7qBz0Nq5iU3`pUUQgu#yijH9aiyja*yIf8<6$8qqs%A9)ustHR{CGMjIRqVc~9KU|>dk zqsAvhqj+HO0r89d=b}S1#`0dA!4ieH)6$~soY-_^&}g@vlB1Q0u6S0WR2f^+7cq6m zy-~jLnOnXyStL4m%MH^yXs{_#J!}@g?$J?{|N^geVKDHc#CY|L%W9 zHpZH9M*9mZV<_qQ&0JT)&x+owblwa$BR`NgT1ew-gVsV6nTuN#qGXNX#d1DWC{YsZ z?k(4(&Nlso#9AXPZDvz5CmL2v*iX+VA&{+tBh!296BV@U)KfQ&3-X0WAHiQ1br`*| zV|7g4rco7++Fu>Cml+f8S}rSEvN8$Lwr=#c#grlOG}#5y@BJu!sk;+{1U+1qE*w61 zvO5Xwys$o^G@Y#{$)RbmP5?D`s_q8rF41u6E*tqNleE2pyN*5noxR;Sx%{r&Rz#zX)0-jVjj<99)uMD8gSgwP-p_hX22o7j z)Vp*V{?~&COizx2=RKb%$#H!*{;|*drnKNyooE!Td2)juRP)SdKA9HmcfL<>C7>|o^WS{%NvGNht^TxPC)?j(D^(b;r5=^*m+NYcMV z_N|ASeaJw+q<^$>m*kgi{BbnxBxmf1E~*o4=;ApeDV%h`%!OI1Xc_Bf1y9AqTf zx@h2ce;4v$#cdI^rZ45Rr*C|Q-?K-S+EBS?(YN!j*Vp>q*EceCE?33y66xm6L~VhA z%h@Vk6^O9==*eM3t!M*A2a8R;V)OoS0IE}f%te;aLZ!uZ_p7z1zjBWb6;<5ws3`gh z*B#Ej=BI~y+P~J8hZkjbT)Zg zPjbRXY-{)@aJ7deA|G)H(n~h#?m2gSwOoV0&iOh!>-wvL(n~o)M!qL{u9DRneSbf9 zPg#bO>v=JF$o|I8)-^$sVP`9+rv!JA9sIj4^Tj9GyAhGj%B$LCx)eUbh*?dNb_0QH z?W=kI42O}pFd0?e#L>JC>?n%yTZZ)wI#cGpE5= zmFbKPZ~wPVvSgV%?j;<4ZwtNZ)P_9GZ+=)Lv`qLmhV{qF;>0Bo)NAEDMSu_GJ-2HR z$4yR_K}0pSEF!?cGS6J-L_y)Q){C% zIl`v~U7H-=0v2#Qd=@b1OGK(}pxjH)qAUj%$%~|O0dS+8zTY~wUyQgNI4E4}{oE10 zZKgx*$wxAi_ea6?5rSa~OhlYcNCOEc8O8H;9{N{}`P^DKn3q4Q1q<>H&%EzFwY<6@ z^hjRgS!ABhjl?;45w%EnXJzwD? zy9g9zE`DuOu(#v#wGFt>`HG`Tigdt?#OGQ5^Xs;N=!kY7VMP|->Jdk<&kd?+3x5x8 zBsz3B>^Me=Y?2pRM0R#<0_GLX5Bwm|f?~zJC9pwJe{BMBP^iYia=N)!n0)3v$97Db zgX#IA0`c>UIIQqrYs+%=c&TYw!)}oqQ3I5$sdRBWRhL@`_#cGju~@|-{Ic@{;a0}e zl}v`d?87eP5)uet6SH$V%K5yK&JE`m;1Aml1h;gCS(&D}A%dr0&g#6^B2~5HzNTc~ z7d5?2FRdr7@rI-N7Hd7D7nkpCv}wLg@^*yeU51xajxu-l!o)oPNX5kv^|!bU7EuwDQW_eSRkq97uSh1kBCBvbuftKGi z`)B=}ZCW!KG}5kh5RGX>ESilnl5|`V6ZJ*Vz3RCTWqCSiWcG`%Ra!R{{;W+clokc& z#j(-FlX`e-c|@)yRwkzGN1s@WrIWluaXB^a1Nq+Ls${>hz0Ze7(*@NTtc-zM}O;hXH^ zzzqg%Lb3PIO*_VGj~>hUZh_lWe#NhXtJp<+l{xQj44se#TtrS-uZtPudy4q!1>*F^ zkNE4qmUW&}dl1xsfPaVTqsOL99fnhq+(|5`myruFnE`0O6-isL$U`O zEVyh(6wNH;!ArV6{d4wRnx{h}zo&S7;O?gc{NZAIUd%vH{5AMJwELDmrD=8*+dL!L zd0hV^Zcd~q6V!d;wMn%HX+av82b#29aD(BpBmW*V*i(OZO%M{ZS=?(P@|%C?4w|n2I+dH z;;NnctGud}cF_Qmp&!9fdOb8x2A0ZUv2{p~{dh9k@6kHgOZ+SG-IrFH72h8H2sBKn zIBAj~Ma+on&9TZlr;PjFTeq3(Lw62?a6-XFl$W!)(E=MSsmI^uG)i;{CL<3)sjKiS zDvB|jZ_?}airol=Eu@ARykQT3phxX2+vu3k=*Bg#T^F|N`M85qzLrehkyEE1=;{DC8x%DgAtYQNIT6b>P*fBTFv>@G$_s(Wp|(+ zr=;GyzSAh?iSL%m#B*5v`!fP6ohl+fd+$svI<(!jx*U#Vg6AVRDr?EzTCLv;myke& zE`;R#i-^43S8^;?Bs2d_jVlp%==gxl5d#^t)v+EKaV>5toJ1k@(2?4d@o~~fh8WsJ zZ+qcuYwt~Iweh)QWC@B>W(n?&Q4;d32DhjP#WEBE@k&SVDi3APPc0oEo2B>fbmhTh zWOAQ2=~kp$l{!qZmCwU>+6uFO+@W(liGOhwHbJISBw#xV!j4YgCX-emqBn*MRZ|}Y zz5JYB&P&g25i&h2W}Sz=+(zTV*LSa(F<7|P*ii6phVvG7KUj{NBKqlVo0s5A|8*m_ z;Vk!*H^4fqK%V}$uxHR5#a(?Xcg_iNGv%-pn#;1Lp<7D^NP-?5y5b*hfvL`Fs10jc z(Btc);Awd#l-18=_wdNV;w#h>a75~3C6qn%S};&=DIy@jIc6Ycs+>ac2Z8?d4?*)Q z8eY0=)QO-i!iv^MvG)5sM@SCI;`}d)w>Etc;Awiq))#`h6fh|d;~$faAYl+g zBA@>F>Ey`o+cM2LMoR5Ev;jq-iMm?gjYW+isu-H%v|bbFR=v|;wwbFSy%Ri#rzsk!^wH*vA#CnqTqS5JGBH`aL-Ly zkoEfEf=}#81RV}GOO;?17Nf_Zrs{ZRV{4_KgAh)Mr)x`#07C!cM4eJOKgLVL0or+F zOs>4)wdr~A-&Jq|=9cdti=)*&LWVVQ|G63X&yC$I|6cPntfl&2Yo2EOC(YCBEUXNF z(>#p`c{3R|o^ClED^u=P&l%)2*}x^6Y7h}KrMSys6+K7R7f-9^*KmQXTI!^LRfk+4 zSjJxJ-GKS-V>7}_I4*(G!0f5I(u7w?j~}Bx8^sA>xw!EaLc8m~XSTn0ySPP&3VZ^O zj72G*$?yG-3$4CENjVxJh0I)9D9Gq|9JkYIJgO)(t6W+ykkQdp6NH~W6)LkdlbENd zqmXi}{eVS?F_&J?!}^Z;AgH$Go+aJOhZU>5zP5BM?rZ4=BNG!759!J&&fMl@t@u)G zGhfoUIa}j!#n{=`(txtkQU(qVjt@+{Of2c`%!8X_eLGcBd!v$T87p$X_8@zdyDHMM zvdC~g%gbMKY!%ps*pn}8qduk2iuLU>hV5|-64q3YVP}_8Fn;(zjfET#5U|lH{G10K zSA&n55Mp$$wsAha!d$YN^7Zj#2lh7gs0{9Pbe{m!}SumHT?u2py^gjWV7q zTJh1Aov^G;oQ@95g5q1#TTV}ZOd*x%nmC&{2)3J?JTt#FpUy;Vp?PQXK1YhDSYu?5oYAUfEJa6ykx4~f@sP!d`rDC)e68&KaZGi0n5=XV`SOj||6?`E$hLg$C#n78bNHEJ(||#&kExviW5e27%PWs#+>+dCv_bhR75q4?fo=yDbDrR z1?gWaO614QMXq)6DY6S2_Pu@DMO^iP)`Y?OsT$~6!|LyC@*KV~s+mcq_=?-AN1(>i zQk#uI5B7xNuO}xbQc_YEb(sy}>+9?BzkaE+)isuc4{a>gI-ry-U*+3CY&Eeyx&?x- zu`b`rQBRfmJQkZw)*hqIV4%G&c)C+Cy>6M-hN|d?5a3+v78)M9a8O>YSyGXsBB*(D zOJ@SrcQir>HCwZZ+%_}U&`{5;(5V&`70uStCE;r=-OXTj>o>Ssh@uW~Z%aW>O1Sp{ z-;z${s8t^zU$i?eJaL9sR8#x-_18t+dv3DltY3dpOO7&7bTC!)4#Ly}hU_ivNX!DYTmShJD{jVDast`ar16cP8el@53Zt zN^@(~TMKh@$q$kJLnar{ido5VPCnD1nsXSboMreumHq6BSQiWrapq4*PRMN3DS8GT zF$XizT-Utasi|2H+N*&N2K6fRmGHQ2LlpH!9dVFr8pA;sFZ0Lee!jza*3N7Ds-!gg zq+E2Q5w4N^Czm+9V~Zm^Ya{LXC;AuJq|s6XP*&t~vSXRb4ID}^6(Xu%V+r9~S`RLi zy{-NDbKg-U-WLURg(#AqRx7eik=81A`PyyfhOHHhX}u&(JA2w1vG=V7SDdDbL1Q65 zdb@Ng5R5oxZBVeFvHD`pey=cEG6lgTFZ3G2`uv!&WJ2iJJqxw3BIFNP+cw7Po?BAi zPnK*;1R=ss81Mwu4DS5-o>RdaI11{>)L5+7gfHw!F0aPT{owsDG0XSkLP4eX-F(dU zPAe>Pu8yh`>mdm?6;p{RaoBN8ia4Obp}vU5m)I|GJXdGVG>z!+Vx`rN>$kVJdvy#n z!-5;JTo`{{g=vy2-4*Tc?}whEx>MIopI4suhe|JJ%G+i8PI}XzBO?TPQBY7wNl9V^ zAJ4c%b3ZGzvrnA-3jI2m&)GdTGfMs2(cJ=d1f(nAol_$+H!w9471g+~Aj_-G!I7U$ zzKAk2JJXEM1eS0o4HzqBc&~T1C{MHClyxgS9!V4$?a<*RB-0ZUEPk#cmmGiRHl%zu zL5(MjV3VwZ7vzA1DQ#;%udomr48 z-~_RAwD~zcIx;p978;nC(ZqHB*{!{e&+RJo>A+x|te6o2ttERc&4SMDWpNRY=KQXyqxv>pezRXQakr6yc%a`9*LfAH3hKkcDn>0(IT7G zZ2u{0_r%!zQs2SUs#r;GEp-C2MOr>xwbsqj+7jPOC7PP`N{AZmeK4Hy7ux zSUqYCDf!I2adURA>dB4aR$7*6YtQ;|yCcrwg9?kkm|B|pC8A9f6xv&!KdbY%%q$G3 zLpajul{a?(N?w}UjY>$0{$4}Cituu4ZOM6kbUnEIc%}90ZvXqrpJR1IAJxF*99gNL z3wJ$z<2DbOZ>{ZJWro|_tgN=q z-kFv`LeDS`!A=^Cr(Mf~!{W;*1|?6?K7Gx69MJaIYf0FENm@o00Y1H}Mkj5J*x69m z3=K`Q#(lcGxm2peUr9*`cMx&8uzZxMr-k;Ux4#`rj1CU|i+A_4Ps2+X`cK>NIQ^Sz zF3!&d1;xXazkkzhf3<~5mgCjb{N}N4=8p2RiqhM>CYNLdB9qRmDEqUMUWp1Ls9DB^ zsmm|+_@Unv;6>%)X2eTkRTXSC|NcP(H5&;9p*4dxKmGLne-$LH%pQRd)jb;4ytX|yhTTI$FKG|U}347&IhDwk|P8&9H zxen@%t`2X8$dAsd&ve%?i!i{`6`Lk-*JFB!Vx2S`NW`V871 zYUtwUeAww=7;I}^FDWB2J~iBgRs!|L#{(&xrq!Yd7Kc8G=n zf51hl-ptZsmDK^Ubg{zV7O|jg+gY0HE{t7T){~#_*!v`k-4eZF^{{8DS~BMOOU#!? z3^X*<=_{e*3A$no~nW zcE5KmQ=t6R*%xWN_Z7<-D0mXu87vrZrsigh-|dx~LU@`BZ=PUfe+C5}_&gcXNLHg< z!8X-=?|XD)TZ$6?#v%UvF0e>t*Vs^$7hRxcpRz_(1%~Lh1 z+*SJRuScQd0#_SwP*@RgQQbiQGM%7YAm^c#R`x}~+2oaQU2s(?qO*$pBY)y|BwT)D z&E6MPdD9-a;1}IP1I%m;$j{ND@CynGM#e_wfBi(2}^1L&d?t8H9%$ zCP;Ilg;Mv@EX`rAV0O#a*zg7#>F=y9UCz6ZVIga!ZV{I3W3D=7bx*{rzG4(tWzzNH zd-Q>fjQp2@ah6QW3tu+sQuAWpQ?uem3*~OtmWYd~q|K`AFzhH#<731uP0CMhVT;OHE!`K)VU7gr8F>r3* zidkGST`lYD@-?XUq4pOJco&oOmeJ*!Z8y5o-)Q?vEGj)YH?MeSeK}}S48pN^z)H@L zvSn)e+r>q=*N~~Dq<(HL_}FeZB~)uKOe-wx`95n?mEogMac6s*pG{oM8hz+<_2pXjTIn#uQ!Nk6=d?bYR1#Da5nf7K9nq=LsfjmdWve+I z$c|hA#o@Jf@L^9{k#;2|WLvb}byDzuG0iU!N^A9(+R)b064_}Ul-`#M1J>Ys?w%Br`g`$_E*|xhkoh}4n{=~N(Kr0jFV2Os4EJ-A8(1N!DTCvDC+me zV4jFT(qErFzhqxN*k|V~{1Uw%H=7jf~Hr&`*WTCmo?mj7Lu&9cA*N}ZLcKxa< ztx*b@>(lYEoxieJd~gAE9&NecK;@QpL^{_^{JO1QHfgSoR&P_KEg2=Z&E~-Y=p}aa zad1x?U^g$k}5}%t6yb)iuLwVHmNrynU&b^h9Ys>Iq|~uSGwwyuWNyqZGVL=;+tKtmM{Dyj{^QZc?P2zkn5rNKc4`>sSh|gOgK$ z|L2!GST`QGo3exAIb`KA4I(Q&@ky^=w=dd%{~m-CBzqoCCp!ox1aovu%#iy=eaR+v zlnH}YMSRb~+sG1+Lj5Gt_mB__mwN5&7K&jVN@-Kgp_+iK>UHcILw*6uh&&`?*JO_=Dq) zs;zF+mvm2OlSJuXODHYv0*S0BO7^8+OG>OD5G)GWP&s84)vc{faAUsCA$-DfR1RZ3 zli0L49?Dm#+KJlNW!x<~nifUzwFrcMkYB%lyGYrGsGrZopXri(5k>G%3Jnbo4)*r0 z^Y}GhWM=uiIPmZpZw5M!jEwAaG&CGs>=e9|cFiV|R~VnH+n1{4DSwVl6hIesX1wH= ze%s$YPVs=$VWj%|XScTxBo=9^%DO2h6=?M}5u$v!%+0Y-ekKKP>yV>kFV$ex*}l26 zNu}8ihpTU-7d-jSEj3oQ@3emAw`pGN8G;8kNWJ)kj8?t7e7fq z3O|bV^q zXE;1Ms+Vm&HG6WQ#$sD$J6Qr287qlY51{is*?1BqcGVrW!@fis*DooEXpL zR<*Sf0TVK3t{dJKZ7h_3prb9*DO!SK6$)ijdCZOrdTsv>o!e#$DN-#d3HG>1^=?NF zDLXeiJ)s=EO{mSNGxzygRxRn#5ZRQvnmyjWI_InWz)~A~Oy=KP7qCZIjuLmFl(lRF z*kcS5^bryL!DTgvU(#}<=|p9RNVfThbJF2zj5jNW1){|_l1BEz0|`b&8H9GXny$y= zBv-g{=1q8@l3jYp4&$!MWJb+~un0Sl!E= z(mq@{vs<>v7qta~D90$fmZs@e`&JNKCtQ#p5_LyM&uiTT14&efTkKpX0n;OBwmyw; zGK9idQ$|AIvUg#Ej^<>4a5?j_EEV62317syB5 zba`~Dc^F&zd%Ty`cNB?@LAqdDTPs7GF;AXft6mjqI~^Z3-A`flnuT&MZl;==2B-p# zGULJ$?8^EM$tr{|CrtsO9D8b^>le9kC$fAWGt8U)ZoQ%dS zw-^c+Jb&Y0$@A*doX;|;>OQ3b2>Uzhm#Lw$-xOUw3toN4DUaeZv+2y9k;~(fp3ghBf?cYcYASB=mf)npf^P zXCXG4sh-;{Dq3q7-gK#XoJMU#*p_kKpOn&cAM!hhf<-%sibRVp4~}_^SrsR~HzBo8 zAEn;R%z}lD1&{cEH#*=YnAMIWx0Bpg@?uX~PO6GUV{(Z2u^_g@BCoI(`ES*C=+v3+ zu+}nU6lBGi7$qe|oyPs=n`;TooKdN5t}YRcnP>4AFfe|6!D@zq^OB0)MoXK8V z^HVp_ISWxP$BEDF<=W&2Hve41((9=ZnGSZ2i;F_~5Y)cIv;^1HRdp7iDFM3t)GoU+3 z`3qZKMJ=Zw*OrmVPFPa-?Ch)sr-h+)Vy)F24U(k@a@!H0^F$0qhxQO<>!jIFbi<_JJf~cTV{~yPjTU!V` z7R5!j1F#101;X@0bK!gr$DQ~PW-W%yM39h>P$z=_n}rUcklKPJcEO7dEGxY%5;Fd_ zfH}YUXUNpsulIP{X)Z*Z!N5l1&JB2XM0qaeT_S!x!(<-R8)OIKtQmBt%i$S z_^r36#n#FZQ{y?eh4_J8rlEV|SjCh>cW=MXF2yy@0yD{73qHn8BZHe5hPo~zBU#~R zBBS@TwA0hWqY`Y6OOd~548tmz^Z?qrZ z$js83M|)90B<&E3Qe5BBu`*_HdS-OA(=z6UBbJ-IlY#?ZHe!^BEc^eh1M(lbB>nFZ zNs}@@|6}LvgE;!HutiM&r1KWAMSv~(Kgx{&TLjo5z!m|v2(U$fEdp#2V2c1-1lS_L z76G;hutk6^0&EdrivU{$*do9d{oiPd*#5od>3`W4G5?e1X}}f%wg|9AfGq-S5nzh| zTLjo5z!m|v2(U$fEdp#2V2c1-1lS_L76G;hutk6^0&EdrivU{$*do9d0k#ORMSv~( zA8(5uhB5lDuthBYr1KWAMSv{=Y!NVS5io9%Juq$&Fm4erZV@nU5io8MFm4erZV@nU z(ck^T|HE5g+#+DyB4FGiVB8{L+#+DyB4FGiVBDhr*5ej&{CmyQ|FSJ&{U^=SfGq-S z5nzh|TLjo5z!m|v2(U$fEdp#2V2c1-1lS_L76G;hutk6^0&EdrivU{$*do9d0k#OR zMSv{=Y!P6K09*7w-WGBGd!4t;Ec8Ow7S?u3HoE$T^umUYruv48VuJL7rVx91LpvdB zOB-t|Ln{a|2feJWr6Iip+dt{N1#A&uivU{$*do9d0k-HXV2c1-1lS_L76G;hutk6^ z0&Ee|LGOKW2^ey@8iiZ5qs|@cwguQCz!m|v2(U$fEdp#2V2l1++agAWf3JP|U&cl3 z|D=5yz(oKq0&o$4ivU~%;35DQ0k{ajMF1`Wa1nru09*v%A^;ZwxCp>S04@S>5rB&T zTm;}E02cwc2*5=EE&^~7fQ$ae<03}Jf3NrUU)Dt&|D^X8&_#eQ0(23eivV2&=psND zfdO3v=psND0lEm#MSv~>bP=G7Jj;K6-4+lX(e5Lx$l_Z);t2KubP=G709^#=B0v`b zy6C^RE@Jxk+Nb|zUBvlM+NS|s1n43_7Xi8m&_#eQ0(23eivV2&=psND0lEm#MSv~> zbP=G709^#=B0v`bx(LukfGz@b5ul3zT?FVNKoHqYGn2m#jm)9O*XQ*q5e0O_ycXx|>b9=LW zwmP=m*1A;KyqMd!UbTEUadCZi2HQvso6fIJ+LueB+U=}Mlt5`hDyjUsIg+324N52D zex1zU-Co|2pI#kR&!*|;iHk;Yibs7=N#O;h@+l_qh(~e4_6Tb=+c;UcyT7};r@Fei z{63MclPxY1!LE|Rr=2BalrL&hAZA=3rkgFKl*B6)!xi0bzjeBDckeZ~GwS?ZODuvz zDTznxiy$nd6;#r$T-vrw%H@ZKZJ`fk@?KKFMefP#V8(wy;Hs zgmaagd#$2Jt?J-f>-Nd7@CHxWI4-XWgRPS_*YA3=@m$ieADl}KgX`_Bp;E5Z3IVO) z+PSRjo8LD#*TX9V3JD}U#<{XXYaKdSV$!i(T4^#*xx;XuP{Of4>>gHB7Fp?Ji-<*W@&&WW#fZzt zio!m4(Fk^-FgDRh4xMaawG`>D`HqLLk$m#kq;9&FUXIwubOBg?J_Imx1u*jlvnnL= z=;eya#q*a8l>EN_{SfvoESs~Nvz9rgA5+D(vqZJCMYOVnb@Rm3(?#?$Gzxn1VF9mU z0UrY6Z6EJI`=K5sZtBS@Dv5Gt*+$9D$rI}nut+ZNkqxa3{#RP-Pw6-vy7q!qewkKw0 z`eUP@ptZ9GEeks*F)ceg3o#=L3j;AD0|T2DFYjN@Teg2pfnLc$58`5DNUv5)h6FUd-!^F(WO3cd2_Q%A*!AZ=<#!k%0$w1VbwvLs9;AIe zU-tR`oBuUDERKhG{+M9tF)}c+{*l;2g6vF8e@qX{Y%qb$4$IiXx`&-v85sTu_mB_= zECYXpg{}W%zdwij>wd6(9+o*-SpFLKLwtY9#KWA04R%1VxE>DhubGFX@t5U4(){xW z!t{^_*z}NxhxtSLe+l=O`5*B-Y=6k>!+x;khde&a|2%2^Q6B!I>=45aoq<^KKgt37 zKbC|42>o{@?O!fr9RC>k|5qVn{JTO%|JU;LzgWK5IDzu@UthjpWf^vYvN8NIe>lM& zeu4de@83gd{+l!8FJ}ZSm4{OPu#KIW{U1yALs9-uCHwEjDwWqYHl&x=wR;$5^x+RV z73pOR4NP?(u9Oe?W#we0V`7Iz&%we-2fG?^Xu&G}RuEWddt#=C0E+aAhW6GDcKU|) z+}wYh7!Nbpb@ky&sz@)&NX+<9++i7@|0t(tW~dK=ZI^(3Fn?^8U?hI{Y+zd-UVpv_ z{}C^&7WEL3sRhIkc4fEFg&6+r^8zCWE5lzO7(V)?sma*Sa=dKV0WBpK;Ei39iVqr^ z-LM{&>>bS{59k`BmBLA+%71>Uc(i%3YkI%VCLH|PLc*dh`rA8HaFZ3w)t5JSY>Kg$ z+R>t^&m^PYhmW#I^%%Ig%JldV+?tCoQY7br?CLH0!f*?b8-U7@k;QOP?9)bIPq^sCTNtu7~55u_Lc?l-nA zq=T3?^b({kRPL#su9~z7GsQg)oZqbqcIM*wLhtdk!Y}Y78wlixLi|1TTRhm!cRW7? z>OT+f(OUAz-9*ZX)qO^GVB_R!YGd$tlO*H)kCbQ)tx1##g4;@Fd@rX};SJIxkXOLm%L`TiU=M*p)$9-d`s;>70Av{3=2|sg3exj4{ zgxe*{A51%u?2aGUrL<+yf6Y=m83n&Zm!4BY%@g-6P6zR`i&3yQgX2#G6O>Tj zqCQs*}98b|Hde);QU_7}qYN1w%=U*EP7gi}A#@hx=h}VVL%==PX+IKYLli5#meCtQ;hk;#8y;5Sk0q(;I9099s%UCN-W7ge(JF8}EoTIvSoMA^g|S zjHq32Z;idi5??tJOl>5po3|)?>R~ttk6z@G`214&;~?^nIL} zTzyl+JHr8&LNPu87ecw!|MO!G3M%S|VfA>it*leLc2uFDzN;lk?d=+d0<#9bQY1qn zVF*&L_U%Uv;|a0O5{7(2HjZ*<9J;DLzMZ`K2{}I!^a;8s3m#3qsl?5x(DSN^0C6`O z-)vsPSnC87@ua4$?rCcJ@O;pcEqdd)%QS&*YzEJHYN_-vW_-_{6xeq&&mlcGrK6>^ z@s@?7@FcrXdFySzzy{mUheW*fZ-qW+9P*j8iDX=xJLPO8KjHnE!xyUxdEQcflYEw7 z+nxzO_0vkX_>E}!GuMS!n|$<+%U@)-xW=c%82cVb)RG*a7g>rfOJ4DW2$Sm3k+$oT ztsB#M18?yyo19Pc_z*L4mRzxY%F_bfsZgJeO+bk=hsZPN5e!37UQ4_V0g*hvPA;cB zNl@G@gzDSr7q);z~_qFS>xk#b8!2Tmq&2Yy?eU-ai7BzdMmR=@XDV; z;j6muo^JFG`&KG31MF~3LBPeHM3|19Ttl4WhP>a|w>kxmsYd$0U8Ezu-H3_Xoq488p# z;T5~&<;oJYX~2jRb*Cx8<$2zz*zw00uPG5V0eaASj+e|Mk^8H+HEQFJoXbkU&3vdj z_1ofg7Bvb#lKTQfYsLjR6$@S{-YxC zw-to{5GVhng24GV6@(mh4X0VQm!6Z*=DzdY^cbdUjQZ1oNfwRCGf_vk^QdUzkT=0L z61~CK5m)q1{Az2>Oz`rYE`dTPDb@_ki>Ul-LlVM4zcEKYVG2m3_7Y2se(s+n;K}{mcF~%K!52j>v;SBP2Wd~?ZO-;Y(@A6956`OUd7#O_ifH65)KJkms zJKUqzwrKrvr*?*n-t|@+mx1nQ2*W=MJnHfKxg34R~KiL5LE4dc792 ziUkt{@O4kkPYBK2JKi|-`L1O{0>|w#@3~0Fl0k=93dm&>5y=B zCzpAEcZf@~7SLW>rq>;c#(uv=KE|(1A@!G8EFeFD3^v4)3t+CN!UN#y&v^qS_bsw zB4bGBw3`4hco|$+~5p#t=0v_Et z5<=YU$cbrST7yx%k4*jelU{gZ`Kgf2R7VUMYLwS&s6aG*RFQ}I3h9-ZDdwgJv~q30 z#{}!!td9lhCU{ug=n|i>-(a0BAlTD38bJXw1#H1AV(e7iOH;_G5DSE7I3|A}IyQ=< z@5+dLo%Xam+-wKLQimM%_F~?7Ksyx~rX5X!)FJe}35bzA;oP|;O&{~wb_a;|84LoB zFUd2bcYvMv9rGfBmxzFt_GNL9ri_QG%3Crbl^n7u=|P@8!5H!xo&gKR>a*);fY{rH zoA;1r2jEsW|BgA2bEV*o7pPHRhIuBBGm@o%lX4}8bF__qAc#6rr{`(RK^_@#%UiOU zsN6TFp(NU?LXLig4n4LlvW2g;^qG4E`A(VlAMw4y?r|PG8kBMKt)+9>UVH!~o5jIN z<_KK9en2KeZ&T98pf##*OLdP3{aV1*pF(EIOHDqo#35*C1d(g_f=M^uvyaDsSRye6 zH$_j&ST1j`IEaR|vtH+5fAwd`(fdx}J!-v&{rC#2La@^Z8lSh^@sst+43^nj^*8#z z&|407(7UA~60UTle{0!IY-iQdim~F-!naR&i3@KZ2^U3QM>;k~Kv8JBt>PM-BkPur zl8=LsZM#KN%S{wsyxlFm z`w5I5-sx9}1Ii)7GDkiPzd2LY3{tl}mjKXOdYhMuAkr=8sq~TZA2`yoqA&(mxe-VV#Bch)gad` zBUgB#qWJUL20O-^f$`A}QIUK>IW@gT7~R~Z-r-V1Ys_bib-hzRS@^r1f02ZoQzieKB;*f*;Xh(2B!Pc$Ct^ql@P9ds zzp@m6N<9Bh5+Z>>{Dq|`HPE;>qfC3oQf%|-A_ogoS$tmTfKx(lB+;$}w`&hs__`P~)dR&pPE; z&*mdKFg+x2SgkecT4ef0r++850f*`K_XZi=tpCo#tx!?}Vy_5HZED)<*LW^)z#`ARSwGp$3`AEG%-8YPzVaVtm#?l~Yypp~a_iN;So#>N1Pl>?FUf zdX~k4yiNO6W2fM4bgyejLd3R5FOIit-)BhFGdM+|oG&TRxzOTU65rdDs_}z)GWs`C z6jrN<2l_2FUMb?y`w-Evv;^*74Gyw%g0&&`j-$2}w}8J$qA-?}Zu`9{dBh&7S%!Dntv(&zfsk@@5V#K4#Ri#!>sfXzzkbZ=XAag@RoqLzSr0fC+T6QAtfO55 zeS2*pdQ?b!F%cd9-fNtz4{5SSu-eJo=4nC`e58imvcEVwdKXX(&lk_*OvVY+RV!XY zRzzl*72%`~v87kT{ME$~1nkWZ0*OW!-U@1lIwh27#A!dSu%+DRW~mLUFZ;G0H+yYH z#pGFZf~GEl4{sP@ko5p29PFy*#PXD6vUDnAKx3ph1OEA_qIYd9ONF#~%DdcL zb}Wo9`&Vk_BtqTZfoa%Pn;J#N>I#)cV1AF0 z*n(G;8Qbm`nZ{VMY(_2JBwKuLiIY-cYazVRD&kh&qLf*Pt>Si=9y@sfFf8fy=g^&| zdwO(EgSTx8PZCDPMtbJh!rv-A0`yvu<8VwL?ma-~(Wm$}bUM$yHoRjZbE=9Uqq1#h zcyuZLweCZ&NrhaEpI&V$-Ql46?#=X)u-3qFDWfml9w~x*@c^2^?gJWV-p5Zn{2yVG zcbu}n8XiYvs;$Li?(ST(w@{7*C(jl#nd6#HXGPVLPdiP5hm&XP892Vd9D*vDF8($+ z5xLfaEEQVDk@qKhU3>rIi+h5!gf8FkxPwIUAfDHihqhg?Ra?9|qXcb0(Qkd(EFp=R zn40b>&kRbM!dR|Qh1#Ioy^<48Uwvc5brcIU_%P~?QrzuumZ0USEm{65rt(yg)LQdM z5EoC8P)bW0T};^vv)|--%`Xt0#vYv(8|!#X+TRU9=l8RBMGegLqMz=Js~KYtzgu@z zj#M;1SiWc>GyjSE?H$wH0QY?!XrmN zw=k*Qd`I>NO!VcKK?Rc~h75d>Ff03~GUdqa8M5+`p}k2hHGSbNc~&;cyM;P8rtIq# ze~5YzZl4m_BtF>t7wB?MNBnQlxd!G{TycVdiMgF$s(;MK-=ZIJEK*@Cz%KiIs=-DT?pYj4Sf0R=-`Mz?b z9~F_}slBYzy6d|@U|>hviJRDb^Wmp@PQDWyc`?H@7_r}A(XM~VQK>H|SawkWfOOO+ z;E(|%G3w-dsVx6wB{8eaRXq&%#!s&$~#=2KN^%@2XZs7;-XbOstUiv)igM$PcK_^BL$&(@19s$ZihChV^dIj)dS z?8^d-*F`x_uDKdR;lnMFEf0J-XgkFRwLem;k^gSpjK~+Kv$0gvLe-cXKQ{~*h0#Gb z$w&=k%|_ZkU1&)g1mz09*`f&Lhjqo^ysSYFH`c!{4rHD&vNQfqD47xaI^7Wz2OSjp zIk`W&%(=wO%dvv^W*b}-6Dx!|BaJN&1vjS)%=48xHXCHM`&Th>PVxM2V&acv=|9T2 z;J`orc6}BU2q5?$>=E)TIKcl@i_X-ff1wxssi*$?P(i>Ue+iX>{Yj@G$=ZR1N<>Jw*G`!9E$|YCHjcL$L;}j3dDqZ{U`?8JNVG+z-QM6lP+OR|SY-kbw?AI_^CAN_SUHJ&G>78ygT4vNTh3s#-^F{thcW zk&ZWw+?>WwdtcI;L(jPz=#&QJ*ayayd@k;&A_Du%~ zk?-y1^ZRyfm3B^0VvYt9M3qzg!Xpn)bxRr=IYx4zN_r(1CK zy1|p(ul9A(k2t6Swc#xRQqw+?K-UkYKHRIU`=LVE!@y;ARpE7;iq{wAklvjFPf${Q zBXOD?UC}OAZ^z1AgUP8?7L@61jU~-#cAUu3vIXB-4DN+q;3T*4m7HjOWx^E_`fcLM z?Nc%dZN}0*QtNMdoz)zZtc9L!1=JrLn>HfrU%W`Z^Blad<4Wwj%{WJTlAx4@GMw3p zLm^)so>qq}kI?ul3*DO}YLH>fpe$rI?ipha?66DjJQZAQv-vyJ<1tp&6lbL%2nx{u zH6RoMk$?cLfq&Z0koflobpF!@IXiyN*}xKKIQ^>)g7}>;|HTHDfSif{=WGZG@w24k zmww_#VjKlZF57kd8Lpl}4_Z}TC3uc-Rui77*rg3{(ZpC9{6wm!9cLHMC_kyip8Mp9IY@E($W-S s_w2l^e&1JOI%xD+LGE0QbyFIsgCw literal 0 HcmV?d00001 diff --git a/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py b/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py index c05f1437..b376f876 100644 --- a/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py +++ b/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py @@ -84,6 +84,7 @@ _SUMMARY_000890_PDF = _FIXTURES / "Summary_000890.pdf" # cert 7800 (two electri _SUMMARY_000565_PDF = _FIXTURES / "Summary_000565.pdf" # cert 000565 (5-bp Elmhurst-only) _SUMMARY_001431_CASE20_PDF = _FIXTURES / "Summary_001431_case20.pdf" # sim case 20 (storage heaters + RR type-2 + wrapped "Double between 2002 and 2021" glazing) _SUMMARY_001431_TOPFLOOR_PDF = _FIXTURES / "Summary_001431_topfloor_flat.pdf" # gas-boiler-upgrade recommendation "after" — top-floor flat, PS sloping roof; exercises the Date-Built age-band + flat-position layout regressions +_SUMMARY_001431_LPG_PDF = _FIXTURES / "Summary_001431_lpg_boiler.pdf" # lpg-boiler recommendation "before" — §14 SAP code 115, §15 "Bottled gas"; exercises the bottled-LPG main-fuel mapping # GOV.UK EPB API JSON for cert 001479 — the API-path counterpart of the # Summary_001479.pdf fixture. Together they drive the API ≡ Summary @@ -180,6 +181,27 @@ def test_summary_001431_topfloor_extracts_main_property_age_band() -> None: assert survey.construction_age_band == "C 1930-1949" +def test_summary_001431_lpg_boiler_maps_main_fuel_to_bottled_lpg() -> None: + # Arrange — the lpg-boiler recommendation "before" Summary lodges + # §14.0 SAP code 115 (a Table 4b gas-family boiler row), §15.0 + # "Water Heating Fuel Type: Bottled gas", and §14.2 "Main gas: Yes". + # The boiler burns bottled LPG, not mains gas; the mapper must + # resolve the carrier from the "Bottled gas" label, NOT default to + # mains gas via the (contradictory) meter flag. Table-route code 3 = + # bottled LPG main heating (Table 32/12 10.30/9.46 p/kWh) — NOT code + # 5, which collides with anthracite (`canonical_fuel_code`). + pages = _summary_pdf_to_textract_style_pages(_SUMMARY_001431_LPG_PDF) + site_notes = ElmhurstSiteNotesExtractor(pages).extract() + + # Act + epc = EpcPropertyDataMapper.from_elmhurst_site_notes(site_notes) + + # Assert + main = epc.sap_heating.main_heating_details[0] + assert main.main_fuel_type == 3 + assert epc.sap_heating.water_heating_fuel == 3 + + def test_summary_001431_topfloor_flat_classified_as_top_floor() -> None: # Arrange — the recommendation "after" Summary lodges §6.0 "Position # of flat in block of flats: Top Floor": floor "A Another dwelling diff --git a/datatypes/epc/domain/mapper.py b/datatypes/epc/domain/mapper.py index 01512ddf..32af358d 100644 --- a/datatypes/epc/domain/mapper.py +++ b/datatypes/epc/domain/mapper.py @@ -4537,6 +4537,17 @@ _ELMHURST_MAIN_FUEL_TO_SAP10: Dict[str, int] = { # existing oddity as "Oil" → 8; both labels are unused by any live # fixture). Live form on Elmhurst worksheets is "Bulk LPG". "Bulk LPG": 27, + # Elmhurst Summary §14.0 / §15.0 lodging form for BOTTLED LPG + # (cylinders) — the recommendation worksheets lodge "Bottled gas" as + # the §15.0 "Water Heating Fuel Type" for an SAP-code-115 boiler. + # 3 = API/epc-codes `main_fuel` code for bottled LPG main heating, + # which routes via `API_FUEL_TO_TABLE_32`/`API_FUEL_TO_TABLE_12` → + # Table-code 3 (bottled LPG main heating, 10.30 / 9.46 p/kWh). NOT + # the legacy "LPG bottled": 5 above — API code 5 = anthracite, and + # `canonical_fuel_code` resolves the same-valued Table-32 code 5 to + # anthracite (3.64 p/kWh), so a 5 here would mis-price the dwelling + # as cheap solid fuel (the cohort-2100 -61-SAP collision class). + "Bottled gas": 3, # Elmhurst Summary §15.0 "Water Heating Fuel Type" labels for the # bio-liquid fuels added to the EES dict above. Values are Table 32 # codes verbatim (no API enum collision). Spec: SAP 10.2 Table 12 @@ -4873,7 +4884,12 @@ _GAS_BOILER_SAP_MAIN_HEATING_CODES: Final[frozenset[int]] = ( # of these so it can't mis-assign electricity from a separate immersion # (where §15.0 lodges the immersion's fuel, not the boiler's) — that # case still strict-raises `MissingMainFuelType` to force a mapper fix. -_GAS_LPG_MAIN_FUEL_CODES: Final[frozenset[int]] = frozenset({1, 5, 6, 7, 26, 27}) +# 3 = bottled LPG main heating ("Bottled gas" label); the other LPG +# carriers are the legacy API LPG codes (5/6/7) + the live "Bulk LPG" +# (27). All count as a gas/LPG carrier so `_elmhurst_gas_boiler_main_fuel` +# adopts a §15.0-lodged bottled-LPG water fuel for the boiler's space- +# heating carrier instead of falling through to the mains-gas meter flag. +_GAS_LPG_MAIN_FUEL_CODES: Final[frozenset[int]] = frozenset({1, 3, 5, 6, 7, 26, 27}) # SAP10 main-fuel code for mains gas (`_ELMHURST_MAIN_FUEL_TO_SAP10` # "Mains gas"). Used when a Table 4b gas boiler's carrier can't be read From 5a74897fed8773c54b4845a7534384868ece2065 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Wed, 10 Jun 2026 10:07:27 +0000 Subject: [PATCH 36/87] =?UTF-8?q?fix(water-heating):=20gate=20DHW=20separa?= =?UTF-8?q?te-timing=20on=20programmer=20+=20boiler=20age=20(RdSAP=2010=20?= =?UTF-8?q?=C2=A710.5)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `_separately_timed_dhw` returned True for any boiler+cylinder+from-main cert, applying the SAP 10.2 Table 2b note b) ×0.9 temperature-factor reduction unconditionally. For the lpg-boiler "before" worksheet (pre- 1998 LPG boiler SAP code 115 + 210 L cylinder, NO cylinder thermostat, control 2113 "Room thermostat and TRVs" — no programmer) this dropped the (53) temperature factor to 0.702 (= 0.60 × 1.3 × 0.9) where the worksheet lodges 0.78 (= 0.60 × 1.3), under-counting cylinder storage loss (55) by ~119 kWh/yr and over-rating SAP by ~0.25. RdSAP 10 §10.5 (PDF p.57) "Hot water separately timed": No programmer, pre-1998 boiler → No Programmer, pre-1998 boiler → Yes Post-1998 boiler → Yes DHW is therefore NOT separately timed only when a pre-1998 boiler is paired with a no-programmer control. Add the two SAP 10.2 Table 4c(2) / Table 4b lookups (controls without a programmer = {2101, 2103, 2111, 2113}; pre-1998 gas/LPG boilers 110-119 + oil 124/125/128) and return False for that combination; every other boiler+cylinder cert keeps the separately-timed default, so the change is confined to old low-control stock and the heating corpus + goldens are unchanged. Effect: the full chain (Summary PDF → extractor → mapper → cert_to_inputs → calculator) now reproduces the lpg-boiler worksheet's §11a unrounded SAP -6.6499 at abs < 1e-4 (was -6.4013). Full regression suite green bar the 3 pre-existing unrelated fails. Co-Authored-By: Claude Opus 4.8 --- .../tests/test_summary_pdf_mapper_chain.py | 22 ++++++++++ .../sap10_calculator/rdsap/cert_to_inputs.py | 41 ++++++++++++++++++- 2 files changed, 62 insertions(+), 1 deletion(-) diff --git a/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py b/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py index b376f876..fb5e8af8 100644 --- a/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py +++ b/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py @@ -202,6 +202,28 @@ def test_summary_001431_lpg_boiler_maps_main_fuel_to_bottled_lpg() -> None: assert epc.sap_heating.water_heating_fuel == 3 +def test_summary_001431_lpg_boiler_full_chain_sap_matches_worksheet_pdf() -> None: + # Arrange — the lpg-boiler "before" worksheet (P960-0001-001431): + # pre-1998 LPG boiler (SAP code 115, eff 61%) + 210 L cylinder, NO + # cylinder thermostat, control 2113 (room thermostat + TRVs, no + # programmer). RdSAP 10 §10.5 (p.57) "Hot water separately timed": + # a no-programmer + pre-1998 boiler is NOT separately timed, so the + # Table 2b temperature factor (53) is 0.78 (= 0.60 × 1.3), not + # 0.702 (× 0.9). Worksheet §11a lodges unrounded SAP -6.6499. + pages = _summary_pdf_to_textract_style_pages(_SUMMARY_001431_LPG_PDF) + site_notes = ElmhurstSiteNotesExtractor(pages).extract() + epc = EpcPropertyDataMapper.from_elmhurst_site_notes(site_notes) + + # Act + result = calculate_sap_from_inputs( + cert_to_inputs(epc, prices=SAP_10_2_SPEC_PRICES) + ) + + # Assert + worksheet_unrounded_sap = -6.6499 + assert abs(result.sap_score_continuous - worksheet_unrounded_sap) < 1e-4 + + def test_summary_001431_topfloor_flat_classified_as_top_floor() -> None: # Arrange — the recommendation "after" Summary lodges §6.0 "Position # of flat in block of flats: Top Floor": floor "A Another dwelling diff --git a/domain/sap10_calculator/rdsap/cert_to_inputs.py b/domain/sap10_calculator/rdsap/cert_to_inputs.py index 395626d8..8a97cae0 100644 --- a/domain/sap10_calculator/rdsap/cert_to_inputs.py +++ b/domain/sap10_calculator/rdsap/cert_to_inputs.py @@ -5150,6 +5150,28 @@ _TABLE_4A_SOLID_FUEL_BOILER_CODES: Final[frozenset[int]] = frozenset( ) +# SAP 10.2 Table 4c(2) boiler controls (21xx) that carry NO programmer / +# time switch: 2101 "No time or thermostatic control", 2103 "Room +# thermostat only", 2111 "TRVs and bypass", 2113 "Room thermostat and +# TRVs". Every other 21xx control includes a programmer (2102/2104/2105/ +# 2106 …) or time-and-temperature zone control (2110/2112). Used by the +# RdSAP 10 §10.5 (PDF p.57) "Hot water separately timed" rule below. +_BOILER_CONTROLS_WITHOUT_PROGRAMMER: Final[frozenset[int]] = frozenset( + {2101, 2103, 2111, 2113} +) + +# SAP 10.2 Table 4b (PDF p.168) gas/LPG/biogas boilers lodged pre-1998 +# (fan-assisted flue 110-114 + balanced/open flue 115-119) plus the +# pre-1998 liquid-fuel boilers (124 pre-1985, 125 1985-1997, 128 combi +# pre-1998). Gas/LPG 101-109 and oil 126/127/129/130 are 1998-or-later. +# Used by the RdSAP 10 §10.5 separate-timing rule: a 1998-or-later boiler +# is always separately timed; a pre-1998 boiler only when a programmer +# is present. +_PRE_1998_BOILER_SAP_CODES: Final[frozenset[int]] = frozenset( + set(range(110, 120)) | {124, 125, 128} +) + + def _separately_timed_dhw( epc: EpcPropertyData, main: Optional[MainHeatingDetail], ) -> bool: @@ -5232,7 +5254,24 @@ def _separately_timed_dhw( # DHW is not separately timed. if epc.sap_heating.water_heating_code not in _WATER_INHERIT_FROM_MAIN_CODES: return False - return bool(epc.has_hot_water_cylinder) + if not epc.has_hot_water_cylinder: + return False + # RdSAP 10 §10.5 (PDF p.57) "Hot water separately timed": + # No programmer, pre-1998 boiler → No + # Programmer, pre-1998 boiler → Yes + # Post-1998 boiler → Yes + # i.e. DHW is NOT separately timed only when a pre-1998 boiler is + # paired with a no-programmer control (Table 4c(2): room-thermostat- + # only / TRV-only). Every other boiler+cylinder cert keeps the + # separately-timed default — so the change is confined to old, low- + # control stock (this lpg-boiler "before" worksheet: code 115 + 2113 + # → (53) temperature factor 0.78, not 0.702). + if ( + main.main_heating_control in _BOILER_CONTROLS_WITHOUT_PROGRAMMER + and main.sap_main_heating_code in _PRE_1998_BOILER_SAP_CODES + ): + return False + return True def _table_2b_note_b_multiplier_applies( From e89b4041c73d469ac857d856392a7a6a3e5574a8 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Wed, 10 Jun 2026 11:09:12 +0000 Subject: [PATCH 37/87] test(efficiency): lock solid-fuel room-heater space eff to Table 4a column (B) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An API audit flagged the solid-fuel room-heater space efficiencies (_SPACE_EFF_BY_CODE 631-636) as reading the "Water" column of SAP 10.2 Table 4a. That was a misread: the two room-heater columns are (A) minimum-for-HETAS-approved and (B) other appliances — BOTH are space efficiency, not space/water. RdSAP defaults to column (B) when HETAS approval is not lodged, which is what these values already hold and what the reference software produces (Elmhurst worksheet "solid fuel 9", SAP code 636 → (206) space efficiency = 70 = column B; flipping to column A 75 broke that pin and three sibling solid-fuel corpus pins). No value change — add a pin test + spec-cited comment so the column-(A)/ (B) distinction is explicit and this misread can't recur. Co-Authored-By: Claude Opus 4.8 --- domain/sap10_ml/sap_efficiencies.py | 8 +++++++- domain/sap10_ml/tests/test_sap_efficiencies.py | 16 ++++++++++++++++ 2 files changed, 23 insertions(+), 1 deletion(-) diff --git a/domain/sap10_ml/sap_efficiencies.py b/domain/sap10_ml/sap_efficiencies.py index 59ce819c..29550950 100644 --- a/domain/sap10_ml/sap_efficiencies.py +++ b/domain/sap10_ml/sap_efficiencies.py @@ -62,7 +62,13 @@ _SPACE_EFF_BY_CODE: Final[dict[int, float]] = { 607: 0.45, 609: 0.58, 610: 0.72, 611: 0.85, 612: 0.20, 613: 0.90, # Room heaters — liquid. 621: 0.55, 622: 0.65, 623: 0.60, 624: 0.70, 625: 0.94, - # Room heaters — solid (column B non-HETAS). + # Room heaters — solid. SAP 10.2 Table 4a (p.167): column (A) is the + # minimum for HETAS-approved appliances, column (B) for other + # appliances. RdSAP defaults to column (B) when approval is not + # lodged (Elmhurst worksheet "solid fuel 9" code 636 → (206)=70 = B), + # so these are the column-(B) space efficiencies: 631 open fire 32, + # 632 +back boiler 50, 633 closed room heater 60, 634 +boiler 65, + # 635 pellet stove 65, 636 +boiler 70. 631: 0.32, 632: 0.50, 633: 0.60, 634: 0.65, 635: 0.65, 636: 0.70, # Room heaters — electric. 691: 1.00, 692: 1.00, 693: 1.00, 694: 1.00, diff --git a/domain/sap10_ml/tests/test_sap_efficiencies.py b/domain/sap10_ml/tests/test_sap_efficiencies.py index 435fae6c..b8245d6c 100644 --- a/domain/sap10_ml/tests/test_sap_efficiencies.py +++ b/domain/sap10_ml/tests/test_sap_efficiencies.py @@ -51,6 +51,22 @@ def test_seasonal_efficiency_ground_source_heat_pump_returns_table4a_value() -> assert result == pytest.approx(2.30, abs=0.005) +def test_seasonal_efficiency_solid_fuel_room_heaters_use_table4a_column_b() -> None: + # Arrange — SAP 10.2 Table 4a (p.167) solid-fuel room heaters give + # column (A) for HETAS-approved appliances and column (B) for other + # appliances. RdSAP defaults to column (B) when HETAS approval is not + # lodged (Elmhurst worksheet "solid fuel 9" code 636 → (206)=70 = B): + # 631 open fire 32, 633 closed room heater 60, 634 with boiler 65, + # 635 pellet stove 65, 636 with boiler 70. + + # Act / Assert + assert seasonal_efficiency(sap_main_heating_code=631) == 0.32 + assert seasonal_efficiency(sap_main_heating_code=633) == 0.60 + assert seasonal_efficiency(sap_main_heating_code=634) == 0.65 + assert seasonal_efficiency(sap_main_heating_code=635) == 0.65 + assert seasonal_efficiency(sap_main_heating_code=636) == 0.70 + + def test_seasonal_efficiency_oil_boiler_returns_table4b_value() -> None: # Arrange — Table 4b, code 126 standard oil 1998+ -> 80% winter. From ba56647401783c8e7b863df7153af5b7d36160cc Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Wed, 10 Jun 2026 18:43:17 +0000 Subject: [PATCH 38/87] fix(heat-network): derive dwelling age band from first non-empty building part MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The GOV.UK API lodges a junk empty leading building part (all fields None) ahead of the real Main Dwelling on some certs. Four sites in cert_to_inputs.py read `sap_building_parts[0].construction_age_band` → got None → silently dropped the dwelling age band. New `_dwelling_age_band` helper takes the first part that lodges a band (a no-op for normal certs where [0] is the Main part). Closes two age-band-keyed defects on the 5 affected certs: - SAP 10.2 Table 12c (p.193): the heat-network Distribution Loss Factor defaulted to the K-or-newer 1.50 instead of the dwelling's true band (cert 8536-0929-6500-0815-7206 is age A → 1.20), inflating distribution loss by 30%. - RdSAP 10 §4.1 Table 5 (p.28): the empty band ("") fell through the age-band branches to the H–M habitable-rooms branch, defaulting in phantom extract fans. The true band A correctly yields 0 fans (bands A–E → 0). Cert 8536: 31.76 → 41.12 vs lodged 39 (was −7.24, now +2.12). API eval mean|err| 1.197 → 1.192, signed −0.229 → −0.218; headline within-0.5 holds at 56.8% (8536 lands at +2.1, a documented overshoot vs the faithful case-31 worksheet — separate slice). Co-Authored-By: Claude Opus 4.8 --- .../sap10_calculator/rdsap/cert_to_inputs.py | 40 ++++++++++++------- .../rdsap/test_cert_to_inputs.py | 34 ++++++++++++++++ 2 files changed, 59 insertions(+), 15 deletions(-) diff --git a/domain/sap10_calculator/rdsap/cert_to_inputs.py b/domain/sap10_calculator/rdsap/cert_to_inputs.py index 8a97cae0..94d5dd44 100644 --- a/domain/sap10_calculator/rdsap/cert_to_inputs.py +++ b/domain/sap10_calculator/rdsap/cert_to_inputs.py @@ -1021,6 +1021,27 @@ def _heat_network_dlf(age_band: Optional[str]) -> float: raise UnmappedSapCode("heat_network_age_band", age_band) +def _dwelling_age_band(epc: EpcPropertyData) -> Optional[str]: + """The dwelling's construction age band, read from the first building + part that lodges one. + + The GOV.UK API can lodge a junk empty leading building part (all + fields absent) ahead of the real Main Dwelling — reading + `sap_building_parts[0]` then yields None and silently drops the age + band (e.g. defaulting the heat-network DLF to the K-or-newer 1.50 + instead of the dwelling's true band). A no-op for normal certs, where + `[0]` is the Main part and already carries a valid band. + """ + return next( + ( + bp.construction_age_band + for bp in epc.sap_building_parts + if bp.construction_age_band + ), + None, + ) + + # SAP 10.2 Table 12 fuel code 50 — "electricity for pumping in # distribution network". Its CO2 / PE factors vary by month per Table # 12d / 12e (= standard-electricity profile); worksheet (372)/(472) @@ -1751,10 +1772,7 @@ def _main_heating_detail_efficiency( else: eff = seasonal_efficiency(main_code, main_category, main_fuel) if _is_heat_network_main(main): - primary_age = ( - epc.sap_building_parts[0].construction_age_band - if epc.sap_building_parts else None - ) + primary_age = _dwelling_age_band(epc) eff = 1.0 / _heat_network_dlf(primary_age) return eff @@ -4726,10 +4744,7 @@ def ventilation_from_cert( # lodged count is below the age-band minimum. The Elmhurst Summary # renders "0" as the form for unknown; the worksheet applies the # default via `max(lodged, table_5_default)`. - age_band = ( - epc.sap_building_parts[0].construction_age_band - if epc.sap_building_parts else "" - ) + age_band = _dwelling_age_band(epc) or "" is_park_home = (epc.property_type or "").strip().lower() == "park home" table_5_fan_default = _rdsap_extract_fans_default( age_band, epc.habitable_rooms_count, is_park_home=is_park_home, @@ -5116,10 +5131,7 @@ def _apply_rdsap_no_water_heating_system_default( """ if epc.sap_heating.water_heating_code != _WHC_NO_WATER_HEATING_SYSTEM: return epc - age_band = ( - epc.sap_building_parts[0].construction_age_band - if epc.sap_building_parts else None - ) + age_band = _dwelling_age_band(epc) band = (age_band or "")[:1].upper() default = _TABLE_29_DEFAULT_CYLINDER_INSULATION_BY_AGE.get(band) if default is None: @@ -6642,9 +6654,7 @@ def cert_to_inputs( # 10-hour) instead of `ALL_OTHER_USES` (0.80) — see # `_pumps_fans_fuel_cost_gbp_per_kwh`. Zero when no MEV is lodged. mev_kwh_for_cost_split = _mev_decentralised_kwh_per_yr_from_cert(epc) - primary_age = ( - epc.sap_building_parts[0].construction_age_band if epc.sap_building_parts else None - ) + primary_age = _dwelling_age_band(epc) # SAP 10.2 Appendix D2.1: if the cert lodges a PCDB index number that # resolves to a Table 105 (gas/oil boilers) record, the PCDB winter diff --git a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py index a0a777d0..e528ec42 100644 --- a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py +++ b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py @@ -234,6 +234,40 @@ def test_heat_network_main_applies_table12c_dlf_to_main_heating_efficiency() -> assert inputs.main_heating_efficiency == pytest.approx(1.0 / 1.41, abs=0.005) +def test_heat_network_dlf_uses_first_non_empty_building_part_age_band() -> None: + # Arrange — the GOV.UK API lodges a junk empty leading building part + # (all fields absent) before the real Main Dwelling. The dwelling age + # band (here A, 1.20 DLF per SAP 10.2 Table 12c) must be read from the + # first NON-empty part, not `sap_building_parts[0]` — otherwise the + # heat-network DLF defaults to the K-or-newer 1.50, inflating the + # distribution loss by 30%. Reproduces cert 8536-0929-6500-0815-7206. + main = MainHeatingDetail( + has_fghrs=False, + main_fuel_type=20, # mains gas (community) + heat_emitter_type=1, + emitter_temperature=1, + main_heating_control=2106, + main_heating_category=6, + sap_main_heating_code=301, + ) + empty_leading_part = make_building_part(construction_age_band="") + main_dwelling_part = make_building_part(construction_age_band="A") + epc = make_minimal_sap10_epc( + total_floor_area_m2=_TYPICAL_TFA_M2, + habitable_rooms_count=4, + country_code="ENG", + sap_building_parts=[empty_leading_part, main_dwelling_part], + sap_heating=make_sap_heating(main_heating_details=[main]), + ) + + # Act + inputs = cert_to_inputs(epc) + + # Assert — age band A → Table 12c DLF = 1.20 → efficiency = 1/1.20, + # NOT the empty-band default DLF 1.50 (1/1.50 = 0.667). + assert abs(inputs.main_heating_efficiency - 1.0 / 1.20) <= 1e-9 + + def test_heat_network_distribution_electricity_per_sap_10_2_appendix_c_3_2() -> None: # Arrange — heat-network main (Table 4a code 301 = community heating, # category 6). SAP 10.2 Appendix C §C3.2 (PDF p.51): distribution From e6543c76ca2868e7aca2c8078df7b9da71ea6f21 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Wed, 10 Jun 2026 19:59:21 +0000 Subject: [PATCH 39/87] fix(water-heating): heat-network DHW with no cylinder uses SAP 10.2 HIU default store, not combi keep-hot MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A heat-network main with DHW from the network and no lodged cylinder was billed the Table 3a keep-hot 600 kWh/yr combi loss (cat 6 sat in `_TABLE_3A_COMBI_LOSS_MAIN_HEATING_CATEGORIES`). A heat network is not a combi boiler — SAP 10.2 §4 line 7702 says combi loss is 0 for non-combi systems. SAP 10.2 p.24 "Heat networks" (c): when neither a PCDB Heat Interface Unit nor a lodged cylinder applies, "a measured loss of 1.72 kWh/day should be used, corrected using Table 2b. This is equivalent to a cylinder of 110 litres and a factory insulation thickness of 50 mm". RdSAP 10 Table 29 (p.56): a cylinder thermostat is assumed present when DHW is from a heat network (Table 2b temperature factor 0.60). New `_apply_heat_network_hiu_default_store` rebinds the 110 L / 50 mm- factory store (thermostat present) onto a heat-network DHW cert with no cylinder and no PCDB index, mirroring `_apply_rdsap_no_water_heating_ system_default`. The injected store routes storage loss (56) ≈ 376.7 kWh/yr (= 1.72 × 0.60 × 365) + primary loss (59) through the existing machinery and zeroes the combi (61) loss via the has_hot_water_cylinder gate. Verified against the user's faithful case-32 worksheet: storage (56) 376.58 vs worksheet 376.94. Cert 8536 storage 0→376.6, combi 600→0. API eval within-0.5 56.8% → 57.0%; signed err −0.218 → −0.205. Reworked `test_heat_network_main_with_hw_from_main_dlf_scales_hot_water_kwh` to assert the DLF scaling directly (fuel ÷ §4 output = 1.41) since the old two-cert baseline premise (both combi-600) no longer holds. Co-Authored-By: Claude Opus 4.8 --- .../sap10_calculator/rdsap/cert_to_inputs.py | 55 ++++++++++ .../rdsap/test_cert_to_inputs.py | 103 ++++++++++++------ 2 files changed, 123 insertions(+), 35 deletions(-) diff --git a/domain/sap10_calculator/rdsap/cert_to_inputs.py b/domain/sap10_calculator/rdsap/cert_to_inputs.py index 94d5dd44..20ac377c 100644 --- a/domain/sap10_calculator/rdsap/cert_to_inputs.py +++ b/domain/sap10_calculator/rdsap/cert_to_inputs.py @@ -5151,6 +5151,56 @@ def _apply_rdsap_no_water_heating_system_default( return replace(epc, has_hot_water_cylinder=True, sap_heating=sap_heating) +# SAP 10.2 p.24 "Heat networks" (c): the default storage loss for heat- +# network DHW with no PCDB HIU and no lodged cylinder is "equivalent to a +# cylinder of 110 litres and a factory insulation thickness of 50 mm". +_HEAT_NETWORK_HIU_DEFAULT_INSULATION_MM: Final[int] = 50 + + +def _apply_heat_network_hiu_default_store( + epc: EpcPropertyData, +) -> EpcPropertyData: + """SAP 10.2 p.24 "Heat networks" (c) — when domestic hot water is + provided by a heat network and neither a PCDB Heat Interface Unit nor + a lodged hot-water cylinder applies: + + "a measured loss of 1.72 kWh/day should be used, corrected using + Table 2b. This is equivalent to a cylinder of 110 litres and a + factory insulation thickness of 50 mm." + + RdSAP 10 Table 29 (PDF p.56) assumes a cylinder thermostat is present + when DHW is from a heat network → Table 2b base temperature factor + 0.60 (no ×1.3 absent-thermostat penalty). + + Mirrors `_apply_rdsap_no_water_heating_system_default`: rebinds the + 110 L / 50 mm-factory store onto `epc` (keeping the community water- + heating code + fuel) so every downstream §4 gate — storage loss (56), + primary loss (59) and combi-loss suppression — sees the spec default. + A heat network is NOT a combi boiler, so the injected cylinder zeroes + the Table 3a keep-hot (61) loss via the `has_hot_water_cylinder` gate + in `_water_heating_worksheet_and_gains`; without this the cascade + wrongly billed a 600 kWh/yr keep-hot loss on heat-network DHW. + + No-op (returns `epc` unchanged) unless the DHW main is a heat network, + no cylinder is lodged, and the network is not in the PCDB (an indexed + network uses the PCDB HIU loss per branch (a)).""" + if epc.has_hot_water_cylinder: + return epc + dhw_main = _water_heating_main(epc) + if not _is_heat_network_main(dhw_main): + return epc + if dhw_main is not None and dhw_main.main_heating_index_number is not None: + return epc + sap_heating = replace( + epc.sap_heating, + cylinder_size=_CYLINDER_SIZE_CODE_NORMAL_110L, + cylinder_insulation_type=_CYLINDER_INSULATION_TYPE_FACTORY, + cylinder_insulation_thickness_mm=_HEAT_NETWORK_HIU_DEFAULT_INSULATION_MM, + cylinder_thermostat="Y", + ) + return replace(epc, has_hot_water_cylinder=True, sap_heating=sap_heating) + + # SAP 10.2 Table 4a solid-fuel boiler sub-rows (PDF p.163) — independent # boilers (151, 153, 155, 159), open-fire + back boiler (156), closed # room heater + back boiler (158), range cooker boiler (160, 161). @@ -6601,6 +6651,11 @@ def cert_to_inputs( # means every downstream helper sees the spec default; the demand # cascade reuses this entry point so it is covered too. epc = _apply_rdsap_no_water_heating_system_default(epc) + # SAP 10.2 p.24 "Heat networks" (c) — heat-network DHW with no PCDB + # HIU and no lodged cylinder uses a default 110 L / 50 mm-factory + # store (storage + primary loss, no combi keep-hot). Rebinds before + # the §4 cascade so every loss gate sees the spec default. + epc = _apply_heat_network_hiu_default_store(epc) dim = dimensions_from_cert(epc) # SAP §3 heat transmission + §2 ventilation cascades — see the # respective `_from_cert` helpers for cert→inputs mapping rules. diff --git a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py index e528ec42..c7e3f2f0 100644 --- a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py +++ b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py @@ -48,6 +48,7 @@ from domain.sap10_calculator.exceptions import ( ) from domain.sap10_calculator.rdsap.cert_to_inputs import ( SAP_10_2_SPEC_PRICES, + _apply_heat_network_hiu_default_store, # pyright: ignore[reportPrivateUsage] _has_suspended_timber_floor_per_spec, # pyright: ignore[reportPrivateUsage] _heat_network_code_302_effective_factor, # pyright: ignore[reportPrivateUsage] _heat_network_community_fuel_code, # pyright: ignore[reportPrivateUsage] @@ -268,6 +269,54 @@ def test_heat_network_dlf_uses_first_non_empty_building_part_age_band() -> None: assert abs(inputs.main_heating_efficiency - 1.0 / 1.20) <= 1e-9 +def test_heat_network_dhw_no_cylinder_applies_sap_10_2_hiu_default_store() -> None: + # Arrange — community heat-network main (Table 4a code 301, cat 6), + # DHW from the network (WHC 901), NO cylinder lodged. SAP 10.2 p.24 + # "Heat networks" (c): when neither a PCDB HIU nor a lodged cylinder + # applies, "a measured loss of 1.72 kWh/day should be used, corrected + # using Table 2b. This is equivalent to a cylinder of 110 litres and a + # factory insulation thickness of 50 mm". RdSAP 10 Table 29: heat- + # network DHW → cylinder thermostat assumed present → Table 2b temp + # factor 0.60. A heat network is NOT a combi boiler, so the Table 3a + # keep-hot combi loss must NOT apply. + main = MainHeatingDetail( + has_fghrs=False, + main_fuel_type=20, # mains gas (community) + heat_emitter_type=1, + emitter_temperature=1, + main_heating_control=2106, + main_heating_category=6, + sap_main_heating_code=301, + ) + part = make_building_part(construction_age_band="A") + epc = make_minimal_sap10_epc( + total_floor_area_m2=_TYPICAL_TFA_M2, + habitable_rooms_count=4, + country_code="ENG", + sap_building_parts=[part], + sap_heating=make_sap_heating( + main_heating_details=[main], water_heating_code=901, + ), + ) + + # Act — preprocess the HIU default store, then run the §4 cascade. + epc_with_store = _apply_heat_network_hiu_default_store(epc) + wh_result, _ = _water_heating_worksheet_and_gains( + epc=epc_with_store, + water_efficiency_pct=100.0, + is_instantaneous=False, + primary_age="A", + pcdb_record=None, + ) + + # Assert — combi keep-hot loss suppressed; storage loss = the SAP p.24 + # default 1.72 kWh/day × Table 2b 0.60 × 365 ≈ 376.7 kWh/yr. + assert wh_result is not None + assert sum(wh_result.combi_loss_monthly_kwh) == 0.0 + expected_storage_kwh = 1.72 * 0.60 * 365.0 + assert abs(sum(wh_result.solar_storage_monthly_kwh) - expected_storage_kwh) <= 1.0 + + def test_heat_network_distribution_electricity_per_sap_10_2_appendix_c_3_2() -> None: # Arrange — heat-network main (Table 4a code 301 = community heating, # category 6). SAP 10.2 Appendix C §C3.2 (PDF p.51): distribution @@ -432,15 +481,13 @@ def test_heat_network_distribution_electricity_none_for_individual_main() -> Non def test_heat_network_main_with_hw_from_main_dlf_scales_hot_water_kwh() -> None: # Arrange — when main heating is a heat network AND water heating - # inherits from main (water_heating_code=901), the HW also incurs - # the network's distribution losses. The water efficiency must be - # overridden to 1/DLF so that the delivered HW kWh (and therefore - # cost/CO2/PE applied to it) reflects q_useful × DLF. - # Compare against a gas-boiler baseline at the same age band: the - # heat-network HW kWh should be greater by the ratio 0.80/(1/DLF) = - # DLF × 0.80 = 0.80 × 1.41 = 1.128 (i.e. ~13% higher) since the - # non-heat-network baseline inherits water efficiency 0.80 from - # the heat-network main's pre-DLF efficiency. + # inherits from main (water_heating_code=901), the HW also incurs the + # network's distribution losses: the water efficiency is overridden to + # 1/DLF, so the delivered HW FUEL kWh = q_useful (the §4 output) × DLF. + # Asserted directly as fuel ÷ output to isolate the DLF scaling from + # the §4 loss structure (the SAP 10.2 p.24 HIU default store now + # supplies storage + primary loss in q_useful — see + # `_apply_heat_network_hiu_default_store`). part = make_building_part(construction_age_band="E") hn_main = MainHeatingDetail( @@ -463,34 +510,20 @@ def test_heat_network_main_with_hw_from_main_dlf_scales_hot_water_kwh() -> None: ), ) - # Comparable gas-boiler baseline that ALSO inherits a 0.80 water - # efficiency through `water_heating_code=901` for direct comparison. - # Use sap_main_heating_code = None so cascade returns 0.80 default. - gas_main = MainHeatingDetail( - has_fghrs=False, - main_fuel_type=26, - heat_emitter_type=1, - emitter_temperature=1, - main_heating_control=2106, - main_heating_category=2, - ) - gas_epc = make_minimal_sap10_epc( - total_floor_area_m2=_TYPICAL_TFA_M2, - habitable_rooms_count=4, - country_code="ENG", - sap_building_parts=[part], - sap_heating=make_sap_heating( - main_heating_details=[gas_main], - water_heating_code=901, - ), + # Act — HW fuel kWh from the full cascade, and the §4 output (q_useful) + # from the worksheet on the same (HIU-store-applied) epc. + hn_hw_fuel = cert_to_inputs(hn_epc).hot_water_kwh_per_yr + wh_result, _ = _water_heating_worksheet_and_gains( + epc=_apply_heat_network_hiu_default_store(hn_epc), + water_efficiency_pct=100.0, + is_instantaneous=False, + primary_age="E", + pcdb_record=None, ) - # Act - hn_hw = cert_to_inputs(hn_epc).hot_water_kwh_per_yr - gas_hw = cert_to_inputs(gas_epc).hot_water_kwh_per_yr - - # Assert — DLF (1.41) for age E × 0.80 baseline / (1/1.41) HN = 1.128. - assert hn_hw / gas_hw == pytest.approx(1.41 * 0.80 / 1.0, abs=0.02) + # Assert — delivered HW fuel = q_useful × DLF; age E → Table 12c 1.41. + assert wh_result is not None + assert abs(hn_hw_fuel / wh_result.output_kwh_per_yr - 1.41) <= 1e-6 def test_hot_water_only_heat_network_whc_950_applies_table12c_dlf() -> None: From 00921f71e83470a6b8ef212ff6dab2ea027583bd Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Wed, 10 Jun 2026 20:12:49 +0000 Subject: [PATCH 40/87] fix(water-heating): heat-network primary loss uses Table 3 h=3 all months MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit SAP 10.2 Table 3 (PDF p.160) verbatim: "For heat networks apply the formula above with p = 1.0 and h = 3 for all months." The primary circulation hours for a heat-network main are fixed at h=3 winter and summer, independent of the cylinder-thermostat / separate-timing lodgement that selects the h=5/h=11 rows for boiler systems. `primary_loss_monthly_kwh` / `primary_circuit_hours_per_day_table_3` gain a `heat_network` flag (→ (3, 3)); `_primary_loss_override` passes `_is_heat_network_main(main)`. p=1.0 was already pinned via `_HEAT_NETWORK_PIPEWORK_INSULATION_FRACTION`; only the hours were wrong. Before, cert 8536 routed through the h=5/3 row because its community biomass DHW fuel (31) collides with electricity code 31, so `_separately_timed_dhw` returned False. The Table 3 heat-network rule overrides that path: 8536 primary loss (59) 335.81 → 273.90, EXACT to the faithful case-32 worksheet (storage (56) 376.58 also matches 376.94). API eval within-0.5 57.0% → 56.9% (one offsetting-error cert crosses out; signed err −0.205 → −0.202). Applied spec-uniformly per the determinism principle — the heat-network primary hours are unambiguous. Co-Authored-By: Claude Opus 4.8 --- .../sap10_calculator/rdsap/cert_to_inputs.py | 6 +++ .../worksheet/water_heating.py | 18 ++++++- .../rdsap/test_cert_to_inputs.py | 48 +++++++++++++++++++ 3 files changed, 70 insertions(+), 2 deletions(-) diff --git a/domain/sap10_calculator/rdsap/cert_to_inputs.py b/domain/sap10_calculator/rdsap/cert_to_inputs.py index 20ac377c..0f8bdbde 100644 --- a/domain/sap10_calculator/rdsap/cert_to_inputs.py +++ b/domain/sap10_calculator/rdsap/cert_to_inputs.py @@ -6154,6 +6154,12 @@ def _primary_loss_override( pipework_insulation_fraction=pipework_p, has_cylinder_thermostat=epc.sap_heating.cylinder_thermostat == "Y", separately_timed_dhw=_separately_timed_dhw(epc, main), + # SAP 10.2 Table 3 (PDF p.160): "For heat networks apply the + # formula above with p = 1.0 and h = 3 for all months." The h=3 + # row applies regardless of the thermostat / separate-timing + # lodgement (and so is robust to the community-fuel-as-electric + # collision that would otherwise route DHW to the h=5 row). + heat_network=_is_heat_network_main(main), ) # SAP 10.2 §12.4.4 (PDF p.36-37): for back-boiler combos summer DHW # comes from an electric immersion, not from the boiler — the boiler diff --git a/domain/sap10_calculator/worksheet/water_heating.py b/domain/sap10_calculator/worksheet/water_heating.py index 52272d84..0c8ebd5c 100644 --- a/domain/sap10_calculator/worksheet/water_heating.py +++ b/domain/sap10_calculator/worksheet/water_heating.py @@ -550,13 +550,21 @@ def primary_circuit_hours_per_day_table_3( *, has_cylinder_thermostat: bool, separately_timed_dhw: bool, + heat_network: bool = False, ) -> tuple[float, float]: - """SAP 10.2 Table 3 (PDF p.159) — hours of primary circulation per - day, returned as `(winter_hours, summer_hours)`: + """SAP 10.2 Table 3 (PDF p.159-160) — hours of primary circulation + per day, returned as `(winter_hours, summer_hours)`: no thermostat → (11, 3) thermostat, not separately timed → ( 5, 3) thermostat, separately timed → ( 3, 3) + + SAP 10.2 Table 3 (PDF p.160) verbatim: "For heat networks apply the + formula above with p = 1.0 and h = 3 for all months." So a heat- + network main uses h=3 winter and summer regardless of the cylinder- + thermostat / separate-timing lodgement. """ + if heat_network: + return (3.0, 3.0) if not has_cylinder_thermostat: return (11.0, 3.0) if separately_timed_dhw: @@ -569,6 +577,7 @@ def primary_loss_monthly_kwh( pipework_insulation_fraction: float, has_cylinder_thermostat: bool, separately_timed_dhw: bool, + heat_network: bool = False, ) -> tuple[float, ...]: """SAP 10.2 §4 line (59)m via Table 3 (PDF p.159): (59)m = n_m × 14 × [{0.0091 × p + 0.0245 × (1 − p)} × h + 0.0263] @@ -576,6 +585,10 @@ def primary_loss_monthly_kwh( hours of primary circulation per day (winter / summer split per `primary_circuit_hours_per_day_table_3`). + `heat_network=True` selects the SAP 10.2 Table 3 (PDF p.160) heat- + network row — h=3 for all months — regardless of the cylinder- + thermostat / separate-timing lodgement. + Returns 12 monthly values in calendar order Jan..Dec. Callers must gate this helper on the spec's zero-loss configurations (combi boilers, integral-vessel HPs, CPSUs, thermal stores ≤ 1.5 m @@ -587,6 +600,7 @@ def primary_loss_monthly_kwh( winter_h, summer_h = primary_circuit_hours_per_day_table_3( has_cylinder_thermostat=has_cylinder_thermostat, separately_timed_dhw=separately_timed_dhw, + heat_network=heat_network, ) return tuple( n * 14.0 * ( diff --git a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py index c7e3f2f0..330fe4ce 100644 --- a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py +++ b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py @@ -317,6 +317,54 @@ def test_heat_network_dhw_no_cylinder_applies_sap_10_2_hiu_default_store() -> No assert abs(sum(wh_result.solar_storage_monthly_kwh) - expected_storage_kwh) <= 1.0 +def test_heat_network_primary_loss_uses_p1_h3_all_months_per_table_3() -> None: + # Arrange — heat-network DHW (Table 4a code 301, cat 6, WHC 901) with + # the SAP p.24 HIU default store applied. SAP 10.2 Table 3 (PDF p.160) + # verbatim: "For heat networks apply the formula above with p = 1.0 and + # h = 3 for all months." So the primary loss is independent of the + # cylinder-thermostat / separately-timed hours (which would give h=5/3) + # and equals Σ n_m × 14 × (0.0091×1.0×3 + 0.0263) = 273.9 kWh/yr. + # Community biomass fuel (31) collides with electricity code 31, so + # `_separately_timed_dhw` would route to h=5/3 — but the Table 3 heat- + # network rule must override that to h=3 regardless of the DHW fuel. + # This reproduces cert 8536-0929-6500-0815-7206. + main = MainHeatingDetail( + has_fghrs=False, + main_fuel_type=31, + heat_emitter_type=1, + emitter_temperature=1, + main_heating_control=2106, + main_heating_category=6, + sap_main_heating_code=301, + ) + part = make_building_part(construction_age_band="A") + epc = make_minimal_sap10_epc( + total_floor_area_m2=_TYPICAL_TFA_M2, + habitable_rooms_count=4, + country_code="ENG", + sap_building_parts=[part], + sap_heating=make_sap_heating( + main_heating_details=[main], + water_heating_code=901, + water_heating_fuel=31, + ), + ) + + # Act + wh_result, _ = _water_heating_worksheet_and_gains( + epc=_apply_heat_network_hiu_default_store(epc), + water_efficiency_pct=100.0, + is_instantaneous=False, + primary_age="A", + pcdb_record=None, + ) + + # Assert — p=1.0, h=3 all months → 365 × 14 × (0.0091×3 + 0.0263). + assert wh_result is not None + expected_primary_kwh = 365.0 * 14.0 * (0.0091 * 3.0 + 0.0263) + assert abs(sum(wh_result.primary_loss_monthly_kwh) - expected_primary_kwh) <= 1e-6 + + def test_heat_network_distribution_electricity_per_sap_10_2_appendix_c_3_2() -> None: # Arrange — heat-network main (Table 4a code 301 = community heating, # category 6). SAP 10.2 Appendix C §C3.2 (PDF p.51): distribution From 3cb2711418fdf753f133164b65c7d5be53d037e5 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Wed, 10 Jun 2026 21:01:05 +0000 Subject: [PATCH 41/87] fix(water-heating): assume cylinder thermostat present for electric/immersion/heat-network DHW (SAP 9.4.9) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit SAP 10.2 §9.4.9 (PDF p.32) verbatim: "A cylinder thermostat should be assumed to be present when the domestic hot water is obtained from a heat network, an immersion heater, a thermal store, a combi boiler or a CPSU." RdSAP 10 Table 29 (p.56) points the no-access default at this rule. The storage-loss Table 2b temperature factor previously read only the lodged `cylinder_thermostat` ("Y") — so an unlodged thermostat always took the ×1.3 absent-penalty, over-stating storage loss by 30%. New `_cylinder_thermostat_present` assumes it present when DHW is from a heat network, WHC 903 (immersion), or a direct-acting electric boiler (SAP code 191 — electric-resistance, immersion-equivalent). Found via the worksheet-folder harness: cert 2474-3059-4202-4496-3200 (Summary path: WHC 901, main SAP 191, electric, no lodged cylinder stat) diverged −1.86 from its dr87 worksheet. The worksheet lodges (53) temperature factor 0.6000 (present) and "add cylinder thermostat (SAP increase too small)" — already assumed present. Fix lands HW output (64) 2701.99 → 2323.88, EXACT to the worksheet; 2474 −1.86 → −0.87 (residual is a separate space-demand fabric thread). No other worksheet in the 47-cert harness moved. API eval within-0.5 56.9% → 57.6%; mean|err| 1.197 → 1.185; signed −0.202 → −0.165. Regression green (only pre-existing fails); goldens + heating corpus unaffected. Co-Authored-By: Claude Opus 4.8 --- .../sap10_calculator/rdsap/cert_to_inputs.py | 38 ++++++++++++- .../rdsap/test_cert_to_inputs.py | 55 +++++++++++++++++++ 2 files changed, 92 insertions(+), 1 deletion(-) diff --git a/domain/sap10_calculator/rdsap/cert_to_inputs.py b/domain/sap10_calculator/rdsap/cert_to_inputs.py index 0f8bdbde..af3448b9 100644 --- a/domain/sap10_calculator/rdsap/cert_to_inputs.py +++ b/domain/sap10_calculator/rdsap/cert_to_inputs.py @@ -6174,6 +6174,42 @@ def _primary_loss_override( return base +def _cylinder_thermostat_present( + epc: EpcPropertyData, main: Optional[MainHeatingDetail], +) -> bool: + """Whether a cylinder thermostat is present for the Table 2b temperature + factor (absent → ×1.3 penalty on the storage loss). + + A lodged "Y" wins. Otherwise SAP 10.2 §9.4.9 (PDF p.32) verbatim: "A + cylinder thermostat should be assumed to be present when the domestic + hot water is obtained from a heat network, an immersion heater, a + thermal store, a combi boiler or a CPSU." RdSAP 10 Table 29 (PDF p.56) + points the no-access default at this rule. So a cylinder heated by an + immersion (WHC 903), a direct-acting electric boiler (SAP code 191 — + electric-resistance, immersion-equivalent), or a heat network gets the + base Table 2b factor (no absent-thermostat ×1.3). + + Cert 2474 (Summary path: WHC 901, main SAP 191, electric, no lodged + cylinder thermostat) is the case: the dr87 worksheet lodges (53) + temperature factor 0.6000 (thermostat present), and the "add cylinder + thermostat" recommendation reads "SAP increase too small" because it + is already assumed present. Without this the cascade applied ×1.3 and + over-stated storage loss by ~378 kWh/yr (SAP −1.86).""" + if epc.sap_heating.cylinder_thermostat == "Y": + return True + dhw_main = _water_heating_main(epc) + if _is_heat_network_main(dhw_main): + return True + if epc.sap_heating.water_heating_code == _WHC_ELECTRIC_IMMERSION: + return True + if ( + dhw_main is not None + and dhw_main.sap_main_heating_code == _DIRECT_ACTING_ELECTRIC_BOILER_CODE + ): + return True + return False + + def _cylinder_storage_loss_override( epc: EpcPropertyData, main: Optional[MainHeatingDetail], @@ -6217,7 +6253,7 @@ def _cylinder_storage_loss_override( volume_l=volume_l, insulation_type=insulation_label, thickness_mm=float(thickness_mm), - has_cylinder_thermostat=sh.cylinder_thermostat == "Y", + has_cylinder_thermostat=_cylinder_thermostat_present(epc, main), # SAP 10.2 Table 2b note b (PDF p.159) verbatim restricts the # ×0.9 multiplier to boiler / warm-air / heat-pump systems — # community heating excluded. Gate via the dedicated helper so diff --git a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py index 330fe4ce..b5497ec8 100644 --- a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py +++ b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py @@ -49,6 +49,7 @@ from domain.sap10_calculator.exceptions import ( from domain.sap10_calculator.rdsap.cert_to_inputs import ( SAP_10_2_SPEC_PRICES, _apply_heat_network_hiu_default_store, # pyright: ignore[reportPrivateUsage] + _cylinder_thermostat_present, # pyright: ignore[reportPrivateUsage] _has_suspended_timber_floor_per_spec, # pyright: ignore[reportPrivateUsage] _heat_network_code_302_effective_factor, # pyright: ignore[reportPrivateUsage] _heat_network_community_fuel_code, # pyright: ignore[reportPrivateUsage] @@ -365,6 +366,60 @@ def test_heat_network_primary_loss_uses_p1_h3_all_months_per_table_3() -> None: assert abs(sum(wh_result.primary_loss_monthly_kwh) - expected_primary_kwh) <= 1e-6 +def test_cylinder_thermostat_assumed_present_per_sap_9_4_9() -> None: + # Arrange — SAP 10.2 §9.4.9 (PDF p.32): "A cylinder thermostat should be + # assumed to be present when the domestic hot water is obtained from a + # heat network, an immersion heater, a thermal store, a combi boiler or + # a CPSU." A direct-acting electric boiler (SAP code 191) heats the + # cylinder by electric resistance (immersion-equivalent) → assumed + # present even with no lodged cylinder thermostat. Reproduces cert + # 2474-3059-4202-4496-3200 (Summary path: WHC 901, main SAP 191). + direct_electric = MainHeatingDetail( + has_fghrs=False, + main_fuel_type=30, # standard electricity + heat_emitter_type=0, + emitter_temperature="NA", + main_heating_control=2106, + main_heating_category=2, + sap_main_heating_code=191, + ) + part = make_building_part(construction_age_band="D") + epc_191 = make_minimal_sap10_epc( + total_floor_area_m2=_TYPICAL_TFA_M2, + habitable_rooms_count=4, + country_code="ENG", + sap_building_parts=[part], + sap_heating=make_sap_heating( + main_heating_details=[direct_electric], + water_heating_code=901, # from main, no lodged cylinder stat + ), + ) + # A gas boiler is NOT in the §9.4.9 list — its cylinder keeps the + # absent-thermostat default (Table 2b ×1.3) when none is lodged. + gas_boiler = MainHeatingDetail( + has_fghrs=False, + main_fuel_type=26, + heat_emitter_type=1, + emitter_temperature=1, + main_heating_control=2106, + main_heating_category=2, + sap_main_heating_code=102, + ) + epc_gas = make_minimal_sap10_epc( + total_floor_area_m2=_TYPICAL_TFA_M2, + habitable_rooms_count=4, + country_code="ENG", + sap_building_parts=[part], + sap_heating=make_sap_heating( + main_heating_details=[gas_boiler], water_heating_code=901, + ), + ) + + # Act / Assert — 191 electric DHW assumes present; gas boiler does not. + assert _cylinder_thermostat_present(epc_191, direct_electric) is True + assert _cylinder_thermostat_present(epc_gas, gas_boiler) is False + + def test_heat_network_distribution_electricity_per_sap_10_2_appendix_c_3_2() -> None: # Arrange — heat-network main (Table 4a code 301 = community heating, # category 6). SAP 10.2 Appendix C §C3.2 (PDF p.51): distribution From 020ac6f22094f12f9860f38eb9361ea11ce5d9c0 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Wed, 10 Jun 2026 21:25:42 +0000 Subject: [PATCH 42/87] fix(elmhurst-mapper): strip wrapped building-part fragment from glazing label MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit pdftotext can wrap the §11 building-part column onto the glazing-TYPE token without an intervening glazing-gap descriptor, e.g. "Double between 2002 and 2021 1st" (the "1st" marks the 1st Extension). The existing trailing-gap fallback only strips the fragment when preceded by "N mm"; the bare ordinal raised UnmappedElmhurstLabel. New `_ELMHURST_GLAZING_LABEL_TRAILING_BP_RE` strips a trailing ordinal ("1st"/"2nd"/…) or "Main" and retries the lookup. No glazing-type key ends in an ordinal or "Main", so it is loss-free. Surfaced by worksheet `simulated case 33` (direct-acting electric boiler + immersion), which previously could not be routed through the Summary cascade. Co-Authored-By: Claude Opus 4.8 --- .../tests/test_summary_pdf_mapper_chain.py | 14 ++++++++++++++ datatypes/epc/domain/mapper.py | 16 ++++++++++++++++ 2 files changed, 30 insertions(+) diff --git a/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py b/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py index fb5e8af8..5a0d47d2 100644 --- a/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py +++ b/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py @@ -1639,6 +1639,20 @@ def test_elmhurst_glazing_label_full_coverage_per_sap10_table_6b() -> None: ) +def test_elmhurst_glazing_label_strips_wrapped_building_part_fragment() -> None: + # Arrange — pdftotext wraps the §11 building-part column (e.g. "1st" + # for the 1st Extension) onto the glazing-TYPE token even when no + # glazing-GAP descriptor ("16 mm") sits between them, so the lodged + # label reads "Double between 2002 and 2021 1st". The fragment is a + # building-part marker, not part of the glazing type — it must be + # stripped so the label resolves to its base code. Worksheet + # `simulated case 33` (direct-acting electric boiler + immersion) + # surfaced this. + # Act / Assert — base "Double between 2002 and 2021" → code 3. + assert _elmhurst_glazing_type_code("Double between 2002 and 2021 1st") == 3 + assert _elmhurst_glazing_type_code("Single glazing 2nd") == 1 + + def test_extension_party_wall_type_read_independently_of_as_main_wall() -> None: # Arrange — RdSAP 10 §3.3: "As Main Wall: Yes" inherits only the # external wall CONSTRUCTION; the party wall type is lodged diff --git a/datatypes/epc/domain/mapper.py b/datatypes/epc/domain/mapper.py index 32af358d..39c3365b 100644 --- a/datatypes/epc/domain/mapper.py +++ b/datatypes/epc/domain/mapper.py @@ -5435,6 +5435,15 @@ _ELMHURST_GLAZING_LABEL_NOISE_SUFFIX_RE: Final[re.Pattern[str]] = re.compile( _ELMHURST_GLAZING_LABEL_TRAILING_GAP_RE: Final[re.Pattern[str]] = re.compile( r"\s+\d+\s*mm\b.*$" ) +# Fallback only: pdftotext can wrap the §11 building-part column onto the +# glazing-TYPE token WITHOUT an intervening glazing-gap descriptor, e.g. +# "Double between 2002 and 2021 1st" (the "1st" marks the 1st Extension). +# The ordinal / "Main" fragment is a building-part marker, not part of the +# glazing type — strip it and retry. No glazing-type key ends in an ordinal +# or "Main", so this is loss-free. Surfaced by `simulated case 33`. +_ELMHURST_GLAZING_LABEL_TRAILING_BP_RE: Final[re.Pattern[str]] = re.compile( + r"\s+(?:\d+(?:st|nd|rd|th)|Main)$" +) def _elmhurst_glazing_type_code(label: Optional[str]) -> int: @@ -5459,6 +5468,13 @@ def _elmhurst_glazing_type_code(label: Optional[str]) -> int: code = _ELMHURST_GLAZING_LABEL_TO_SAP10.get(degapped) if code is not None: return code + # Fallback: strip a trailing wrapped building-part fragment (ordinal / + # "Main") and retry. + debp = _ELMHURST_GLAZING_LABEL_TRAILING_BP_RE.sub("", cleaned).strip() + if debp != cleaned: + code = _ELMHURST_GLAZING_LABEL_TO_SAP10.get(debp) + if code is not None: + return code raise UnmappedElmhurstLabel("glazing_type", label) From 0202b045de337085713cb67200fd32b266df8631 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Wed, 10 Jun 2026 22:01:35 +0000 Subject: [PATCH 43/87] fix(water-heating): 18-/24-hour immersion DHW bills 100% low-rate (Table 12a scope) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit SAP 10.2 Table 12a (PDF p.191) is titled "High-rate fractions for systems using 7-hour and 10-hour tariffs"; its "Immersion water heater" row lists the tariff as "7-hour or 10-hour" only, routing to Table 13. An 18-hour or 24-hour tariff is OUTSIDE the table's scope — it provides at least 18 hours/day at the low rate, more than enough to heat any immersion cylinder off-peak, so the high-rate fraction is 0 (all DHW billed at the low rate). `electric_dhw_high_rate_fraction` previously mapped 18-/24-hour to the 10-hour equations (returning ~0.10 for a 110 L dual immersion) on an over-literal reading of Table 13 Note 1 ("at least 10 hours"). The Elmhurst dr87 worksheet for solid fuel 5 (cert 001431: 18-hour meter, 110 L dual immersion, WHC 903) refutes that: HW (245) high-rate = 0.0 kWh, (246) low-rate = 100%. Table 12a's title bounds the table to the two named tariffs; 18-/24-hour fall outside it. Resolves the Table-13 blocker on the immersion-extractor fix: once the Summary extractor captures the dual immersion, the 18-hour solid-fuel corpus certs stay at high_frac=0 (matching their worksheets) instead of regressing to the 10-hour-column 0.10. API SAP eval unchanged: 57.6% within 0.5, mean|err| 1.185, signed -0.165 (the cached sample has no 18-hour WHC-903 certs; one 24-hour cert shifts sub-threshold). Regression gate green (3 pre-existing fails unrelated). Co-Authored-By: Claude Opus 4.8 --- domain/sap10_calculator/tables/table_13.py | 40 ++++++++++++++----- .../domain/sap10_calculator/test_table_13.py | 23 +++++++---- 2 files changed, 44 insertions(+), 19 deletions(-) diff --git a/domain/sap10_calculator/tables/table_13.py b/domain/sap10_calculator/tables/table_13.py index f96bd71d..c1dabe25 100644 --- a/domain/sap10_calculator/tables/table_13.py +++ b/domain/sap10_calculator/tables/table_13.py @@ -22,12 +22,24 @@ is needed: Single: [(14530 - 762 N) / (1.5 V) - 80 + 10 N] / 100 where V is the cylinder volume (litres) and N is the assumed occupancy -(Appendix J Table 1b). Per Note 2 the result is clamped to [0, 1]. Per -Note 1 the 10-hour equations apply to any tariff providing at least 10 -hours/day at the low rate (so 18-hour and 24-hour use the 10-hour -column). Heat pumps providing water heating only are treated as dual -immersion (Note 1) — out of scope of this helper (callers route those -via Table 12a). +(Appendix J Table 1b). Per Note 2 the result is clamped to [0, 1]. + +Table 12a (PDF p.191) — whose title reads "High-rate fractions for +systems using 7-hour and 10-hour tariffs" — routes the "Immersion water +heater" row to Table 13 for the tariff "7-hour or 10-hour" ONLY. An +18-hour or 24-hour tariff is outside Table 12a/13's scope: it provides +at least 18 hours/day at the low rate, more than enough to heat any +immersion cylinder off-peak, so the high-rate fraction is 0 (all DHW +billed at the low rate). The Elmhurst dr87 worksheet for solid fuel 5 +(cert 001431: 18-hour meter, 110 L dual immersion, WHC 903) confirms +this — HW (245) high-rate = 0.0 kWh, (246) low-rate = 100%. (An earlier +reading mapped 18-/24-hour to the 10-hour column via Note 1's "at least +10 hours"; the worksheet refutes it — Table 12a's title bounds the table +to the two named tariffs.) + +Heat pumps providing water heating only are treated as dual immersion +(Note 1) — out of scope of this helper (callers route those via Table +12a). """ from __future__ import annotations @@ -47,13 +59,19 @@ def electric_dhw_high_rate_fraction( `single_immersion` selects the single- vs dual-immersion equation (RdSAP10 §10.5 p.54: an immersion is assumed dual on a dual meter). - The 7-hour tariff uses the 7-hour equations; every other off-peak - tariff (10/18/24-hour, all >= 10 hours low-rate per Note 1) uses the - 10-hour equations. STANDARD has no off-peak split and is rejected — - callers must early-return before this fires. + The 7-hour tariff uses the 7-hour equations; the 10-hour tariff uses + the 10-hour equations. The 18-hour and 24-hour tariffs are outside + Table 12a/13's "7-hour and 10-hour" scope (PDF p.191 title) — they + provide >= 18 hours/day at the low rate, so the high-rate fraction is + 0. STANDARD has no off-peak split and is rejected — callers must + early-return before this fires. """ if tariff is Tariff.STANDARD: raise ValueError("Table 13 high-rate fraction is undefined for STANDARD") + if tariff in (Tariff.EIGHTEEN_HOUR, Tariff.TWENTY_FOUR_HOUR): + # Outside Table 12a's 7-hour/10-hour scope — >= 18 h/day low rate + # heats the cylinder entirely off-peak (high-rate fraction 0). + return 0.0 v = cylinder_volume_l n = occupancy_n if tariff is Tariff.SEVEN_HOUR: @@ -62,7 +80,7 @@ def electric_dhw_high_rate_fraction( else: fraction = ((6.8 - 0.024 * v) * n + 14 - 0.07 * v) / 100 else: - # >= 10 hours/day at the low rate (10/18/24-hour) — Note 1. + # 10-hour tariff (18-/24-hour handled above as out-of-scope). if single_immersion: fraction = ((14530 - 762 * n) / (1.5 * v) - 80 + 10 * n) / 100 else: diff --git a/tests/domain/sap10_calculator/test_table_13.py b/tests/domain/sap10_calculator/test_table_13.py index d6857cc2..1063fe15 100644 --- a/tests/domain/sap10_calculator/test_table_13.py +++ b/tests/domain/sap10_calculator/test_table_13.py @@ -83,21 +83,28 @@ def test_table_13_large_cylinder_single_immersion_clamps_to_zero() -> None: assert fraction == 0.0 -def test_table_13_eighteen_hour_uses_ten_hour_column() -> None: - # Arrange — SAP 10.2 Table 13 Note 1 (PDF p.197): the table applies - # "for tariffs providing at least 10 hours ... at the low rate", so an - # 18-hour tariff resolves to the 10-hour equations, not a separate - # column. +def test_table_13_eighteen_and_twenty_four_hour_bill_full_low_rate() -> None: + # Arrange — SAP 10.2 Table 12a (PDF p.191) is titled "High-rate + # fractions for systems using 7-hour and 10-hour tariffs"; its + # "Immersion water heater" row lists the tariff as "7-hour or 10-hour" + # only, routing to Table 13. An 18-hour / 24-hour tariff is OUTSIDE the + # table's scope: it provides at least 18 hours/day at the low rate, more + # than enough to heat any immersion cylinder off-peak, so the high-rate + # fraction is 0 (100% billed at the low rate). The Elmhurst dr87 + # worksheet for solid fuel 5 (cert 001431: 18-hour meter, 110 L dual + # immersion, WHC 903) bills HW (245) high-rate = 0.0 kWh, (246) low-rate + # = 100% — confirming high_frac = 0 for an 18-hour immersion DHW. # Act eighteen = electric_dhw_high_rate_fraction( cylinder_volume_l=110.0, occupancy_n=_N_AT_TFA_100, single_immersion=False, tariff=Tariff.EIGHTEEN_HOUR, ) - ten = electric_dhw_high_rate_fraction( + twenty_four = electric_dhw_high_rate_fraction( cylinder_volume_l=110.0, occupancy_n=_N_AT_TFA_100, - single_immersion=False, tariff=Tariff.TEN_HOUR, + single_immersion=False, tariff=Tariff.TWENTY_FOUR_HOUR, ) # Assert - assert abs(eighteen - ten) <= 1e-9 + assert eighteen == 0.0 + assert twenty_four == 0.0 From 85d6f8468cf604867ce382384901f34ef8382ca0 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Wed, 10 Jun 2026 22:16:21 +0000 Subject: [PATCH 44/87] feat(elmhurst-extractor): capture section 15.1 Immersion Heater (Dual/Single) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Elmhurst Summary section 15.1 "Hot Water Cylinder" block lodges "Immersion Heater: Dual" / "Single"; the extractor dropped it, so the Summary path left immersion_heating_type = None while the API path already captured it. Capturing it drives SAP Table 13's high-rate-fraction DHW-cost split (RdSAP 10 section 10.5 p.54: 1 = dual, 2 = single) and brings the two front-ends to parity. Three-file change: WaterHeating.immersion_type field + _extract_water_heating parse (scoped to the 15.1..15.2 slice) + _elmhurst_immersion_type_code mapper (strict-raise on an unmapped label, mirroring _elmhurst_cylinder_insulation_code). Safe to land now that the preceding commit zeroes the high-rate fraction for 18-/24-hour tariffs: the 20 solid-fuel corpus certs (solid fuel 4-11: WHC 903 dual immersion, 18-hour meter, 110 L) carry a dual immersion, but their 18-hour tariff bills 100% low-rate per Table 12a's 7-/10-hour scope — so they stay EXACT instead of regressing to the 10-hour-column ~0.10. 7-/10-hour Summary immersion certs now correctly cost the Table 13 high-rate fraction instead of falling to the immersion=None 100%-low default. Regression gate green (3 pre-existing fails unrelated); API gauge unchanged (Summary-path-only): 57.6% within 0.5, mean|err| 1.185. Co-Authored-By: Claude Opus 4.8 --- .../documents_parser/elmhurst_extractor.py | 4 +++ .../tests/test_heating_systems_corpus.py | 18 ++++++++++ .../tests/test_summary_pdf_mapper_chain.py | 32 +++++++++++++++++ datatypes/epc/domain/mapper.py | 34 +++++++++++++++++++ datatypes/epc/surveys/elmhurst_site_notes.py | 5 +++ 5 files changed, 93 insertions(+) diff --git a/backend/documents_parser/elmhurst_extractor.py b/backend/documents_parser/elmhurst_extractor.py index e2a5f1e7..87075f59 100644 --- a/backend/documents_parser/elmhurst_extractor.py +++ b/backend/documents_parser/elmhurst_extractor.py @@ -1531,6 +1531,9 @@ class ElmhurstSiteNotesExtractor: if cylinder_thermostat is None: if "Cylinder thermostat (Already installed)" in self._lines: cylinder_thermostat = True + # §15.1 "Immersion Heater" lodging ("Dual" / "Single"). Scoped to + # the §15.1..§15.2 slice so the lookup can't collide elsewhere. + immersion_type = self._local_val(cylinder_lines, "Immersion Heater") return WaterHeating( water_heating_code=self._str_val("Water Heating Code"), water_heating_sap_code=self._int_val("Water Heating SapCode"), @@ -1540,6 +1543,7 @@ class ElmhurstSiteNotesExtractor: cylinder_insulation_label=cylinder_insulation_label, cylinder_insulation_thickness_mm=cylinder_insulation_thickness_mm, cylinder_thermostat=cylinder_thermostat, + immersion_type=immersion_type, ) def _extract_baths_and_showers(self) -> BathsAndShowers: diff --git a/backend/documents_parser/tests/test_heating_systems_corpus.py b/backend/documents_parser/tests/test_heating_systems_corpus.py index ab7889e4..da85136a 100644 --- a/backend/documents_parser/tests/test_heating_systems_corpus.py +++ b/backend/documents_parser/tests/test_heating_systems_corpus.py @@ -960,6 +960,24 @@ def test_heating_systems_corpus_residual_matches_pin( ) +def test_solid_fuel_5_captures_section_15_1_dual_immersion() -> None: + # Arrange — solid fuel 5 (cert 001431: House Coal main SAP 153, WHC 903 + # electric immersion HW, 18-hour meter, 110 L Normal cylinder). The + # Elmhurst Summary §15.1 "Hot Water Cylinder" block lodges "Immersion + # Heater: Dual". The extractor must surface it and the mapper map it to + # the SAP10 `immersion_heating_type` code 1 (dual) per RdSAP 10 §10.5. + summary_pdf, _p960 = _variant_paths('solid fuel 5') + pages = _summary_pdf_to_textract_style_pages(summary_pdf) + + # Act + site_notes = ElmhurstSiteNotesExtractor(pages).extract() + epc = EpcPropertyDataMapper.from_elmhurst_site_notes(site_notes) + + # Assert + assert site_notes.water_heating.immersion_type == "Dual" + assert epc.sap_heating.immersion_heating_type == 1 + + def test_oil_6_no_room_thermostat_applies_table_4c2_minus_5pp_space_efficiency() -> None: # Arrange — oil 6 (B30K standard liquid-fuel boiler, Table 4b code # 126 winter 80 / summer 68) lodges "Main Heating Controls Sap: SAP diff --git a/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py b/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py index 5a0d47d2..f89c4c72 100644 --- a/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py +++ b/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py @@ -43,6 +43,7 @@ from datatypes.epc.domain.mapper import ( UnmappedApiCode, UnmappedElmhurstLabel, _elmhurst_glazing_type_code, # pyright: ignore[reportPrivateUsage] + _elmhurst_immersion_type_code, # pyright: ignore[reportPrivateUsage] ) from domain.sap10_calculator.calculator import calculate_sap_from_inputs from domain.sap10_calculator.rdsap.cert_to_inputs import ( @@ -1539,6 +1540,37 @@ def test_summary_mapper_raises_on_unmapped_cylinder_insulation_label() -> None: assert excinfo.value.value == "Polyester wool" +def test_elmhurst_immersion_type_code_maps_dual_and_single() -> None: + # Arrange — Elmhurst Summary §15.1 "Immersion Heater" lodges "Dual" + # or "Single". RdSAP 10 §10.5 (PDF p.54): an immersion is "assumed + # dual" on a dual/off-peak meter; the SAP10 cascade code is 1 = dual, + # 2 = single (cert_to_inputs `_IMMERSION_TYPE_DUAL`). + + # Act + dual = _elmhurst_immersion_type_code("Dual", cylinder_present=True) + single = _elmhurst_immersion_type_code("Single", cylinder_present=True) + no_cylinder = _elmhurst_immersion_type_code("Dual", cylinder_present=False) + absent = _elmhurst_immersion_type_code(None, cylinder_present=True) + + # Assert + assert dual == 1 + assert single == 2 + assert no_cylinder is None + assert absent is None + + +def test_elmhurst_immersion_type_code_raises_on_unmapped_label() -> None: + # Arrange — a lodged §15.1 "Immersion Heater" label outside the + # {Dual, Single} set must strict-raise (mirror of the cylinder-size / + # cylinder-insulation helpers) rather than silently drop the field. + + # Act / Assert + with pytest.raises(UnmappedElmhurstLabel) as excinfo: + _elmhurst_immersion_type_code("Triple", cylinder_present=True) + assert excinfo.value.field == "immersion_type" + assert excinfo.value.value == "Triple" + + def test_all_seven_ashp_cohort_certs_extract_without_unmapped_label_raise() -> None: # Arrange — coverage forcing function: every cohort cert must # extract through `from_elmhurst_site_notes` without triggering an diff --git a/datatypes/epc/domain/mapper.py b/datatypes/epc/domain/mapper.py index 39c3365b..e5c794c8 100644 --- a/datatypes/epc/domain/mapper.py +++ b/datatypes/epc/domain/mapper.py @@ -5167,6 +5167,15 @@ _ELMHURST_CYLINDER_INSULATION_LABEL_TO_SAP10: Dict[str, int] = { } +# Elmhurst §15.1 "Immersion Heater" label → SAP10 `immersion_heating_type` +# cascade code. RdSAP 10 §10.5 (PDF p.54) + cert_to_inputs +# `_IMMERSION_TYPE_DUAL`/`_IMMERSION_TYPE_SINGLE`: 1 = dual, 2 = single. +_ELMHURST_IMMERSION_TYPE_LABEL_TO_SAP10: Dict[str, int] = { + "Dual": 1, + "Single": 2, +} + + # Elmhurst §15.0 "Water Heating Fuel Type" labels that route to solid- # fuel Table 32 codes (Anthracite, House coal, Wood logs/pellets, etc.). # Used by `_resolve_elmhurst_inaccessible_cylinder_size` to detect the @@ -5282,6 +5291,23 @@ def _elmhurst_cylinder_insulation_code( return code +def _elmhurst_immersion_type_code( + immersion_type_label: Optional[str], cylinder_present: bool, +) -> Optional[int]: + """Map an Elmhurst §15.1 "Immersion Heater" label ("Dual" / "Single") + to the SAP10 `immersion_heating_type` cascade code (1 = dual, 2 = + single per RdSAP 10 §10.5 p.54). Returns None when no cylinder is + present or the label is genuinely absent. Raises `UnmappedElmhurstLabel` + when the label IS lodged but isn't in the mapping dict — same + strict-fallback as `_elmhurst_cylinder_insulation_code`.""" + if not cylinder_present or immersion_type_label is None: + return None + code = _ELMHURST_IMMERSION_TYPE_LABEL_TO_SAP10.get(immersion_type_label) + if code is None: + raise UnmappedElmhurstLabel("immersion_type", immersion_type_label) + return code + + def _resolve_elmhurst_inaccessible_cylinder_insulation( age_band: str, ) -> tuple[int, int]: @@ -5850,6 +5876,14 @@ def _map_elmhurst_sap_heating(survey: ElmhurstSiteNotes) -> SapHeating: and survey.water_heating.cylinder_thermostat is not None else None ), + # §15.1 "Immersion Heater" (Dual / Single) → SAP10 + # `immersion_heating_type` code (1 = dual, 2 = single). Drives the + # Table 13 high-rate-fraction split for WHC-903 electric immersion + # DHW on a 7-/10-hour off-peak tariff (18-/24-hour bill 100% low). + immersion_heating_type=_elmhurst_immersion_type_code( + survey.water_heating.immersion_type, + survey.water_heating.hot_water_cylinder_present, + ), water_heating_code=survey.water_heating.water_heating_sap_code, water_heating_fuel=water_heating_fuel, secondary_heating_type=mh.secondary_heating_sap_code, diff --git a/datatypes/epc/surveys/elmhurst_site_notes.py b/datatypes/epc/surveys/elmhurst_site_notes.py index 2fa55acc..16464766 100644 --- a/datatypes/epc/surveys/elmhurst_site_notes.py +++ b/datatypes/epc/surveys/elmhurst_site_notes.py @@ -367,6 +367,11 @@ class WaterHeating: # §15.1 "Cylinder Thermostat" lodging (Yes / No). False or absent # keeps the cascade's no-thermostat Table 2b temperature factor. cylinder_thermostat: Optional[bool] = None + # §15.1 "Immersion Heater" lodging ("Dual" / "Single"). Drives the + # SAP10 `immersion_heating_type` code (1 = dual, 2 = single) used by + # the Table 13 high-rate-fraction DHW-cost split. None when no + # cylinder is present or the line is absent. + immersion_type: Optional[str] = None @dataclass From b0a47cda05c3ad958929649354931bf20d414c40 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Thu, 11 Jun 2026 07:07:08 +0000 Subject: [PATCH 45/87] fix(elmhurst-mapper): strip interleaved Alternative-wall fragments from glazing label MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit When a property lodges an Alternative Wall, pdftotext interleaves the §11 "Location" column ("Alternative wall 1") into the wrapped glazing-TYPE cell, producing labels like "Double between 2002 Alternative wall and 2021 1 Alternative wall" (cert 001431 storage-heater variants, simulated case 34). The existing greedy trailing-suffix strip (\s+Alternative wall.*$) truncates at the FIRST "Alternative wall", losing "and 2021" and yielding the unmatchable "Double between 2002". Added a fallback that removes EVERY " wall [n]" fragment and any stray 1-2 digit location index from the raw label, then retries the lookup. Loss-free: no glazing-type key contains a wall-location phrase or a bare 1-2 digit number (install-date years are 4 digits). Unblocks the Summary cascade for any property with an Alternative Wall; Summary-path only (the API path receives structured glazing codes, so the API gauge is unaffected). Regression gate green (1 pre-existing fail unrelated). Co-Authored-By: Claude Opus 4.8 --- .../tests/test_summary_pdf_mapper_chain.py | 17 ++++++++++++ datatypes/epc/domain/mapper.py | 27 +++++++++++++++++++ 2 files changed, 44 insertions(+) diff --git a/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py b/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py index f89c4c72..a495addd 100644 --- a/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py +++ b/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py @@ -1540,6 +1540,23 @@ def test_summary_mapper_raises_on_unmapped_cylinder_insulation_label() -> None: assert excinfo.value.value == "Polyester wool" +def test_elmhurst_glazing_type_code_strips_interleaved_alternative_wall() -> None: + # Arrange — when a property lodges an Alternative Wall (cert 001431 + # storage-heater variants, "simulated case 34"), pdftotext interleaves + # the §11 "Alternative wall 1" location column into the wrapped + # glazing-type cell, e.g. "Double between 2002 Alternative wall and 2021 + # 1 Alternative wall". The wall-location fragments are not part of the + # glazing type — the helper must recover "Double between 2002 and 2021". + + # Act + code = _elmhurst_glazing_type_code( + "Double between 2002 Alternative wall and 2021 1 Alternative wall" + ) + + # Assert + assert code == _elmhurst_glazing_type_code("Double between 2002 and 2021") + + def test_elmhurst_immersion_type_code_maps_dual_and_single() -> None: # Arrange — Elmhurst Summary §15.1 "Immersion Heater" lodges "Dual" # or "Single". RdSAP 10 §10.5 (PDF p.54): an immersion is "assumed diff --git a/datatypes/epc/domain/mapper.py b/datatypes/epc/domain/mapper.py index e5c794c8..d6573aac 100644 --- a/datatypes/epc/domain/mapper.py +++ b/datatypes/epc/domain/mapper.py @@ -5470,6 +5470,21 @@ _ELMHURST_GLAZING_LABEL_TRAILING_GAP_RE: Final[re.Pattern[str]] = re.compile( _ELMHURST_GLAZING_LABEL_TRAILING_BP_RE: Final[re.Pattern[str]] = re.compile( r"\s+(?:\d+(?:st|nd|rd|th)|Main)$" ) +# Fallback only: when a property lodges an Alternative Wall, pdftotext +# INTERLEAVES the §11 location column ("Alternative wall 1") into the +# wrapped glazing-TYPE cell, e.g. "Double between 2002 Alternative wall +# and 2021 1 Alternative wall" (cert 001431 storage-heater variants, +# `simulated case 34`). The greedy trailing-suffix strip truncates at the +# first "Alternative wall" (losing "and 2021"), so remove EVERY +# wall-location fragment + any stray 1-2 digit location index globally and +# retry. Loss-free: no glazing-type key contains a wall-location phrase or +# a bare 1-2 digit number (install-date years are 4 digits). +_ELMHURST_GLAZING_LABEL_EMBEDDED_WALL_RE: Final[re.Pattern[str]] = re.compile( + r"\s*(?:External|Alternative|Party)\s+wall(?:\s+\d+)?" +) +_ELMHURST_GLAZING_LABEL_STRAY_LOCATION_DIGIT_RE: Final[re.Pattern[str]] = re.compile( + r"\b\d{1,2}\b" +) def _elmhurst_glazing_type_code(label: Optional[str]) -> int: @@ -5501,6 +5516,18 @@ def _elmhurst_glazing_type_code(label: Optional[str]) -> int: code = _ELMHURST_GLAZING_LABEL_TO_SAP10.get(debp) if code is not None: return code + # Fallback: remove INTERLEAVED wall-location fragments from the raw + # label (Alternative/External/Party wall + stray location index) and + # collapse whitespace. Operates on `label`, not the greedily-truncated + # `cleaned`, so "Double between 2002 ... and 2021" survives. + dewall = _ELMHURST_GLAZING_LABEL_EMBEDDED_WALL_RE.sub(" ", label) + dewall = _ELMHURST_GLAZING_LABEL_STRAY_LOCATION_DIGIT_RE.sub(" ", dewall) + dewall = _ELMHURST_GLAZING_LABEL_NOISE_PREFIX_RE.sub("", dewall) + dewall = re.sub(r"\s+", " ", dewall).strip() + if dewall != cleaned: + code = _ELMHURST_GLAZING_LABEL_TO_SAP10.get(dewall) + if code is not None: + return code raise UnmappedElmhurstLabel("glazing_type", label) From f3dcd7b43e150e937766aaed76a9cbe3f6b907a5 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Thu, 11 Jun 2026 07:23:56 +0000 Subject: [PATCH 46/87] fix(elmhurst-mapper): single-storey flat with exposed roof is Top-floor, not Ground-floor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Elmhurst dwelling-type classifier keyed "Top-floor flat" on a "dwelling below" floor lodgement. A single-storey flat exposed BOTH top (a real external roof) AND bottom (floor over partially-heated space, no dwelling below) therefore fell through to "Ground-floor flat" — which the cascade's _dwelling_exposure maps to has_exposed_roof=False, dropping the external roof entirely. Surfaced by simulated case 34 (cert 001431 reconfigured as a slimline electric-storage flat): the worksheet bills (30) External roof = 39.98 m² x U=2.30 = 91.95 W/K — the dominant heat-loss element — but the cascade dropped it, under-stating space-heating demand by 42% (6550 vs 11357 kWh/yr) and over-predicting SAP by +21.76 (57.07 vs worksheet 35.31). Fix: an exposed (non-party) roof puts the flat on the top storey regardless of what is below it. Classify as "Top-floor flat" whenever the roof is exposed; the flat's exposed floor is recovered downstream by the existing per-BP is_above_partially_heated_space / is_exposed_floor override in heat_transmission (§3). Party-roof flats ("another dwelling above") are unaffected and stay Ground-/Mid-floor. This is an Elmhurst-mapper (dwelling_type) bug, NOT a calculator bug: the calculator correctly trusts dwelling_type, and the gov-API path supplies the position directly (cert 0036 — a genuine ground-floor flat whose API data lodges a "Pitched, no access" roof construction under another dwelling — stays party, 2.51 W/K). API SAP gauge unchanged (57.6% within 0.5); worksheet harness 47/47 unaffected; case 34 roof now exact (residual -1.61 is a separate flat-corridor wall-U thread). Regression gate green (3 pre-existing fails unrelated). Co-Authored-By: Claude Opus 4.8 --- .../tests/test_summary_pdf_mapper_chain.py | 58 +++++++++++++++++++ datatypes/epc/domain/mapper.py | 11 +++- 2 files changed, 68 insertions(+), 1 deletion(-) diff --git a/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py b/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py index a495addd..25c0b63f 100644 --- a/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py +++ b/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py @@ -42,9 +42,14 @@ from datatypes.epc.domain.mapper import ( EpcPropertyDataMapper, UnmappedApiCode, UnmappedElmhurstLabel, + _elmhurst_dwelling_type, # pyright: ignore[reportPrivateUsage] _elmhurst_glazing_type_code, # pyright: ignore[reportPrivateUsage] _elmhurst_immersion_type_code, # pyright: ignore[reportPrivateUsage] ) +from datatypes.epc.surveys.elmhurst_site_notes import ( + FloorDetails as ElmhurstFloorDetails, + RoofDetails as ElmhurstRoofDetails, +) from domain.sap10_calculator.calculator import calculate_sap_from_inputs from domain.sap10_calculator.rdsap.cert_to_inputs import ( SAP_10_2_SPEC_PRICES, @@ -1540,6 +1545,59 @@ def test_summary_mapper_raises_on_unmapped_cylinder_insulation_label() -> None: assert excinfo.value.value == "Polyester wool" +def test_elmhurst_dwelling_type_single_storey_flat_with_exposed_roof_is_top_floor() -> None: + # Arrange — a single-storey flat exposed BOTH top (external pitched + # roof, access to loft) AND bottom (floor over partially-heated space, + # not another dwelling) — simulated case 34 (cert 001431). Pre-fix the + # absence of a "dwelling below" routed it to "Ground-floor flat", which + # the cascade's `_dwelling_exposure` maps to has_exposed_roof=False, + # dropping the 91.95 W/K external roof (+21.76 SAP over-prediction). An + # exposed (non-party) roof means the flat is on the top storey → + # "Top-floor flat"; its exposed floor is recovered downstream by the + # per-BP is_above_partially_heated_space override. + roof = ElmhurstRoofDetails( + roof_type="PA Pitched (slates/tiles), access to loft", + insulation="N None", u_value_known=False, + ) + floor = ElmhurstFloorDetails( + location="P Above partially heated space", + floor_type="", insulation="A As built", u_value_known=False, + ) + + # Act + result = _elmhurst_dwelling_type( + built_form="Mid-Terrace", property_type="Flat", + floor=floor, roof=roof, room_in_roof=None, + ) + + # Assert + assert result == "Top-floor flat" + + +def test_elmhurst_dwelling_type_party_roof_flat_stays_ground_floor() -> None: + # Arrange — a genuine ground-floor flat with a dwelling ABOVE lodges a + # party roof ("A Another dwelling above") + a real ground floor. It must + # stay "Ground-floor flat" (roof party, floor exposed) — the fix must + # not over-promote party-roof flats to top-floor. + roof = ElmhurstRoofDetails( + roof_type="A Another dwelling above", + insulation="N None", u_value_known=False, + ) + floor = ElmhurstFloorDetails( + location="G Ground floor", + floor_type="", insulation="A As built", u_value_known=False, + ) + + # Act + result = _elmhurst_dwelling_type( + built_form="Mid-Terrace", property_type="Flat", + floor=floor, roof=roof, room_in_roof=None, + ) + + # Assert + assert result == "Ground-floor flat" + + def test_elmhurst_glazing_type_code_strips_interleaved_alternative_wall() -> None: # Arrange — when a property lodges an Alternative Wall (cert 001431 # storage-heater variants, "simulated case 34"), pdftotext interleaves diff --git a/datatypes/epc/domain/mapper.py b/datatypes/epc/domain/mapper.py index d6573aac..985760ab 100644 --- a/datatypes/epc/domain/mapper.py +++ b/datatypes/epc/domain/mapper.py @@ -2368,7 +2368,16 @@ def _elmhurst_dwelling_type( floor_loc = (floor.location if floor is not None else "") or "" has_dwelling_below = "dwelling below" in floor_loc.lower() has_exposed_roof = room_in_roof is not None or _elmhurst_roof_is_exposed(roof) - if has_dwelling_below and has_exposed_roof: + # An exposed (non-party) roof puts the flat on the TOP storey, whether + # or not a dwelling sits below it. A single-storey flat exposed both top + # (external roof) and bottom (floor over partially-heated space, no + # dwelling below) is still top-floor for roof purposes — its exposed + # floor is recovered by the per-BP is_above_partially_heated_space / + # is_exposed_floor override in §3. Keying "Top-floor" on + # has_dwelling_below dropped that roof, routing such flats to + # "Ground-floor flat" → has_exposed_roof=False → no roof heat loss + # (simulated case 34, cert 001431: 91.95 W/K roof dropped, +21.76 SAP). + if has_exposed_roof: position = "Top-floor" elif has_dwelling_below: position = "Mid-floor" From 48b36d3d7ee62ebae6aaf52a3b98364c89a0be5e Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Thu, 11 Jun 2026 07:35:46 +0000 Subject: [PATCH 47/87] =?UTF-8?q?fix(elmhurst-mapper):=20carry=20=C2=A77?= =?UTF-8?q?=20alternative-wall=20"Sheltered=20Wall"=20flag?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Elmhurst Summary §7 lodges "Alternative Wall N Sheltered Wall: Yes" for a sub-area adjacent to an unheated buffer (e.g. a flat's corridor wall), but the extractor dropped it and _map_elmhurst_alternative_wall never set SapAlternativeWall.is_sheltered — so the cascade billed the sub-area at its full exposed U instead of the RdSAP 10 Table 4 (p.22) sheltered U = 1/(1/U + 0.5). The calculator already applies is_sheltered (_alt_wall_w_per_k) and the gov-API path already wires sheltered_wall=="Y"; this brings the Elmhurst front-end to parity. Three-part change: AlternativeWall.sheltered field + _alternative_walls_from_lines parse ("Alternative Wall N Sheltered Wall") + _map_elmhurst_alternative_wall is_sheltered=a.sheltered. Surfaced by simulated case 34 (cert 001431 electric-storage flat): the 6.02 m² corridor wall billed at full U=1.50 (9.03 W/K) instead of the sheltered 0.86 (5.18 W/K) — +3.85 W/K, -1.61 SAP. Post-fix the alt wall matches the worksheet's (29a) 5.177 and case 34 closes from -1.61 to -0.30 (remaining residual is a separate window/wall area-allocation thread). Elmhurst-mapper only: API SAP gauge unchanged (57.6% within 0.5); worksheet harness 47/47 unaffected; regression gate green (3 pre-existing fails unrelated); pyright net-zero. Co-Authored-By: Claude Opus 4.8 --- .../documents_parser/elmhurst_extractor.py | 6 ++++ .../tests/test_summary_pdf_mapper_chain.py | 31 +++++++++++++++++++ datatypes/epc/domain/mapper.py | 5 +++ datatypes/epc/surveys/elmhurst_site_notes.py | 5 +++ 4 files changed, 47 insertions(+) diff --git a/backend/documents_parser/elmhurst_extractor.py b/backend/documents_parser/elmhurst_extractor.py index 87075f59..fa5dadc0 100644 --- a/backend/documents_parser/elmhurst_extractor.py +++ b/backend/documents_parser/elmhurst_extractor.py @@ -340,6 +340,12 @@ class ElmhurstSiteNotesExtractor: dry_lined=self._local_bool( lines, f"Alternative Wall {n} Dry-lining" ), + # RdSAP 10 Table 4 (p.22): a sheltered alt sub-area + # (adjacent to an unheated buffer, e.g. a flat corridor + # wall) adds R=0.5 m²K/W → U = 1/(1/U + 0.5). + sheltered=self._local_bool( + lines, f"Alternative Wall {n} Sheltered Wall" + ), )) return result diff --git a/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py b/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py index 25c0b63f..fbc1c6f9 100644 --- a/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py +++ b/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py @@ -45,8 +45,10 @@ from datatypes.epc.domain.mapper import ( _elmhurst_dwelling_type, # pyright: ignore[reportPrivateUsage] _elmhurst_glazing_type_code, # pyright: ignore[reportPrivateUsage] _elmhurst_immersion_type_code, # pyright: ignore[reportPrivateUsage] + _map_elmhurst_alternative_wall, # pyright: ignore[reportPrivateUsage] ) from datatypes.epc.surveys.elmhurst_site_notes import ( + AlternativeWall as ElmhurstAlternativeWall, FloorDetails as ElmhurstFloorDetails, RoofDetails as ElmhurstRoofDetails, ) @@ -1545,6 +1547,35 @@ def test_summary_mapper_raises_on_unmapped_cylinder_insulation_label() -> None: assert excinfo.value.value == "Polyester wool" +def test_map_elmhurst_alternative_wall_carries_sheltered_flag() -> None: + # Arrange — Elmhurst Summary §7 lodges "Alternative Wall N Sheltered + # Wall: Yes" for a sub-area adjacent to an unheated buffer (e.g. a flat's + # corridor wall). RdSAP 10 Table 4 (p.22) gives a sheltered wall an added + # R=0.5 m²K/W → U=1/(1/U+0.5). The cascade applies this via + # SapAlternativeWall.is_sheltered (the API path already wires it); the + # Elmhurst path must surface it too. Surfaced by simulated case 34 (cert + # 001431 flat: 6.02 m² corridor wall billed at full U=1.50 instead of + # the sheltered 0.86 → +3.85 W/K, -1.61 SAP). + sheltered = ElmhurstAlternativeWall( + area_m2=12.5, wall_type="CA Cavity", insulation="A As Built", + thickness_unknown=False, thickness_mm=250, u_value_known=False, + dry_lined=False, sheltered=True, + ) + plain = ElmhurstAlternativeWall( + area_m2=12.5, wall_type="CA Cavity", insulation="A As Built", + thickness_unknown=False, thickness_mm=250, u_value_known=False, + dry_lined=False, sheltered=False, + ) + + # Act + mapped_sheltered = _map_elmhurst_alternative_wall(sheltered) + mapped_plain = _map_elmhurst_alternative_wall(plain) + + # Assert + assert mapped_sheltered.is_sheltered is True + assert mapped_plain.is_sheltered is False + + def test_elmhurst_dwelling_type_single_storey_flat_with_exposed_roof_is_top_floor() -> None: # Arrange — a single-storey flat exposed BOTH top (external pitched # roof, access to loft) AND bottom (floor over partially-heated space, diff --git a/datatypes/epc/domain/mapper.py b/datatypes/epc/domain/mapper.py index 985760ab..b47db127 100644 --- a/datatypes/epc/domain/mapper.py +++ b/datatypes/epc/domain/mapper.py @@ -3864,6 +3864,11 @@ def _map_elmhurst_alternative_wall( wall_insulation_thickness=None, wall_thickness_mm=measured_thickness_mm, is_basement=_elmhurst_wall_is_basement(a.wall_type), + # Summary §7 "Alternative Wall N Sheltered Wall: Yes" → RdSAP 10 + # Table 4 (p.22) sheltered U = 1/(1/U + 0.5), applied by the + # cascade's `_alt_wall_w_per_k`. Mirror of the API path's + # `sheltered_wall == "Y"` wiring. + is_sheltered=a.sheltered, ) diff --git a/datatypes/epc/surveys/elmhurst_site_notes.py b/datatypes/epc/surveys/elmhurst_site_notes.py index 16464766..3d5b2b21 100644 --- a/datatypes/epc/surveys/elmhurst_site_notes.py +++ b/datatypes/epc/surveys/elmhurst_site_notes.py @@ -71,6 +71,11 @@ class AlternativeWall: thickness_mm: Optional[int] u_value_known: bool dry_lined: bool = False + # Summary §7 "Alternative Wall N Sheltered Wall: Yes/No". RdSAP 10 + # Table 4 (p.22): a sheltered sub-area (adjacent to an unheated buffer + # such as a flat's corridor) carries an added R=0.5 m²K/W → U = + # 1/(1/U + 0.5). Drives SapAlternativeWall.is_sheltered. + sheltered: bool = False @dataclass From 06989d6b0f360b7d4e946a04aee2f8ee4fbffd3a Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Thu, 11 Jun 2026 07:54:06 +0000 Subject: [PATCH 48/87] fix(elmhurst-extractor): allocate single-glazed alt-wall windows to the alternative wall MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The §11 layout parser keys a window's wall Location on the glazing-prefix / orientation tokens around its data row. An alt-wall window lodges its "Alternative wall 1" Location wrapped across the lines bracketing the W×H×A row. For a DOUBLE-glazed alt window the prefix line also carries the glazing phrase ("Double between 2002 Alternative wall"), so the partition breaks there and the location survives into the window's pre-data slice. For a SINGLE-glazed alt window the "Alternative wall" line stands alone with no glazing-type word, so _partition_after_manuf scanned past it and swallowed it into the PREVIOUS window's suffix — the window then defaulted to "External wall" and its opening deducted from the wrong wall. Fix: treat a standalone wall-location line ("Alternative wall" / "External wall" / "Party wall") as a window boundary in _partition_after_manuf, so it attaches to the following window's prefix. Surfaced by simulated case 34 (cert 001431 electric-storage flat): 2 of 4 single-glazed alt-wall windows were mis-allocated, splitting 2.75/10.78 m² instead of the worksheet's 4.63/8.90 corridor/external opening areas. Elmhurst-extractor only; API gauge unaffected. Regression gate green (3 pre-existing fails unrelated); worksheet harness 47/47 unchanged. Case 34's alt-wall opening area now matches the worksheet; the corridor wall net area is correct (the cert's residual is now isolated to the unheated-corridor door, a separate slice). Co-Authored-By: Claude Opus 4.8 --- .../documents_parser/elmhurst_extractor.py | 14 +++++++- .../fixtures/Summary_case34_storage_flat.pdf | Bin 0 -> 81121 bytes .../tests/test_summary_pdf_mapper_chain.py | 33 ++++++++++++++++++ 3 files changed, 46 insertions(+), 1 deletion(-) create mode 100644 backend/documents_parser/tests/fixtures/Summary_case34_storage_flat.pdf diff --git a/backend/documents_parser/elmhurst_extractor.py b/backend/documents_parser/elmhurst_extractor.py index fa5dadc0..55dd04d6 100644 --- a/backend/documents_parser/elmhurst_extractor.py +++ b/backend/documents_parser/elmhurst_extractor.py @@ -1007,7 +1007,17 @@ class ElmhurstSiteNotesExtractor: joined to the data line (no separate prefix line exists), so the only signal of window-transition is the orientation tokens rotating: orient_suffix(k) → orient_prefix(k+1). Falls through - to `next_data_idx` when neither marker is present.""" + to `next_data_idx` when neither marker is present. + + (c) A standalone wall-location line ("Alternative wall", "External + wall", "Party wall") in the gap belongs to the NEXT window's + prefix — it is that window's §11 Location cell, wrapped above its + W×H×A data row. When the next window is single-glazed its prefix + line carries no glazing-type word (branch a never fires), so + without this the "Alternative wall" line is swallowed into the + current window's suffix and the next window defaults to "External + wall" (simulated case 34: 2 of 4 single-glazed alt-wall windows + mis-allocated → wrong corridor-wall net area).""" scan_start = manuf_idx + 4 seen_orient = False for j in range(scan_start, next_data_idx): @@ -1015,6 +1025,8 @@ class ElmhurstSiteNotesExtractor: first_word = stripped.split(" ", 1)[0] if first_word in self._GLAZING_TYPE_PREFIX_WORDS: return j + if "wall" in stripped.lower(): + return j if stripped in self._ORIENTATION_TOKENS: if seen_orient: return j diff --git a/backend/documents_parser/tests/fixtures/Summary_case34_storage_flat.pdf b/backend/documents_parser/tests/fixtures/Summary_case34_storage_flat.pdf new file mode 100644 index 0000000000000000000000000000000000000000..33cf586d1c18f9a1857ebbc52fd9217bc1a41676 GIT binary patch literal 81121 zcmeF)1ymf(z9{+#5}X7}2%g}xyF>8cZoz{G8Jytm4#C~sHRv0@E$5uQ z@7wS0{np*PGa_Q7r>7Is zH#Rh~BVu4;hVIhL(ni5bTUVb>NZ-L&S6^OKfKI^J&Q?y}M$ppS%F;p~S|oNl8EtcD zKJ;`V#%6Zbktr#IB3`6UK94zvsN;1;sxum6Q$e6MM3J{?nsvxl@Mg-g*7pP=U#^ zjs&#{FC-vdV3YP!ts8_TQmlR-mjln@ivZ4H#S8gBOmJZR{tpg3<^CpcWR2ZfQlDtn z1C8gegZ4NXft~98nv_D0T_~W>j6!TR{GBFtLC7IWviw2nHhNJ-;AFAn{6wQ z+2gA;D%16*g`FMu;7WUT%)I@*IF8fF-RjljpQ%Oec2a{5-_!NnO?L+-Czj{Cb~ZMS z`#KoaT*!0WPegTRbDM6!gPst}qtNk*Wx4cY@~Wze9<7(uE-Z92s~9x^rpTv_k&uU`SWIo^mpylE@U`uD zsJRS-?N!HW%jwHRg!2aXH`qVxt7zXwb zvF@{)v1t^8km2-U2}Xi z+A5)EYDgY@&Y*4OfZB9pX5ue@v-R3zexu|3^IpqoWRv1NvhngOnRw*|r%|SwGZQwS z2t1-6qsCVpo|+J{bjh?3>94HUxsBV?ILq1brlxN?F`S>*f1I=e9l9mie}KB(KbJrtc6718~$RAJ4?$qcq9O?hSZ1uZL`rPBORvRYfKCl+0+!I3N+XuWS{(MDr9AQbBY zR!TWS3ffP(!q&7)t^l__vp6u>Q$9n-ql!TJW zFcKuTy-n50Q@t^M)e^yHoE6ui-KW|@azwY|V(9pV)gT;HBDyC`ii}c!;=FGE03)r` zp0S0SCPfg^Xty^@E$8@f<9c3%0ZW>Yq5_`ZeTxhc(sKFA%v?Rp&7d?pT?A7ffg$4M z9-nhyYYNVgoW6FmD}7NrHoBwAT-ER9FD{8$iu)7-DjO$}1hbqm z{%Bpan8^J=h>h~gOs@)u?zKfPT8gEK2Q$|XF5fc4+0F{5*g3W7B*Pm|JL2S>ajgjW zbo6f4r*@;OSVu}4(f2Ev86##D2552hIs?tWv8Y|T937u`u#XK;ol~Dk zY|3>V&L`&_DvHIo=rq1N*d)g^_K9zfOdNh!E?DIaBv#7%^;73ff zW9S6LJJ+(?y}qA8BW|Ka`$s(sqhw2!5`5Yk|zG`{x1>ZPmog6l&A$I1ZJl*uW)vyKYqj$+(-w^7~RaGu+;PKQm@?VGB>2dX=; zbKYsMkfJ13->Z`HFymH?S2wviq|p%-y_z>Uh6>m>2d9YMIAm71bU6dm4s^|!6X^JU z)6=Pn#FlSF1GNWEsB{(-j4HwVQh^*Y?0vXo(`M*TMgSq8d+Th z4aa>n!!&Qfg-*%aJi&n=KJ7k11&=ePHIWVcyLk^7=Ez>c%Dy*K`SjV};t?P@nzXx3 zFOW!llnpSpk0l6Gi7lzU2Doa@dIPBB&RxlJ;0!iDigZ0x7Zam^&AV195vH#yc+KCv zv=OJ1n?{2bKY@H9BR*&EV{;#RVuL5D;wm?WSPuiKU7sprkH zQv7`2qkKA6n@UsPYIb&$b|x7Px36xBWhwZzwXXf`<9APfhkP>#JCtM4iKB@+I2g>#)kIIv1piir zSCCr%Q12srhOKYmkqJXtFZN*Zr+3p*B5WL3w4{(I*WKddm5Hu67Q$p{Ym%4Ibtk=% zK5^-uDa20hdSci^JyU1>Nkabh*M;9_L6W-P3N;sW^`SqB-i644tRM(}=FfT(J z(KMU;;>1@xu2y@hr9(u$B#?!EX>3=$R_Za6*e3<*Dl;GUDqh(s3Me;E-4^}se@!~Z zl6FqBftfaxh;lpE75}TU_d1m)jn%*pP_;QtJ)>m!jsS7j72R5FRU0X zqjxEk1*5jthwY_?1bgPo^5!gzf;6q0y=~E@2;5CJL3H~SC9kx1qY)rS%TfiyhtKvV zA)S|&#}vl1^~BlK4VLks=1%3kui8u09NNnUeu~6xwL+uK1!yNH+QJSY2lMu^3cE{3 zZ5ff!b`aVK9^QBC^X=~M#meS&<+Q>ZY@Xc?8E%dht0@^0|d0UeIx=tjL#`If*4n+Ojd;S{@*ze*1|40%X2IBdR z^i6HLzmPljgqF*zK{=TQT$)>@R}eHW%JWI?L`C>~dsO1qNBU|k@q{WaqA+yTi8NjP z;w5E&e$Jk%SGZer%Zv^gn12^NchBtTO|?EH-P2N2%?n4Fht1BWvq^j5U&j*u#WL^Q zRcwO?`X&6M6uTrccJRhgH4~k%!n-I>H6e=^3?wj8Wnym94(y+4H(z~jJDvU&5;Xwb zaC?xhK%k!fbL4#&r3&jer|dl>=r?qOCmg)ruJTfqEG1WGTuibuqNK-u1Y|EQ!P-R) zyZ5_*7c+JnzcqC!yFGQ&7k1wkQF2r1fmzqazg}0PysvL$>O!`f{uRRQ+lktIJ?HaP z+-eX(_wlo%@LG`u^bTgLdimyqlK^DL0O`w2!G$Wbo9@?Z&o^?8kK~nHb1BLD3fAq< zzvrcfdDyqv1C8}fNC2w$_kB4d1c<)ZZYAou78|N9GyBcxiL@UcxdK$tf zr;gO0lJ9y%;9YB3LTwZF6!VhLZ0ko$b=mLwQrUITSg&%+CoOT93smEH&}gl4x1VK) zjab+4P2gw_i$^@+6rhu6)ZTaM_-?)idz1ZrcGhL1lEPClTw1OtYOadK67Aq1XJ1j8 zgA=6)JY;)oW9<^JPQSaI-BXOS$Oishm;UmL%>9UPXVrD>GHntsLHMjXal0P>jpp?{ zUz+_$Y^by{Pr_(!2i7$su3mg9(|+HJJw=K%Y=gVtdqyYdxeN*?>T1g94`1Kmu&*5N0crI)j#BjcS zlZHn&;jEL5SKeS@efSmUUpG8Olg);d5SJISg%_f8q+(~;s`iOPO^uw!$-4Fo z@8%Pg>eni;wpwi3jL%Zc>K9?C2wx$yZj|RI9t2V%6W+SA;NHQ0eFp!JA8)ANCTUL0 zAz~_Y5 z|JJY5%n0vw`3;l`2~2;viEp;#q)&)2!%u`eYeD#i+QQwfq1@FX@%Ves38G286(0PZ zu9L^|4q7<$kb*W8fMu_De~y@nQ^nN|*~$&W0`HT(T3yc0$-Nu6^I?L-*$==~o*##y zQuA@>HkD#wV2idEgsf27vEZ7&c`bl)c#g8O@jTyi?E`aTo$2B9huVfQ4p#8!^-t(I zuen$&)7SO}Nh9lMNxDOxw|p#}@!)F;dxXQ@hoWMz-PLL&PT{U9SB&d6^lEBtlsa4J zOs{K;{YSt8w!8NNI$g1F^(};J2~wD8&n$77c)<^Dw9)lj$MTC7vjGQ&X?&PF#fv1~$f3#ZQ6!+@o`^2M-O;E;~8| z&+$xB52r?=Y~1kL?SmLW)sin*bat4=7Vhdq2oraMFC59M)$2e4lf=;r0N3`JxPrh4CRj)tFk=2MI0Xu$M8j-kN%*e&~}U!tZ$yab(&I_ct<(`a zQ`LTHm~m7iv=oJ}+AHM+R?@jhC=Cf+!@N&-m{>HZgw{JSR>lvJBu9>>^_3jQn{YLb z4Ft!cc(GMVA`DZZ=hU*Xd0qxvraQ#%Mu^|1c{*k*a%C+{%=3*@UJgZ|8>@16o_sAKD&h`AvOr z-p|pdF_T6ug}}=NeCTnPg{T!DH%TO8quf1 zq{vYnn-XNcf|$25E2z)adqga3i3X+wzNQgs)>5T@aU+&pSGEzjU#&T?3VBcPHtXc; zCcP$s=m*G_4a1FlkNJEz|6MYl{C9y>tU}(Z?Dx0&j)?ru!lx`ZMGSF0g?x1Uu{z_& zeD&W;J1?l*@oPYVph;xK$ja+A4_>)lDbh@=6?;UitZ)~TF!AqE5`}g|Zxl|Htn;XQ zv)=KsA?+00h}q-X=Tf_08ii#gM{P>=vWvMe`9$;8lAvGU_6XpeGA^qkvWq6Z5s zuxvvZ#U$v?L$W^oYxaGLhkYZThgjU#y)W^2!$r0{7+*ngH{cJD?mN1qrrA|2)3k3+ z#{37jl$uv7-ktwU<8;@N1wd+T5?(MH1ij8^IupB~+CR1An1 zNumIG^oYytiPAcUwA=nW*O{9mH+H=+0)a)OSF<@${F^PwCqL%YinZ}4BaT4HtFS9d z^3fb`Q|on#T=9j>B!}p|pcjCkC+*BTXc&;F#x>79XV#nf*uyj4mUNzP@!>MWxqLjq zZHlt2{n#P_h4_)gnF(!{!5}>|)Bzd_4z-I014^HfcIq?ane0o{nv3gcP^=-+-atKe zalKc4r$O{HpKYZH6zKW4FB~$hGCUt!?@SCDq}}Cb84U3R_h)cq=8~JGO1~!#0lqM8 zF!4o(u$=67GE5c(6aOu>YhgFY_<-~=Jt?Hsp&k)_Ep{r5_*3$c1C=qu)5MW9QPhdv z_JTK-URzQs<8vp7;^b#c;#?i0#AKNbu94yLrAYWTY(NB3LEBXExVzP^3xQpqFer}Vo|29rpcC0e zJg@lW=)mXGGR-kYLgg~F2|=Qcyk6jmL5?P@9IbO1s=;IH+ifbt4L#W3Kj3ZQvX?4{ zN5Eu|TePvUI)X1^DsisMqtrA&3hAaI6nt4PF z&fMX(>3Q(qoZtk^Ek8UJL#=y)2<_tjb2spxJG+_xweD$XPxU|7Ju z_cXlS+sWARRP*T==`z=PjzGuB22Pn|z3}KM`8{@vs5#QUI2skdhD$`{630)Nb%^-_ zrEDc$4H)k~H^aSx;p8v*nl)8d68{?E=~Ltvqu9aBm$yEGsQ3L3OtufMmv?ZHU!TDu zVp7PZ^LhRKLaR?;Vzydv0TZVN5+WKd``xq(w=xp-DyPOvL^M?8c%kRd1xw9M#OKND z$R!o|2S| z3?hv8@^VJD^(WRLwr`i#kzZ10Mf>&`LigDR32J_hVP%z&Gkp9=g^3st5U|-Pgu)Gr zqsB``U}tclvUxGR!c@GP^!@2%2i6ZH?^d#spFdxavwwh7U!FQrRP5_nBe4G|V339) zZ^27bddj>uaW*ItLt(%u7A%A{5NMUL`AuTPQRoRysa~?aps`iStQI%sY zY0QvDvWTEf=sur6NENf&8WNz5Z_p();I4mA38B9!BHhslQe_sYZuusdoQX#Q+Y#{+ z1S8^aIS?M`y~7i-)(B$V>QT2Wdr=TOQ4~L#%}_O5Gv>&rJ*jPPuOy;gWb3DnLw=#V zE$COdXSm%W_1e(aK8+mXL^Qv!~u`rrs2cTYt&%gz)BVGA)I7AJ8o-=8-Q zwCQ-QOtDAOKQPeqwxI4)#uV1|WaH}Ml_!<}p)<&7(&T(ZzV??S%tC>q!?gj$9xgH> zjC(`F>gwuAy*G6|6LGJ}Wo6;D^T?O-dY2}{GNWuC0@m2@_a_66*A^6e*j8$4(=04^ zcX#O%TU&X^l{uD0XNbE*UH_b7sGJH|7b2+w+}e`R65}7d z!FME6*(yH|5ii>v7M?l5%B!e+{=RWp_koKPh2@+&o_6ujyHOMxH+%w?<)N^COE5{e zZv$b%tZ{K&I&6@tf`?D7{M=hnPs>DG)p)+R#Gf^6o~gEYlRBEHfw*>>T{>&7Vc&~R zl+XCxOtrC9?_)O?G=H%j43l}+_U^K-Fm3}ILvS_WE!%-3|KhtN)qxP{az>`=@?qkP zlAK!A)`FZIvLi(Q;K>ElA{H|2Q(tOiQ+5NT^EAKbGQKYGbwRLpPJHp-;?oxQH#P5eYiia5_iJE-K)s*(in(2P?BsPu9k3B=8pA-Bukyy{e!WM3(avN2y0|3k zv`l2A5vGys7pEAkLyH3}OC!z27rK{OBvFzB5EjG>(i7>bO>7DlNlmpYB1eSS=s(!sQB9tE1$;c|y8ZJT3tDCSfT zlf^sYf$-1@1D>Fq!I{6?0H#a zi%2uGGtGF6U~xB+fUy$#4?5?Ia?}fsnRi0t5riR8_8p#r(ml~ZVi!uX-{S6FhZN5z zsNh;}Zm#pwJ~1&cpB!Jn+3uN{4JCgxHv(O)AMuq z9qnu!tbUD;j*Lx&hI~y(YvR1{b!~6sb-fOGJ}?+7BWi$8W6oAfz2JSulI>hsUyn?( zZDf*x`K=N~##f~|R%4s?Hq8Ficx-Hx_oE%VMt%oh!SA!o@V>rYQc=EpS)SXx`a-j! zxj1Y?c=FJ46GDQIAGX)|4BkpoS27uz7Zq}=&+i#KsP5qAHTyLabsOl3eU-i1%aq06 z#~Y40Vm`A!3qc8Kyt~eV#1CzNJIP9GYiDLgpBd&RgV0q`&d<$JG`Ktp6b4QS>>{?34d9UPOl1Ua=X{yIxV^hIl1Uc&I==Jya@PC%TOHN5C_)(ZOJ+)(GGO0Kz#u$A} z#l;+Qu*Sv7siye(YJZaaJFVvS{8Bp|wW>2c;yv9epTO2T?c%(Ii>wy2 z{b$JC6Jztwk)BAny1Fc5_OKv;vvabREG=a%E5{6v6uu)GF;tfGut|Q zXIcgcJVMz8I;qj0cP$SNi!CGR6+cJ)@;&`&K-*K#B_TaVDQOuv*wn5Xt&}w)Cw*-b zR8;jEx9RTY63JqJ1qB71LHOl@vQfsK7Mj!E{&q}JS{T?2ukIILhL_NFpSR(1_&3*F zUR($Wh=nPZ|IluKy^Tzo?b+1)_NjLIuHv$i!n@oi=Wm|~jXJL*ZO>18#VZXUCTW+( z&KqoTL%+$vi;6|f@K;32N?0oX{UOMqmv$THyl!-e{u+I2dxM9e(c;DP z&f%dhv{0nw`Y6*^c?P9bi6BWGaJ~^F;!4`vL?%-3T;XC|*378TZI+PDp#H+U6JGN= zS*byy{=(sPLGa5+3Amf3tU{W?Rsn%P&mJe(K-9r2mz#bmS?YRJMF!@#12b{6y-EpB z4_-Ht(Xz_ez|;%yTZv16F@8REg{7h4C}RQZdx0EK<=pkNvl$bKwFc9WeAk84GuOVs zRHcC&_hWnOO^nUf*-&6gc^M4v?B3~~CmW89rTzB!xwzN2n0R1#y31A^`r5h1Srzx& zw8}I|DLpvKN31^3Hz0}A>?Nk37$rn1G$i(nsZE%qt)pA43|XCa^4AnhkSeq^-aaDR zzVym3zB1b?lvBq;#KO!HK}7V5X>Kn*yqcBQFvsVDDH9=XIoQNHL1i8yov3*Yx(jYS zfDJ4?+2po}xbC_$AV>X3=&rnZ$)C7PrA$f60ozF!nO;}G2_{r&?Gt|1A|!D~v|uU8 zw0-7!gxLBT68=GuGbA@K9~ojLO{(~QP#`r=`Xneam3D~woqJmnn@+ckxQx{3*aXAy zSb(7Ddy=C3tkRo1B6-l57$y0*|G8w>PHvCWy7EJzYa0`xtH(*)`mxX1yCx4lnwM&rZrOwAU~S(ZSP|TShQ9V>s&mzCK}`bMw{ye~t30qGCX zclL8S>a^DnvNo-kkQN`G9Eo1WGKx-%hpc_LYFSE0`iUgXV*k0y`_@g|GX-XLh?*X6 zz*(W*#N2F^#U8$7u~P32KEHIwNs9A6lub&;gOB&b>ok(h9IavXsAuV?L^R4Pj916> z)YMe0gDz-#^+6$DD@rSNcK6?LzN{DcA}6mX73UQK>b;q*?ks>dNd7KxjPLnZol>xtxZCE3TQwj1FULYQj)%tJoCG-CS_{3^U6Y^!3pD*^t^d6^a!s zWAzU{$H&$sNMUd7<1X&M7HSVORz-sNY;A2DoU(Fi^2LQs&Y458SlU%e39Ns3C`XpL zNWI&55;D$zy$J(>hJds3CR)aHykfqryGBZB#wRDE*Ftqc)g|ywN^(#9iQW@)`i(Vv zT~_B#yW@agb`K3OvC<==phn{5=jV@%jm&TSMy6nnZpq6phJ!TI9=QpEt=~Eqh3@mP zz!$1KkhW2WcjodJU8aQH4qQljD+;+xFUhh_CM)HmiM?9d3}@F6e~3lK#>O6mg&D?A zaioDz^-?d*VXRa0+#NDspkoB?!QtO8$?*E`!| zDao;77GO@xTQF!D7jpX?&&a{ueV3xb)yYq5ptT=wfBj;A_T=}=V`7-Fx;Lig+W8+L z(38>D0(*OVukRiUjZqo0dha&IY;bX76mM=|n=;>#a=fvIVIn8zq(<`Oqaf*hQm3k> z+U+zxYsXoA9|XH==$ZDu&84M$+xX@rezEdl1%HOYSf04S;P`^=9=E_zS*Z}K{Y#ub znfi0p+7Q9ygoH0J-1aM3XY{n*LSGjuVidQN>jFSd9gPb-^tkAoqav#lTSj_L?c33d zE5@s(eO*3!_1;weLILli6W=j7->~jQRrwq2T#H7ge#^-%+Ff4`oD{WVUp!X|O`h`GP5}{*s%T8XCg8&0~DKKKmz^=kIcy=Xi-; zd1VVk$w{`-y2ZuD35eOMsYOmG6d>$ymtAC0?k=KZeJlkZSvW^2pCIaC?ZqPJM=X?h zJm^+KayD{UmDn<@C}>y)y2ko?=UT-Uo7;z9U?VH%#c5DOc3$SMNGB#G#4f%)YItKi zHrA#04IKgiqA@6aLDJ4~Tfe+&?(KR@%#o~qi{^(2- z;Rw3xvlmxv%ZCSS+{BiJ5f`&=R#WT2rN+|;mOr$twfWg8_O=haMC*!-gan^WUDVWY zj6d($@7n2|QGQLI(ZJoHu)?2r7?!xb?RD>=jzME&t-U&M@$yXzon@PmA7y#<{^ESEEb@<}A8~J{lNquU2qE%;%R?i@C^}R)B>FlB&j&On@Jz{=mKYE^~ zl$fht2y0(z>FK@ara_UH5C+f3f;nLQru8>>7n!Lau)0r+8_cR>-#27Eh~B)ePHB`x z~5!?c}5^DDwWt zYpMxD$M1y|m|ht>R7m;n<2(9wud29o6Yf?ti<;yq<}aZ^5#eEH<}#LqWAEq~;P3lt z7xUKrZcAoRESt0}xcv3kMCifJ z_QtZ&;nIo}xqlED85zkt5|IvDbrmJ0pTGQnnV6c^S@9Ts)AP6oKUGv#m5`u~h90L8 zPZ0YODsfB}nIJA}M0ees4v6n3s6`;V{)rYiGeJ>MRI;g1fx%GTPE z8L1vlMhQ|IOGqv4{0S^b3brNRi;FGn>@dk?LSz+{l()CHz>RrYN3iiI$n1tXMlmU| z+!U{qH4`*%O1WCJ)XfUxYT*d{>^6S?c9ygaR=t>sJJ%-85P|bg3<(Je3i9%*bKe** zG%-gh`g-(&Ck+i-T3QAL6%`u?D+xELUA>9;HToCJ_NAY46u-tM@*#^_GoErwziscI zCb`?yp(p$MXSKHvBowME%eX2g=4iM%E>lUd?5*IZJ(`WE7@St*}k>AMXBBn zgQIJp6EykW{m4X-`5g^DT^}E)>GKFXW4Cd70eO>4N9=)jMSWI!aASVy%U>j)g`ULt zc&#|vrGIe!hR>C5ch5=-Qjpg0*bfooqMPh%8yo6kV`f(@lPBPUH^*FTdYxWS6KL$J zRu{$P3Tw4aG@eth4O>o`ee|94aC3(N(kEHiH0^o&tz*{wc^Qu|!j9?ga9nmktp4!m zs7{vU)a>b{3bS>o^<*(vc&s==HGtOVbn{uH=yiAKF541GY`=s6hHXiC^eqw?+8y*| z^e5&PP4t`lgKL)2DHs8ygYof1nZk4Q`J@Y?pN6|@F9~tbSfkwQHtC_Q9x+HUOWE2& z7)-#FzOH{)xVcdFk(Q=Zt8fX1MKFX_=_wly=#A}rG%l-cgb0x?;q)>-zEQAK!RDZ=v<**kJp(jDWdt1ZCO-bh!=!}88xJ?EFZKhjosaK9&6NDabZX2^!@6t~rc~u&1zk1jyjOh*wY=;eg9jai zBz-cay-3s)tNEr^^c!g8(+Dw+H*rt5gx7_LRRH=Ejz9%Dy!Ov#A>K5!FRKMGs31>T z)qDd4@?nd95{1Q323a+5HLMU0&s!hE5LGN;uw`Kjw1#-IA6|$K&0Vc}jMcr`E$PFN zHMwJrcv+h-fOLYiXKtKoabRJGO4}xS`r$ExDd-ZD+ZV`2UA4Kj zesVLk^!IoztM1AZ8G^LI*47sKR%0IA8>^m`D!UyYx7^O4=QRsuoLr1GH4TtY*oq7b zOVA_hdjyMM-t3f5c8W1)X4-xc%UEWhoSzE8g)y#v^Knup-WKo-CO@$Dw*06qLqY5> z5bX;W1>WzRtRr~^Rg*YtL0cci0Uc7Z_Y_JerVS_~AMj7VrMYcgOfMQ~5;$;NWNlbJ z8}(sR!|Q+dYtLY4<{1Qi_1)a8A|(uD%B<=Zq)klxwiB0|n_C7o>_mC%w&rv+dbvek zC?Dmmy*c;mFLU0@B+3UAdLXQFmW-*P(%W$WI;Rrb~G5>nd5tWtLGIvE|`f7 zW`54?6&9|w3vD^qJWZi8AZSaw=}$~*dIym_KY$wuJmV8tV zv`&H)%dui}`#Dy5Uz>leVe0f$3Qq?)#l}V=d<<;gWn6;k>Z(2u&~iXKn5*%rQqcs( z$0s2#5Y%KyxQ6MkrHbDB`qzA>-pW?=qxJq3q|LG zc5fxF3644{gP`3-*b}=Bj9{-!3d~lN+xh6tK@G|t&^i((7%kB>D zwOrGFY||p-y4P)}d;9vz-s+Tl7f$nj+1mbW=GI^8WreTny$7@>DK@a= zlvJ|wbF3K{ZG}R0x#JGb1`axD;+v!;jrZha>+8idb6*=@ts5Cy%9ckHpu!Ep!@7YI_1Ooj5;9d_e_nKbCr6F`E3f}L>wZx=cQgKP7b*aR*+FfDX4iAnj|0_ObY zUm$XY8=`u473W8$Rl)ewEdP8+*DA?Kwexuf=A=6?_>c!1w1VN;XDL}%R@PcH2PsW0h0(-M#(Zd*nCR3rxiKEqLg+jr6Xf=&ITb45S6Vga#jI zXr`x!N5xqkma1$9qz9^&9XT#Pvip#F2d1VJcU0*>@9gwgMhSGjGa*VEJ~-IHCdk=| z8$n}=Q^t-55fl?n5)j}zlsVSpV$HX#O3TO$4)#O4n2`Khq5WEBk#FbqvcA7`uvd3_db&|8=$%Ky*l4kv{Zb1IsElK|ec#_04 z-@m_k`v{Nz9kz(^AKbhJY!P6K{*Q7az!m|v2(U$fEdp#2V2c1-1lS_L76G;hutk6^ z0&EdrivU{$*do9d0k#ORMgKS2BG!Mcd-|WYMNI#odm6AsfGq-S5nzh|TLjo5z!m|v z2(U$fEdp#2V2c1-1lS_L76G;hutk6^0&EdrivU{$*do9d0k#ORMSv{=Y!P6K{>R&* z$I*=b9kz)1AKbhJY!P6K09yo%TLg?-WDAU21dLk*j9UbZTLg?-1dLk*j9UbZTl6=- z@c;M}7`F%*w+I-w2pG2r7`F%*w+I-w2pG5MzxB99?EhN#^gnHjSpGrxG+>JWTLjo5 zz!m|v2(U$fEdp#2V2c1-1lS_L76G;hutk6^0&EdrivU{$*do9d0k#ORMSv{=Y!P6K z09ypuBES~?kGDk}|Ju!4CT2Q8OEXIw1uJb`eL5k12V-4*c~Jp60b@H`Iei;JOLHqr z3w;YaB6d0%ZF7A(an^ru^A@m0fGq-S5nzh|TLjpm?|>}=Y!P6K09ypuBES{_wg|9A z2#387Ma5vdtJO%H!d+Fa7}ssU76G;hutk6^0&EdrivU~n-`W;2(En@w)BiLsV*3aE z(*Q04a1nru09*v%A^;ZwxCp>S04@S>5rB&TTm;}E02cwc2*5=EE&^~7fQtZJ1mGe7 z7Xi2kz(oKq0&o$4ivV2oKOPq`F#KzGZ~tjs#QqQN-U7M^&_#eQ0(23eivV2&=przn zivV2&=psND0lEm#MSv~>bdg8dukSnjA|sl81eKY*tH|62d_Kdp;6{z3mVpo;)q1n43_7Xi8m&_#eQ0(23eivV2&=psND0lEm#MSv~> zbP=G709^#=B0v`bx(LukfGz@b5ul3zT?FVNKogm({=?M`lJ39}Lt(}d&wmIVc-TnRj9nS6D?auk? z*iKvPQbF@#PTzX<^3lZQ&G|WWB@uMGxIS%PE{<%su`E^qr3fk|^J!;G_}1%{OvY}U z&fni%-IJYNAOD<9(ajYTiR2KA{HT<~14`zVPvjPhw_V#9YW#CghF}Gok%+OkgR;H*_45vnl^wid`nVm6@YOrIaT>fa-+4a%=1J26+ zvR9d-Ts)^{x>)^GscM>lTs&`B&DZ-oq?@~&rrENjF7Km@J?Q%xSsk)1)PsJ2vT>Z? z43Vm_0{yH{(Dy7A$^Ub#^78uf?DxsW@$&WU_2kC5Re_dDDl{uDxdbllOv#p+TCXxY zXy$^UY>IK>(Dcyn4f?jb=i5zl)s>TZKP7NUL~;em3#}|+(MS&7AXeFEF}WBK z=m#$n&L$YjDiXo2l_jK-B-J(F@%S~8O>RtTr)ubAi+)b!hnDAK025~b6HgG!rvz@D z91+<#zT$!6-#5P>)4qcib9QUiJlptlvY2M3h-Q|sMy8NKQZd0^ z!OCB9JnZ3`yC&NtdMGS7Eq6m#v|VHV_I+MOiru*9c3P0+owkOYv3)JIfwY&=#N(i8FsIlxE()4d@1^C_T>x@D{`P1C=j9_69l))5z6Yn7qEz)zzl)<8kH2KPlX$QA{sU}W+Db=W_lt9dU{q39-hCPZL$CT9_SS8b?lt2^yw6B?DZ8N7yj6Z2tA#E zwypkSzI6X-hr$-nYY}4$LpqSL1;2%@@n4^d7~9y|2^wkJJeEdE`_Ib^jEvBfdUi&( zYK&~`M2`~_3kwko3+o>fJ39vvD=Qlj0|x^U6BE;)Cg^J{?96{k3w`Y`+x=ym|F8L9 z(?j!k%;%2@x<3Yb29`f|_P9efM#ev;$7NRN??$je3-mSSdxE*x)v5b%NKTjHe#KV8Y4iUp+XvzOa9Q@5Vy#FxL z{@Ea7{F^}({s^J}mmp*Kn;@h6YdrmL#uqCG5MTfP_=3hVw1TqI|1p28V2@w<|G$5a z(fnU($X{v%bT5xl{y?@5H3^f^UE+IE)xRl_c5DIq_|*e#N1laa+?FVJy6q)VABV1Pgi zF^WAy3_fdR{BU~gw$J#hCXL!FZIFB{qt7Jd?Uu)uf0HbU#2lgca~T*?Dg+;P8J(zN z83@I1Ihp$ex3uL%!EbmOVQTwEv#XeFGIw2!Q@c}&$GDP)L~A%u66B7H#Qc?vWd(> zctdN^56auU=~2Zsu>JDgqGCnp2A5WLltjPKIJNWQJ$;5Mx+Ke!+*}Ob`v8M}WP`$J z9k>3Qx(dJLVaPZO6$=52E;#GPAy4eJxh#c5#KG*z$drp9PlBqUv;EZMvTx>V@!$RW zS7^E##m*@QNX(f$$tLKLU>`SOC&^&sv-tVI+}F>zzFS7O(6BvgQf$twVBC(JTsPBJ zE>1q^zqB_P=nuzByi&Zj3VLmR{{304-{msNo^WZmJ;SN?npm~PP){W-wrRuBoH{aG zZ=ve7Tv^5_&*FmnNu~f1Au28j$B&(gWM_ZhFPbD#$r>&teu3>>8?HL9Sv3UMK#{!m zsY#(|`dlx1x@c;+AZ#% z9qXi}ORG{71CN?`u{u*@jB7pmL&Br{ww|S_YAAlRSHMfOAmE;k^m=RFlsMBbmTkMr zPaDn8S{$@9!aPT4x8D_d%*B;GLYeDU@zkNcv|TOr+Yb^ZOJx{?Lj6#;&*MbIo08EU z@90@DUQ-1ACXAV_;-_$@k{0nH+VXBjS=(e^xvc%H{}mQw< zAMTEigFL^=;W;-g<>a8u##p{24`!*Ckfzz!Vk&m7Nxducr$OD8NUslQ>|nQlSHR4C z_no3Ku0JXj3?RXGBlBY@M-;(2St^etd1GR((a%S5#+x%hlROPo=yx7AU-J@a!YE&$8Z=+SFcmgJBu!aGJ8AKL zJ$Z|-f`=48e-eI8TyQRE(XF9U%ry;nm50m8O6J0vIbU>6Hrf(2ghfPg7>C5p=r|Tx zZXZYZiEi!qN#lshNN!1d+VczVlK1lDA3n`1O%wSWr*s`VFyKzq$!3;+1i5ER;Aq2r ztahI`&i_Pkt~C5y6tM!9snVyB%0w3_<;Kc?dDp>S3r08;bD0tOb;Xm}nSs91uy-m? zxGkmFe~GqVT~OYEOwTCL58M%`B-q2BiRiC=c~Ieat!^q2X}mtsy7`;)pmFpfSg`5l zaH||s9U>rCg%L^ZXRS4N?f2~3sI0?AvGZqyS4@Tz_UKOt1JWx;N^MOI{OH~Pr?jh% zs_NU;2Wg~~2I-IzI2<^K?n6nJbO}gEBO#5Tq;!KICDPp>9UfA;yQDjhJoMiCepkHj zzA^6IWBoPvUTe*@*B*0z`}^h|yNq;)o@d5L7QJ?PpD2N*r?51_H9{bhA*z~}jSlg< z(k_oz{;d4UBjR`YBG4kL$o26Y&5`ABw8d!w=U8NUuLOym<71Qbt7)^#^F|W-9U%36 zj$rzCT@wlwNH{#v2iQ^KpMV&lXG)xb=Osce#h(``WJw2=WqV9mkkKZp;|E3w{2_s} z_qn6Se@y2#mGM|=+=HUsbI1N>xs8K0ei*sw6v2-oulepvo-V2_)({=k%!@=&z0zvk zE_Ziz$t`!b`ao&cUE=8j(cGdZhjo$(?n>sq>)Dl)+kuSB3AnqCr7!FJoO%IEufj2( zG<}wVY|gODWBc99`H<53D4%Ehei;D;OjKfTj$FIjTFZ3Wi1blHThoJ@Ns7aVX8osa zMq%v=R3XEMk*<2S{dhUAXW~uc*vPUBjH$OHlb|L}xq(&uJDDdm66YWGp$zo~8Uh0= zyUDPXb-;N@T&v>MaH_TK7CIi=QIbLX8#3?c5RXb-ox#Zvv;M|;wMs4Z6+RtPHkR2p z7NaJkdq-|@q=&WLmW+vKD@_bH%<)j?rWyuJ|L%tPxKC1bA+t|bYj3{H}4i@2!GrBPIc{-hzwOxsMS z;h=Wrpqi+lS>*!0qSuu!wA6vJBj5M3HSu(B)PYHb?w+q9#w$h=nG)H+kIH#h8^W%V z7lxuZ(4jPK9`aO$ib$FIle6zn=vf(}Cj;*j+7}(~#{?sNsyjHH9Caw|o|ZcN_?rZHY;86IFa|g~yz+;;64ayM<^{hw!`#*?Rz%6gs`uWPT+@l)s!` zGgB7bb!E)ns)4o+K3el{==l=QW57D+K>6K9iyRy47_A%6;alXesn!W`yxPl_^3wxT z^)XY=cMj;?FqB2+9%tB#xc72vtH)M)PUN*kU&i_nEprAj`mMj7)|dLEXex_>gfe9P`Bvr>U&H|^Q|SeGxiEw`MW5H;8yzk@L_&Ws z(T?TUnJb!}f~aGz^;PZ>sZX`aM?p0tldm{f3zqr`olDN--6c{=D_NYne)o zQ)9a5SF3YX>Y4d%>dzSXXS3_Rds!xtOQE#_P4Tv^E(BKMRz3jY5B?K?hz9MnK8P8c?YY32HY%PEDU*pOT}s6 zusuph>w|T&yBiV@zfz;ni`-RU`EvcN>9=DyC|i6;at3|hOH(9nDMSC~DP^<$6^_*! zRr;h2&kfCOO(0T^#FO&L{ACLgDK6r%!nu3n)_%j7-En2<2~lYO z=%4a%gY2d1%j@;`x=_}f7P9Sj6uOfcMK0)SdUZkTyR8~iYz8j9I&w?a+_?=-FEzJk zBKqEca*7#GH0)dy>&-#MB{lsPkl;CR*qRzRc?NLEj(tY3VngrZlBi55_~R(-$js20 z)=GQ2>`R(*6`7-&14_zd?}cz%Yz_@h;1(?11z-TS7IqPRGgYWO21}fK8E;+cT|ZmG z8H69A3iDxW>mO`qa>w`{9qBlmErbEgGrm8gH+<>{3||sG*6_fdA@{jQ*DFu>9N&;S z#t7fgU6e_LA{($-3?NDW+1x_!0j_H|PMg9p4|fB)5cG3|zi>IN$EBjouZrDV1i4T}$t27pElKvQuq zuo;8unxL5#Ig1!sGtiC@D-&?PCq+N>-h?uS;SUT$gkPcspDgoWz$<2>^SX)!I_^uEZ)2;!MYfb9_`R|BGsTY@VOd8c(=X)d! zXJ-Z{fPuajdfB!eW<)Ky$dV6q*b3Z`&5`%@RZ7b-3|Ii`s?3151i|0ptrafnR19}9 zz^zD<%!E-q4>AdSkOGkhH7fx~Pl%cgPwtWI$s!mzWtT{iM8TVV3@&wmC%O2i0tXki zMmz}82}v|F>Vxt(FpGar{ZZUwYF8~vtpKS$r55RF+_7aDa?#9&o2W;><-splI zeq@l#Xe4l2!#siJNW*aRM8Zgj&|Z)!lFsBl@iTwHm?UsVtqir(^SP}M?i z<)m<}Mz!FLYqNRj5(1jGfRAVhss#OZ4YIKxzm2c*($n|cmAE|!hyocDQRPIf@C$GE zm3!&qiLhWC1xMB}leny{;=%+RHL%}s%41_B#0Z}xMDv1|0KYga!@;senuk$&c9R?r zvVjBpWeAx91Caq=le7XYeGtBgnHIiL{CF>2@nacBjJakUl+$!eJIP1_@j)S=SF}%Z z<{MYWM|(Pd6DfS=xQ?~L#{DO1b`{bclSb`K*qr7azPU(D*qun(JUp_;?-1NA5u)Cp zG+>P4-1lWy!#!qFix6)TGm*6!H$1X4bVc&XLj$v`9>S8ePG$wejf0EpeAkty!AAn$ zr9^H}^iC9oy~QhE7YZ4#A>`=7pAiT~+;qEDCa_NzJzjq`G$$C9-&Mp;NM6fGl$OU^*VBlrC`faMp~_>TY!7u7G4kzIzH>K8fkJL&P4 zZ1Zmb3penu+7``#LaiQ$WiW2z7RzOnn#yVvg5Bs)+-m9LvMv&fsILz)sDgk*I7IF# z!e0$?J_aB`hh8e?I7J{m*gHYI6lw7Qskm*-fV?06{Te>C^!o*UKeSz!jcoVzJLOnps9!oGNrEob(S}R$dKA)238R@m7>g;-J$W;LX9piy?jo_3giRQrK;x)GW)u<%JmDzBko%t;jjh z_5yjJTXC6-LptUCD`q%N?0QMu1$f7z9R)XNq9|ni)j8HD=TkbwQJqRTS;QrwY&-2& zVRc_kYUl7d}G=H>$ao2;b!Ak6@Uv?dmp>v(8yu;gfP-iD;PX z<5vktJTPV)M%YkXYNbm+7j~JHHLALGBX*f*n<$uXL*Xs@%)%U!UOW2h6W>L^K1WmC z++N8-x%Ou{deOZi#VsyTVkOO4zO%kSBhJJ{(weMNQmXPE`hc-g#o>N7ij4}Cz&=J^ z)9EpbDf=XREz(tOaL^QhbU-@z$&k2Uv?84)qI9^1PbTucle8A7R@UVN;lin!nN`3S z>r$r0peZUG=HbpLG#~w8`=}*OT%PCqs_}j2=*jkFeVT1%4MreY49!eRLlB<72?%zB z00xP6rtLCT%;A(IqKj#PSjR8ZKAX@uJ&sx})1Xne)(cWYwixePP83UHJ?W6zky#o{ zU_@oFqxC|GmPUrYRq%)i3huAKODm=CSY>%5$0i>JUK-_Td;T z`dFEAI!H0~TF?@`esjp4w9E7%p?drJCHb!UL6liI{zZdA;G2z$Ej)Q3L@_QEl`bT$ zn_(_mKGpFxr9vIqa6q{1-drH&&Tt*`Bj8WIiOU2L&222XP*y#Qw z1{AL_8RkMhxa~4=Jhr^Mc-`cj11UW0(mSPT#zALOr}E@ev0d*m!CIYHw|KCIe*Hqd z%L0S+xbFwYISqI7VM`r%z?ddnt1H6JuG(lrc60lObsOW44nczkrt*e~`QYYlZg#bl zHm*={Ic@wJ>{9x+@HCBl4yAo$R2;k$OfzH;Ifx8)yeY+Zi8jyYr>-c&WzE<%Pnp5z zP$NpL2-{Y<7N=Ih65G>v12=$kpC5_;BB|X`L;stk_BXQrKY0rWQ~lBx*kx`-%D)r+ z|4ilorKkSiS&f(LpPBs1XDv3p?kXZlnP4_ zn+)XO@s7c+bQvq}d)l20C?;J7ukD(kP(R~5xkCE&K@hB&7Pr%v3~G-Qa(x)r7VGfC zu8Ci{9Q`_?o)z=_ptEF7bMKo~*Adl<5!`cOhnCojK6TVJ|C}Vxu^H$oKAv88sWl@=J1Sg17!TDTX z$e%ZF%y5o0q0ffmammoP?p85vA9*^Vys#1HGouJ>@@C;|-}?e8&!N7QpQ})E>cbv6 zlrPe*eTOh(H&0X!)#-m%iVWHrPL*nR0+PcX|?$Ht*7E;3>&NVxCV zbSUY&K@6htcYU+rQ2aC=PE(hWeKWnI^&IS)`Tt$`yAtPAP?oTLW>p_I z{;D`jYr|=HSS4SYl=9uF?pR#|5%7&$tm)g*r)T!-@TdG8ukP)t+f#$&vWJlqzM?g% zd=Tpf#bNRAmu}zsP$oL4fJpk$w79&`U$OOiUWojB1#qrZy{_Nrw_H-L%Cz6An%Ac- ztUjh4-6?%St*il8N+fV)!EZmZexRI&1X1Mkb9T^tXC zXRK#bzl*tWoi52&vDULZG9J33QKsMZNvSLvc!Q^hV+F56*_i}tGzr?Q&~yyhg~dDN zuj)8vBOgrn^qHeag|OBqup&)um{pHh1{&-0OLSd*h?^oWFs0^hLS$Vmwk=W^=1S*-pb3gY$E3vt!21cyqHz zlysdj00x;#|9T)+%NLT-?HgpOLFu%A5tr_0=l?}q0{x08{}I^cq5Ac=3l(-5UMk?< zgi4v)00sPq?g+ds|M9{BwlS_pLHI0js^AaW+Qt^$m9W}7dPeq$ za@Q>@*U z#=l9?E9h~PhPm{~n7n$bs*hihLMm2fbZinpYmF_e9Q|M%BIT4qUXOjOgQsBKx_6ox z#3H2HEUUHJi6Y=6xp}-;1FR*&6trq?qCrFJe=!|6u8W|>>JS&zm2!$$Sx;rBpHgy| z1=*cG_YWHG=;D2pcH&j>BR~LD=||sl6du47y2HKZzy7ig(Lp4om1EOypmqW!CRUgD zEyplB<$F|8^rgzkrvi<1!5Grdc*QrK5h^;0_Bdq|5nPT#z+M$#J2q+V0en`cIxSwQ z-kwdYrED}-CdPhILbO>N;?Z%!%ZKJN9UjQM;`g~vMVf2|$GZ41nKLsM0yEyW@fEVN zGXXJG^_g27(I?^zWeBqHiKMZ$%0-K^GCGU_Pe#-_a6_%UHC0wJgmlx zch}V2g7dkOFs)adK<#(-V&YVU`+{aXUMBW2C%w_-j5W(ipJ!>Oa^~w%8<`lwECGGG zPQ*y%wmx@P&&kOfQbVEFbH*1^iWIGwLslaYY7eEm$(}JYZ%G+;DF-_c0iX<^m zp+3{tD8j$8k2fxs3Wir1r5s_*`nV~08IoydXq>BH_au#11=`Qr;o1bt%X@K9+SH_m zLpuq)*J^241CW_4*ig+nERx#=>#*`3h8JWZwrp2%&s-y_zTH&6a;n3DDK=mgNM*zn z#TqHu=dWHB^4VOC@f+@7gI^jcI{H6(aGcn>Yj4i}C4Ka=BLv~={7k*%oP}rKmpT~V zFmjv*{cDttYVRoxnRmR$;L}-+oq0`W9*^3FN#BNWcyr=dwf%>1j~C5KThq#(BXX6M zuWbXHBho;LYr4byEBu`MOIZQ6Yvxsw+U8l|7i$3%sI7x!3&}*CC(3$7!=H-=go==I z#kdSBqLm$2rQb&T*P??ka{F?4$B}EjzUYr$fsdb-bl^_B#5OaTP+O&5wa}o*;R!_S z`@XV*qyQtBv1$qBHx11$$VKw-?3VAEm)yo9d`yBYL%f&R2z7a7nCeYiAW%RSR-j{! zxBs-^=vAowS6wB&03^5opS)_YaUqX5YD^4aB+aR z?)t(72Hw`r{N9%v1mU{H*T0Q-TMGlaJ02JUJDfOxs@?hypc@iUwL z?jINkxw{__2;`6TLLj_<>>GrK7xc%xyg;toEc}l-oE(fSUYa^!2nb-Xt6ID^{rUW{ ltJvD!emnLvYhwrrVK_P&IXL}#I(ay_K-?Jg^iV~~{{fp&jOYLW literal 0 HcmV?d00001 diff --git a/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py b/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py index fbc1c6f9..a0963de3 100644 --- a/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py +++ b/backend/documents_parser/tests/test_summary_pdf_mapper_chain.py @@ -85,6 +85,11 @@ _SUMMARY_000904_PDF = _FIXTURES / "Summary_000904.pdf" # cert 9285 _SUMMARY_000900_PDF = _FIXTURES / "Summary_000900.pdf" # cert 2225 _SUMMARY_000898_PDF = _FIXTURES / "Summary_000898.pdf" # cert 2636 _SUMMARY_000902_PDF = _FIXTURES / "Summary_000902.pdf" # cert 9418 +# simulated case 34 (cert 001431 reconfigured as a slimline electric-storage +# flat with an unheated corridor / sheltered alternative wall + 4 alt-wall +# windows). Regression net for the flat-roof, sheltered-wall, and §11 +# alt-wall-window-allocation fixes. +_SUMMARY_CASE34_PDF = _FIXTURES / "Summary_case34_storage_flat.pdf" _SUMMARY_000889_PDF = _FIXTURES / "Summary_000889.pdf" # cert 2536 (Normal cylinder) _SUMMARY_000884_PDF = _FIXTURES / "Summary_000884.pdf" # cert 9421 (Normal cylinder) _SUMMARY_000910_PDF = _FIXTURES / "Summary_000910.pdf" # cert 0036 (Flat, party wall U=0) @@ -1547,6 +1552,34 @@ def test_summary_mapper_raises_on_unmapped_cylinder_insulation_label() -> None: assert excinfo.value.value == "Polyester wool" +def test_case34_alt_wall_windows_all_allocated_to_alternative_wall() -> None: + # Arrange — simulated case 34 lodges 4 windows on "Alternative wall 1" + # (0.70 + 1.75 + 1.18 + 1.00 = 4.63 m²) and 6 on the external wall. The + # §11 layout interleaves the wrapped "Alternative wall / 1" Location cell + # around each window's data row; for single-glazed alt windows the + # location line carries no glazing-type word, so the partition swallowed + # it into the previous window's suffix and the window defaulted to + # "External wall" — mis-deducting its opening from the wrong wall. + from domain.sap10_calculator.worksheet.heat_transmission import ( + _window_on_alt_wall, # pyright: ignore[reportPrivateUsage] + ) + pages = _summary_pdf_to_textract_style_pages(_SUMMARY_CASE34_PDF) + site_notes = ElmhurstSiteNotesExtractor(pages).extract() + epc = EpcPropertyDataMapper.from_elmhurst_site_notes(site_notes) + + # Act + alt_area = sum( + round(float(w.window_width) * float(w.window_height), 2) + for w in (epc.sap_windows or []) + if _window_on_alt_wall(w) + ) + alt_count = sum(1 for w in (epc.sap_windows or []) if _window_on_alt_wall(w)) + + # Assert — all 4 alt-wall windows recovered (worksheet alt openings 4.63). + assert alt_count == 4 + assert abs(alt_area - 4.63) <= 0.01 + + def test_map_elmhurst_alternative_wall_carries_sheltered_flag() -> None: # Arrange — Elmhurst Summary §7 lodges "Alternative Wall N Sheltered # Wall: Yes" for a sub-area adjacent to an unheated buffer (e.g. a flat's From c10881ae7a57ac494b1dd7f7dc45fa1b3f0a86f3 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Thu, 11 Jun 2026 08:03:06 +0000 Subject: [PATCH 49/87] feat(heat-transmission): door to unheated corridor uses Table 26 U=1.4 on the sheltered wall MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A door opening to an unheated corridor/stairwell takes U=1.4 W/m²K (RdSAP 10 Table 26, p.51 — any age band) instead of the 3.0 external-door default, and its area deducts from the SHELTERED wall, not the main wall (RdSAP §3.7, p.18: "the door of a flat/maisonette to an unheated stairwell or corridor ... is deducted from the sheltered wall area"). The cascade previously billed every door at the external U on the main wall. Signal: a SHELTERED alternative wall (`is_sheltered`, the RdSAP §5.9 wall-to-unheated-corridor surface, already modelled) is the evidence that the dwelling is accessed via an unheated corridor, so one lodged door opens to it. `_corridor_door_count` returns 1 when a sheltered alt wall is present and >=1 door is lodged, else 0 — so the door channel is unchanged for every non-corridor dwelling (houses, exposed-gable flats). `heat_transmission_ from_cert` gains a `corridor_door_count` param (default 0): it splits the door area into external (main wall, age-default U) + corridor (sheltered alt wall, U=1.4), threading the corridor door's area into that wall's opening deduction and billing it at 1.4. Validated on TWO faithful worksheets: simulated case 34 (cert 001431 storage flat — doors 8.14 exact, fabric 207.47 ≈ ws 207.48) and the long-standing worksheet-harness diverger cert 2474 (−0.87 → −0.32, the "space-demand thread" was the dropped corridor door). The worksheet harness is now 47/47 with ZERO divergers. API SAP gauge: 57.6% → 60.0% within 0.5; mean|err| 1.185 → 1.167; signed −0.165 → −0.115 — ~22 sheltered-corridor flats were a systematic gap. Regression gate green (3 pre-existing fails unrelated); pyright net-zero. Co-Authored-By: Claude Opus 4.8 --- .../sap10_calculator/rdsap/cert_to_inputs.py | 22 +++++++++ .../worksheet/heat_transmission.py | 36 +++++++++++++- .../worksheet/test_heat_transmission.py | 47 +++++++++++++++++++ 3 files changed, 103 insertions(+), 2 deletions(-) diff --git a/domain/sap10_calculator/rdsap/cert_to_inputs.py b/domain/sap10_calculator/rdsap/cert_to_inputs.py index af3448b9..fd3d3190 100644 --- a/domain/sap10_calculator/rdsap/cert_to_inputs.py +++ b/domain/sap10_calculator/rdsap/cert_to_inputs.py @@ -4011,9 +4011,31 @@ def heat_transmission_section_from_cert(epc: EpcPropertyData) -> HeatTransmissio insulated_door_count=epc.insulated_door_count, insulated_door_u_value=epc.insulated_door_u_value, exposure=exposure, + corridor_door_count=_corridor_door_count(epc), ) +def _corridor_door_count(epc: EpcPropertyData) -> int: + """RdSAP §3.7 + Table 26 — number of doors opening to an unheated + corridor/stairwell (each billed at U=1.4 on the sheltered wall). + + The presence of a SHELTERED alternative wall (`is_sheltered`, the + RdSAP §5.9 wall-to-unheated-corridor surface) is the evidence that the + dwelling is accessed via an unheated corridor, so its entrance door + opens to that corridor. RdSAP convention assumes one such access door + when the sheltered wall is present and the cert lodges at least one + door; the remainder are external. Returns 0 when no sheltered alt wall + is lodged (houses, exposed-gable flats) so the door channel is + unchanged for every non-corridor dwelling. + """ + has_sheltered_alt = any( + (bp.sap_alternative_wall_1 is not None and bp.sap_alternative_wall_1.is_sheltered) + or (bp.sap_alternative_wall_2 is not None and bp.sap_alternative_wall_2.is_sheltered) + for bp in (epc.sap_building_parts or []) + ) + return 1 if has_sheltered_alt and epc.door_count > 0 else 0 + + def _rooflight_total_area_m2_from_cert(epc: EpcPropertyData) -> float: """Σ area of `epc.sap_roof_windows` for §5 daylight-factor L2a + §6 horizontal solar gain. Returns 0.0 when none are lodged. diff --git a/domain/sap10_calculator/worksheet/heat_transmission.py b/domain/sap10_calculator/worksheet/heat_transmission.py index 620deded..ceb086ce 100644 --- a/domain/sap10_calculator/worksheet/heat_transmission.py +++ b/domain/sap10_calculator/worksheet/heat_transmission.py @@ -117,6 +117,11 @@ def _decimal_round_half_up_product(a: float, b: float, dp: int) -> float: _WALL_INSULATION_NONE: Final[int] = 4 _DEFAULT_DOOR_AREA_M2: Final[float] = 1.85 +# RdSAP 10 Table 26 (PDF p.51): a door opening to an unheated corridor or +# stairwell takes U=1.4 W/m²K for any age band (vs 3.0 for an external door +# A-J). The door sits on the sheltered wall (RdSAP §3.7 p.18) and its area +# deducts from that wall, not the main wall. +_CORRIDOR_DOOR_U_W_PER_M2K: Final[float] = 1.4 _DEFAULT_STOREY_HEIGHT_M: Final[float] = 2.5 # SAP10.2 §3.2 curtain/blind thermal resistance applied to windows (and # roof windows) — turns raw window U into the worksheet's (27) effective U. @@ -569,6 +574,7 @@ def heat_transmission_from_cert( insulated_door_count: int = 0, insulated_door_u_value: Optional[float] = None, exposure: Optional[DwellingExposure] = None, + corridor_door_count: int = 0, ) -> HeatTransmission: """Conduction HLC + thermal-bridging contribution, summed across every sap_building_part in the cert. Windows and doors are apportioned to the @@ -590,7 +596,18 @@ def heat_transmission_from_cert( floor_description = _joined_descriptions(epc.floors) # RdSAP10 §15 — door area rounds to 2 d.p. before entering the calc. - door_area = _round_half_up(max(0, door_count) * _DEFAULT_DOOR_AREA_M2, _AREA_ROUND_DP) + # A door to an unheated corridor (RdSAP Table 26 / §3.7) is billed at + # U=1.4 and its area deducts from the sheltered wall, not the main wall. + # Only the remaining EXTERNAL doors stay on the main wall at the + # age-default U; the corridor door area is tracked separately and + # assigned to the first sheltered alt wall in the BP loop below. + corridor_door_count = max(0, min(corridor_door_count, door_count)) + external_door_count = max(0, door_count - corridor_door_count) + door_area = _round_half_up(external_door_count * _DEFAULT_DOOR_AREA_M2, _AREA_ROUND_DP) + corridor_door_area = _round_half_up( + corridor_door_count * _DEFAULT_DOOR_AREA_M2, _AREA_ROUND_DP, + ) + corridor_door_area_remaining = corridor_door_area # SAP10.2 §3.2: effective window U includes the 0.04 m²K/W curtain # resistance — `(27)` worksheet column applies it per-window. When # sap_windows have per-window U lodgements (mixed glazing types in @@ -1009,12 +1026,20 @@ def heat_transmission_from_cert( continue # RdSAP10 §15 — alt wall area rounded to 2 d.p. alt_walls_total_area += _round_half_up(alt_wall.wall_area, _AREA_ROUND_DP) + alt_opening = alt_window_area if idx == 0 else 0.0 + # RdSAP §3.7 (p.18): the door to an unheated corridor deducts + # from the SHELTERED wall area. Attach the corridor door to the + # first sheltered alt wall (its U=1.4 contribution is billed in + # the door channel below, not here). + if alt_wall.is_sheltered and corridor_door_area_remaining > 0.0: + alt_opening += corridor_door_area_remaining + corridor_door_area_remaining = 0.0 alt_walls_contribution += _alt_wall_w_per_k( alt_wall=alt_wall, country=country, age_band=age_band, wall_description=wall_description, - opening_area_m2=alt_window_area if idx == 0 else 0.0, + opening_area_m2=alt_opening, ) # Main wall net adds back the alt-wall windows that were initially # deducted from the BP's total gross — those openings should have @@ -1263,6 +1288,13 @@ def heat_transmission_from_cert( total_external_area += part_external_area bridging += y * part_external_area + # RdSAP Table 26 — the unheated-corridor door's heat loss (U=1.4). Its + # area was deducted from the sheltered alt wall above, so the alt wall + # net (and hence its W/K) already excludes it; bill it here at the + # corridor U. (31) is unaffected: the door area stays counted in the + # alt-wall gross contribution, equivalent to the worksheet's separate + # door line. + doors += _CORRIDOR_DOOR_U_W_PER_M2K * corridor_door_area roof_windows_w_per_k = roof_windows_w_per_k_total fabric_heat_loss = ( walls + roof + floor + party + windows + roof_windows_w_per_k + doors # (33) diff --git a/tests/domain/sap10_calculator/worksheet/test_heat_transmission.py b/tests/domain/sap10_calculator/worksheet/test_heat_transmission.py index 6dfbff44..c1632a39 100644 --- a/tests/domain/sap10_calculator/worksheet/test_heat_transmission.py +++ b/tests/domain/sap10_calculator/worksheet/test_heat_transmission.py @@ -1228,6 +1228,53 @@ def test_sheltered_alternative_wall_applies_table4_0p5_resistance() -> None: assert sheltered_wpk < exposed_wpk +def test_corridor_door_on_sheltered_alt_wall_uses_table26_u_1p4() -> None: + # Arrange — RdSAP 10 Table 26 (PDF p.51): a door opening to an unheated + # corridor/stairwell has U=1.4 (any age), versus 3.0 for an external + # door (age A-J). RdSAP §3.7 (p.18): "the door of a flat/maisonette to + # an unheated stairwell or corridor ... is deducted from the sheltered + # wall area" — i.e. the corridor door sits on the sheltered alt wall, + # not the main wall. Simulated case 34: a flat with a sheltered alt + # (corridor) wall + 2 doors → 1 corridor door (U=1.4 on the alt wall) + + # 1 external door (U=3.0 on the main wall). + from dataclasses import replace + + main = replace( + make_building_part( + construction_age_band="B", + wall_construction=4, wall_insulation_type=4, + party_wall_construction=1, roof_construction=4, + floor_dimensions=[ + make_floor_dimension( + total_floor_area_m2=50.0, room_height_m=2.5, + party_wall_length_m=0.0, heat_loss_perimeter_m=28.0, floor=0, + ), + ], + ), + sap_alternative_wall_1=SapAlternativeWall( + wall_area=12.5, wall_dry_lined="N", wall_construction=4, + wall_insulation_type=4, wall_thickness_measured="Y", + wall_insulation_thickness="NI", is_sheltered=True, + ), + ) + epc = make_minimal_sap10_epc( + total_floor_area_m2=50.0, country_code="ENG", sap_building_parts=[main], + ) + + # Act + no_corridor = heat_transmission_from_cert(epc, door_count=2, corridor_door_count=0) + with_corridor = heat_transmission_from_cert(epc, door_count=2, corridor_door_count=1) + + # Assert — no-corridor: both doors external at the age-default U. + door_u = no_corridor.doors_w_per_k / (2 * 1.85) + # with-corridor: 1 external @door_u + 1 corridor @1.4 (both 1.85 m²). + assert abs(with_corridor.doors_w_per_k - (1.85 * door_u + 1.85 * 1.4)) <= 0.02 + # The corridor door (U=1.4) is cheaper than an external door (U≈3.0) and + # its area moves off the main wall onto the sheltered alt wall, so the + # net fabric heat loss drops. + assert with_corridor.fabric_heat_loss_w_per_k < no_corridor.fabric_heat_loss_w_per_k + + def test_window_uses_effective_u_value_with_curtain_resistance_per_sap10_2_section_3_2() -> None: """SAP10.2 §3.2: the window U-value used for heat-transmission is the effective form `U_eff = 1/(1/U_raw + 0.04)` — the 0.04 m²K/W is the From 450e33e15db4a2afbb78642dcf7ad55c5cdd65cd Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Thu, 11 Jun 2026 09:00:54 +0000 Subject: [PATCH 50/87] =?UTF-8?q?fix(ventilation):=20corridor=20flat=20ass?= =?UTF-8?q?umes=20a=20draught=20lobby,=20zeroing=20=C2=A72=20(13)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A flat accessed via an unheated corridor/stairwell assumes a draught lobby is present, so SAP 10.2 §2 line (13) = 0.0 rather than the 0.05 no-lobby infiltration penalty. Per RdSAP 10 Specification (10-06-2025, p.30, "Draught lobby"): "add infiltration 0.05 if draught lobby is not present, or use 0.0 if present. ... Flat or maisonette: Assume draught lobby if entrance door is facing corridor (heated or unheated) or stairwell." Signal: a SHELTERED alternative wall (the RdSAP §5.9 wall-to-unheated-corridor surface) is the evidence that the flat's entrance faces a corridor — the same evidence the corridor door (Table 26 U=1.4) rides on. New helper `_has_sheltered_corridor_wall` factors that check out of `_corridor_door_count` and gates `_has_draught_lobby`. Houses and exposed-gable flats (no sheltered alt wall) keep the lodged value / "assume no lobby if cannot be determined" default, so the §2 cascade is unchanged for every non-corridor dwelling. The cascade previously added the 0.05 penalty unconditionally, over-counting (16)/(18)/(21) by 0.05 ACH. On simulated case 34 (cert 001431 storage flat) this lifted effective air change (25)m from the worksheet's monthly 0.572-0.638 to 0.574-0.668, over-counting space-heating demand (98) by +46.3 kWh/yr (+0.41%) -> SAP -0.18. Closing it lands (25)m exactly on the worksheet (avg 0.6024) and (98) at 11356.3 vs ws 11357.2: case 34 SAP 35.1325 -> 35.3130 vs ws 35.3094 (Δ -0.1769 -> +0.0036) Guard-rails held (both improved): worksheet harness 47/47, 0 divergers (the other corridor flat, cert 2474, -0.32 -> -0.02); API gauge 60.0% -> 60.1% within 0.5, mean|err| 1.167 -> 1.163. Co-Authored-By: Claude Opus 4.8 --- .../sap10_calculator/rdsap/cert_to_inputs.py | 57 ++++++++++++++----- .../rdsap/test_cert_to_inputs.py | 47 +++++++++++++++ 2 files changed, 90 insertions(+), 14 deletions(-) diff --git a/domain/sap10_calculator/rdsap/cert_to_inputs.py b/domain/sap10_calculator/rdsap/cert_to_inputs.py index fd3d3190..0ef51687 100644 --- a/domain/sap10_calculator/rdsap/cert_to_inputs.py +++ b/domain/sap10_calculator/rdsap/cert_to_inputs.py @@ -4015,25 +4015,54 @@ def heat_transmission_section_from_cert(epc: EpcPropertyData) -> HeatTransmissio ) -def _corridor_door_count(epc: EpcPropertyData) -> int: - """RdSAP §3.7 + Table 26 — number of doors opening to an unheated - corridor/stairwell (each billed at U=1.4 on the sheltered wall). +def _has_sheltered_corridor_wall(epc: EpcPropertyData) -> bool: + """Whether the dwelling is accessed via an unheated corridor/stairwell. - The presence of a SHELTERED alternative wall (`is_sheltered`, the - RdSAP §5.9 wall-to-unheated-corridor surface) is the evidence that the - dwelling is accessed via an unheated corridor, so its entrance door - opens to that corridor. RdSAP convention assumes one such access door - when the sheltered wall is present and the cert lodges at least one - door; the remainder are external. Returns 0 when no sheltered alt wall - is lodged (houses, exposed-gable flats) so the door channel is - unchanged for every non-corridor dwelling. + A SHELTERED alternative wall (`is_sheltered`, the RdSAP §5.9 + wall-to-unheated-corridor surface) is the evidence that the dwelling's + entrance faces an unheated corridor or stairwell. False for houses and + exposed-gable flats (no sheltered alt wall lodged). """ - has_sheltered_alt = any( + return any( (bp.sap_alternative_wall_1 is not None and bp.sap_alternative_wall_1.is_sheltered) or (bp.sap_alternative_wall_2 is not None and bp.sap_alternative_wall_2.is_sheltered) for bp in (epc.sap_building_parts or []) ) - return 1 if has_sheltered_alt and epc.door_count > 0 else 0 + + +def _corridor_door_count(epc: EpcPropertyData) -> int: + """RdSAP §3.7 + Table 26 — number of doors opening to an unheated + corridor/stairwell (each billed at U=1.4 on the sheltered wall). + + A sheltered alternative wall (`_has_sheltered_corridor_wall`) is the + evidence that the dwelling is accessed via an unheated corridor, so its + entrance door opens to that corridor. RdSAP convention assumes one such + access door when the sheltered wall is present and the cert lodges at + least one door; the remainder are external. Returns 0 when no sheltered + alt wall is lodged (houses, exposed-gable flats) so the door channel is + unchanged for every non-corridor dwelling. + """ + return 1 if _has_sheltered_corridor_wall(epc) and epc.door_count > 0 else 0 + + +def _has_draught_lobby(epc: EpcPropertyData, sv: Optional[SapVentilation]) -> bool: + """RdSAP 10 §2 (13) — presence of a draught lobby. + + Spec (RdSAP 10 Specification 10-06-2025, p.30, "Draught lobby"): + "add infiltration 0.05 if draught lobby is not present, or use 0.0 if + present. ... Flat or maisonette: Assume draught lobby if entrance door + is facing corridor (heated or unheated) or stairwell." + + A sheltered corridor wall (`_has_sheltered_corridor_wall`) is exactly + that evidence: the flat's entrance faces an unheated corridor/stairwell, + so a draught lobby is assumed present regardless of the lodged value. + Otherwise fall back to the lodged value — which, when undetermined, is + the RdSAP "assume no draught lobby if cannot be determined" default for + houses. + """ + if _has_sheltered_corridor_wall(epc): + return True + return bool(sv.has_draught_lobby) if sv is not None and sv.has_draught_lobby is not None else False def _rooflight_total_area_m2_from_cert(epc: EpcPropertyData) -> float: @@ -4829,7 +4858,7 @@ def ventilation_from_cert( flueless_gas_fires=vc.flueless_gas_fires, has_suspended_timber_floor=eff_has_susp, suspended_timber_floor_sealed=eff_sealed, - has_draught_lobby=bool(sv.has_draught_lobby) if sv is not None and sv.has_draught_lobby is not None else False, + has_draught_lobby=_has_draught_lobby(epc, sv), window_pct_draught_proofed=float(epc.percent_draughtproofed or 0), sheltered_sides=int(sv.sheltered_sides) if sv is not None and sv.sheltered_sides is not None else 2, air_permeability_ap4=ap4, diff --git a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py index b5497ec8..8a055253 100644 --- a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py +++ b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py @@ -30,6 +30,7 @@ from datatypes.epc.domain.epc_property_data import ( EpcPropertyData, MainHeatingDetail, PhotovoltaicArray, + SapAlternativeWall, SapFloorDimension, SapVentilation, ) @@ -1253,6 +1254,52 @@ def test_ventilation_from_cert_routes_mev_decentralised_to_extract_or_piv_outsid assert abs(w25 - expected) <= 1e-4 +def test_corridor_flat_assumes_draught_lobby_present_zeroing_line_13() -> None: + # Arrange — RdSAP 10 Specification (10-06-2025) p.30, "Draught lobby": + # "add infiltration 0.05 if draught lobby is not present, or use 0.0 if + # present. ... Flat or maisonette: Assume draught lobby if entrance door + # is facing corridor (heated or unheated) or stairwell." A SHELTERED + # alternative wall is the RdSAP §5.9 wall-to-unheated-corridor surface — + # the same evidence the corridor door rides on — so the flat's entrance + # faces a corridor and a draught lobby is assumed present, zeroing line + # (13). Simulated case 34 (cert 001431 storage flat): the cascade + # previously added the 0.05 no-lobby penalty, over-counting (16)/(18) by + # 0.05 ACH → +46 kWh/yr space demand → SAP −0.18. + from dataclasses import replace + + corridor_part = replace( + make_building_part(construction_age_band="G"), + sap_alternative_wall_1=SapAlternativeWall( + wall_area=12.5, wall_dry_lined="N", wall_construction=4, + wall_insulation_type=4, wall_thickness_measured="Y", + wall_insulation_thickness="NI", is_sheltered=True, + ), + ) + exposed_part = make_building_part(construction_age_band="G") + corridor_flat = make_minimal_sap10_epc( + total_floor_area_m2=_TYPICAL_TFA_M2, country_code="ENG", + sap_building_parts=[corridor_part], + ) + exposed_flat = make_minimal_sap10_epc( + total_floor_area_m2=_TYPICAL_TFA_M2, country_code="ENG", + sap_building_parts=[exposed_part], + ) + + # Act + v_corridor = ventilation_from_cert(corridor_flat) + v_exposed = ventilation_from_cert(exposed_flat) + + # Assert — the corridor flat zeroes (13); a flat with no sheltered + # corridor wall keeps the 0.05 no-lobby penalty (cannot be determined). + assert abs(v_corridor.draught_lobby_ach - 0.0) <= 1e-9 + assert abs(v_exposed.draught_lobby_ach - 0.05) <= 1e-9 + # The lobby removes 0.05 ACH from (16); shelter (21) drops proportionally. + assert v_corridor.infiltration_rate_ach < v_exposed.infiltration_rate_ach + assert abs( + (v_exposed.infiltration_rate_ach - v_corridor.infiltration_rate_ach) - 0.05 + ) <= 1e-9 + + def test_index_less_mev_uses_table_4g_default_sfp_for_fan_electricity() -> None: # Arrange — an MEV system with NO PCDB record (index absent / not in # Table 322). SAP 10.2 §2.6.3 / Table 4g note 1 prescribes a default From b7d283cd3ae39a4f59d97b2bb3b1e804e966416e Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Thu, 11 Jun 2026 09:01:41 +0000 Subject: [PATCH 51/87] docs(profile-case34): mark the space-demand residual closed (450e33e1) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The §2 (13) draught-lobby fix landed the +46.3 kWh space-heating over-count on the worksheet; the tracked diagnostic's header and run-banner now reflect the closed state (Δ +0.0036 SAP, sub-2dp-rounding) instead of the open gap. Co-Authored-By: Claude Opus 4.8 --- scripts/profile_case34.py | 95 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+) create mode 100644 scripts/profile_case34.py diff --git a/scripts/profile_case34.py b/scripts/profile_case34.py new file mode 100644 index 00000000..21484cc2 --- /dev/null +++ b/scripts/profile_case34.py @@ -0,0 +1,95 @@ +"""Decompose simulated case 34 (electric-storage corridor flat) vs its +dr87/P960 worksheet, channel by channel. + +Routes the tracked fixture +`backend/documents_parser/tests/fixtures/Summary_case34_storage_flat.pdf` +through extractor -> mapper -> calculator (Summary path) and prints the §3 +heat-transmission breakdown + the space-heating demand / ventilation / +gains / MIT intermediates against the worksheet line refs. + +The fabric (33), bridging (36), (31) and the door channel are EXACT after +HEAD c10881ae. The +46.3 kWh/yr (+0.41%) space-heating over-count was the +§2 (13) draught lobby: a corridor flat assumes a lobby (0.0), not the 0.05 +no-lobby penalty (RdSAP 10 spec p.30) — closed at HEAD 450e33e1, which lands +effective air change (25)m on the worksheet (avg 0.6024) and SAP at 35.3130 +vs ws 35.3094 (Δ +0.0036, sub-2dp-rounding). The residual now sits at the +worksheet's own 2dp display floor (walls -0.017 W/K). Use this to audit. + + PYTHONPATH=/workspaces/model python scripts/profile_case34.py +""" +import re +import subprocess +from pathlib import Path + +from backend.documents_parser.elmhurst_extractor import ElmhurstSiteNotesExtractor +from datatypes.epc.domain.mapper import EpcPropertyDataMapper +from domain.sap10_calculator.calculator import calculate_sap_from_inputs +from domain.sap10_calculator.rdsap.cert_to_inputs import ( + SAP_10_2_SPEC_PRICES, + cert_to_inputs, + heat_transmission_section_from_cert, +) + +_FIXTURE = ( + Path(__file__).parent.parent + / "backend/documents_parser/tests/fixtures/Summary_case34_storage_flat.pdf" +) + + +def _pages(pdf: Path) -> list[str]: + info = subprocess.run( + ["pdfinfo", str(pdf)], capture_output=True, text=True, check=True + ).stdout + pc = int(re.search(r"Pages:\s+(\d+)", info).group(1)) # type: ignore[union-attr] + pages: list[str] = [] + for i in range(1, pc + 1): + layout = subprocess.run( + ["pdftotext", "-layout", "-f", str(i), "-l", str(i), str(pdf), "-"], + capture_output=True, text=True, check=True, + ).stdout + toks: list[str] = [] + for line in layout.splitlines(): + if not line.strip(): + toks.append("") + continue + toks.extend([p for p in re.split(r"\s{2,}", line.strip()) if p]) + pages.append("\n".join(toks)) + return pages + + +def main() -> None: + sn = ElmhurstSiteNotesExtractor(_pages(_FIXTURE)).extract() + epc = EpcPropertyDataMapper.from_elmhurst_site_notes(sn) + ht = heat_transmission_section_from_cert(epc) + print("== §3 HEAT TRANSMISSION (all EXACT at c10881ae) ==") + print(f"(31) ext elem area = {ht.total_external_element_area_m2:.4f} | ws 120.369") + print(f"(29a) walls = {ht.walls_w_per_k:.4f} | ws 30.902") + print(f"(30) roof = {ht.roof_w_per_k:.4f} | ws 91.954") + print(f"(28a) floor = {ht.floor_w_per_k:.4f} | ws 27.986") + print(f"(26) doors = {ht.doors_w_per_k:.4f} | ws 8.14 (corridor 1.4 + ext 3.0)") + print(f"(33) fabric = {ht.fabric_heat_loss_w_per_k:.4f} | ws 207.484") + print(f"(36) bridging = {ht.thermal_bridging_w_per_k:.4f} | ws 18.054") + print(f"(37) total fabric = {ht.total_w_per_k:.4f} | ws 225.538") + + res = calculate_sap_from_inputs(cert_to_inputs(epc, prices=SAP_10_2_SPEC_PRICES)) + im = res.intermediate + print("\n== RESIDUAL CLOSED at 450e33e1: was +46.3 kWh, now -0.96 kWh ==") + keys = [ + "useful_space_heating_kwh_per_yr", # ws (98) = 11357.24 + "infiltration_ach", "infiltration_w_per_k", + "internal_gains_annual_avg_w", # ws (73)/(84) + "mean_internal_temp_annual_avg_c", # ws (85)-(93) + ] + for k in keys: + if k in im: + print(f" {k} = {round(im[k], 4)}") + print( + "\nworksheet targets: (98) space heat=11357.24 | (35) TMP=250 |" + " (25)m eff ach 0.46-0.64 | (20) shelter=0.85 | (19) sheltered sides=2" + ) + print(f"\nSAP cont = {res.sap_score_continuous:.4f} | ws 35.3094 | " + f"Δ = {res.sap_score_continuous - 35.3094:+.4f}") + + +if __name__ == "__main__": + main() From 781efd75c0196239037b8900a9fa46d341dfffe6 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Thu, 11 Jun 2026 09:43:50 +0000 Subject: [PATCH 52/87] fix(heat-transmission): apply dry-lining Table 14 R=0.17 to the main wall MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The main-wall `u_wall(...)` call dropped the `dry_lined` kwarg, so the RdSAP 10 §5.7/§5.8 (PDF p.40-41) Table 14 dry-lining adjustment — U_adj = 1/(1/U₀ + 0.17) for a dry-lined (incl. lath-and-plaster) uninsulated wall — was never applied to any main wall, even when the cert lodged `wall_dry_lined=Y`. The ALTERNATIVE-wall path already passes `dry_lined` (line 1367); this one-sided omission billed every dry-lined main wall at the un-adjusted (too-high) U → wall heat loss too high → SAP under-rated. Per-cert: a solid-brick (construction 3) band-A 230 mm main wall computes U₀=1.70; dry-lined it is 1/(1/1.70+0.17)=1.32 — we were 22% too high. Across the API gov-EPC sample the dry-lined `wall_construction=3` (solid brick) sub-cohort sat at 10% within-0.5 / signed -1.33. Fix: pass `dry_lined=bool(part.wall_dry_lined)` to the main-wall `u_wall` call, mirroring the alt-wall path. `part.wall_dry_lined` is already plumbed (Optional[bool], None → False). The three dry-lining branches in `u_wall` (stone §5.6, solid-brick-by-thickness §5.7, generic uninsulated bucket §5.8) are all spec-correct and already worksheet-validated (the bucket-0 cavity case against cert 7700 age-C → 1.20). Worksheet harness UNAFFECTED (47/47, 0 divergers): the Elmhurst/Summary extractor only captures dry-lining for ALTERNATIVE walls (Summary §7), never the main wall, so `part.wall_dry_lined` stays None on that path — this is a pure API-path improvement. API gauge: within-0.5 60.1% -> 64.4% (mean|err| 1.163 -> 1.085, signed -0.097 -> +0.049). Both affected buckets improved with no overshoot: solid brick (wc=3) 50% -> 57% within-0.5; cavity (wc=4, dry-lined via the §5.8 bucket-0 path) 68% -> 72%. Co-Authored-By: Claude Opus 4.8 --- .../worksheet/heat_transmission.py | 6 +++ .../worksheet/test_heat_transmission.py | 44 +++++++++++++++++++ 2 files changed, 50 insertions(+) diff --git a/domain/sap10_calculator/worksheet/heat_transmission.py b/domain/sap10_calculator/worksheet/heat_transmission.py index ceb086ce..5af20455 100644 --- a/domain/sap10_calculator/worksheet/heat_transmission.py +++ b/domain/sap10_calculator/worksheet/heat_transmission.py @@ -812,6 +812,12 @@ def heat_transmission_from_cert( # code feeds the documentary-evidence R-value calc when a # measured wall thickness is also present (else ignored). wall_insulation_thermal_conductivity=part.wall_insulation_thermal_conductivity, + # RdSAP 10 §5.7/§5.8 (PDF p.40-41), Table 14 — a dry-lined + # (incl. lath-and-plaster) uninsulated wall adds R=0.17. + # The alt-wall path already passes this; the main wall must + # too, else every lodged `wall_dry_lined=Y` main wall is + # billed at the un-adjusted U. + dry_lined=bool(part.wall_dry_lined), ) # When the per-bp `roof_insulation_thickness` is explicitly lodged # as 0 (uninsulated — e.g. cert 001479 Ext2 PS sloping ceiling diff --git a/tests/domain/sap10_calculator/worksheet/test_heat_transmission.py b/tests/domain/sap10_calculator/worksheet/test_heat_transmission.py index c1632a39..b6a21d0a 100644 --- a/tests/domain/sap10_calculator/worksheet/test_heat_transmission.py +++ b/tests/domain/sap10_calculator/worksheet/test_heat_transmission.py @@ -1275,6 +1275,50 @@ def test_corridor_door_on_sheltered_alt_wall_uses_table26_u_1p4() -> None: assert with_corridor.fabric_heat_loss_w_per_k < no_corridor.fabric_heat_loss_w_per_k +def test_main_wall_dry_lining_applies_table_14_resistance() -> None: + # Arrange — RdSAP 10 §5.7/§5.8 (PDF p.40-41), Table 14: a dry-lined + # (including lath-and-plaster) uninsulated wall adds R=0.17 m²K/W: + # U_adj = 1/(1/U₀ + 0.17). A solid-brick (construction 3) age-A wall + # with a measured 230 mm thickness has U₀=1.70 (Table 13, 200-280 mm + # band) → dry-lined U=1/(1/1.70+0.17)=1.32 (2 d.p.). The alt-wall path + # already applies this; the MAIN wall dropped the `dry_lined` kwarg, so + # every lodged `wall_dry_lined=Y` main wall was billed at the un-adjusted + # U — under-rating solid-brick stock (API wall_construction=3 cohort: + # 48 dry-lined certs at 10% within-0.5, signed -1.33). + from dataclasses import replace + + base_part = make_building_part( + construction_age_band="A", + wall_construction=3, # solid brick + wall_insulation_type=0, # uninsulated (as-built) + floor_dimensions=[ + make_floor_dimension( + total_floor_area_m2=50.0, room_height_m=2.5, + party_wall_length_m=0.0, heat_loss_perimeter_m=28.0, floor=0, + ), + ], + ) + not_dry = replace(base_part, wall_thickness_mm=230, wall_dry_lined=False) + dry = replace(base_part, wall_thickness_mm=230, wall_dry_lined=True) + epc_not_dry = make_minimal_sap10_epc( + total_floor_area_m2=50.0, country_code="ENG", sap_building_parts=[not_dry], + ) + epc_dry = make_minimal_sap10_epc( + total_floor_area_m2=50.0, country_code="ENG", sap_building_parts=[dry], + ) + + # Act + ht_not_dry = heat_transmission_from_cert(epc_not_dry, door_count=0) + ht_dry = heat_transmission_from_cert(epc_dry, door_count=0) + + # Assert — same net wall area, so the W/K ratio is the U ratio: the + # dry-lined wall is 1.32/1.70 = 0.776× the as-built wall. + assert ht_not_dry.walls_w_per_k > 0.0 + expected_ratio = 1.32 / 1.70 + assert abs(ht_dry.walls_w_per_k / ht_not_dry.walls_w_per_k - expected_ratio) <= 0.005 + assert ht_dry.fabric_heat_loss_w_per_k < ht_not_dry.fabric_heat_loss_w_per_k + + def test_window_uses_effective_u_value_with_curtain_resistance_per_sap10_2_section_3_2() -> None: """SAP10.2 §3.2: the window U-value used for heat-transmission is the effective form `U_eff = 1/(1/U_raw + 0.04)` — the 0.04 m²K/W is the From a97ff60b01def0da0522e7540c035099fdb97cbf Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Thu, 11 Jun 2026 10:02:39 +0000 Subject: [PATCH 53/87] fix(water-heating): complete RdSAP Table 28 cylinder-size map (codes 5 + 6) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `_CYLINDER_SIZE_CODE_TO_LITRES` held only codes 2/3/4 (Normal/Medium/Large → 110/160/210 L); codes 5 (Inaccessible) and 6 (Exact) fell through to None, so the Table-13 high-rate fraction AND the cylinder storage loss were skipped for those certs (20 code-6 certs in the API sample). Per RdSAP 10 Specification (10-06-2025) §10.5 Table 28 (PDF p.55): - Code 6 "Exact": use the lodged measured volume. The gov API carries it in `cylinder_size_measured` (e.g. 150 L) — now plumbed through the 21.0.0/21.0.1 schema → mapper → `SapHeating.cylinder_volume_measured_l`. - Code 5 "Inaccessible": 210 L if off-peak electric dual immersion, 160 L from a solid-fuel boiler, otherwise 110 L (n=0 in the current sample, but spec-complete). New `_cylinder_volume_l_from_code` centralises Table 28 resolution and replaces the three raw-dict call sites (`_hot_water_cylinder_volume_l`, the cylinder storage-loss path, and the PCDB performance check) so all three honour codes 5/6 identically. `_cylinder_inaccessible_volume_l` applies the code-5 context rule via the existing immersion/off-peak-meter/solid-fuel-boiler detectors. Worksheet harness UNAFFECTED (47/47, 0 divergers): the Summary path lodges neither code 5/6 nor a measured volume. API gauge: within-0.5 64.4% -> 65.1% (mean|err| 1.085 -> 1.075) — the 20 code-6 certs now size their cylinder from the measured volume. 4 AAA tests (code 6 measured; code 5 solid-fuel/default/ off-peak-dual-immersion). pyright net-zero. Co-Authored-By: Claude Opus 4.8 --- datatypes/epc/domain/epc_property_data.py | 4 + datatypes/epc/domain/mapper.py | 2 + datatypes/epc/schema/rdsap_schema_21_0_0.py | 3 + datatypes/epc/schema/rdsap_schema_21_0_1.py | 3 + .../sap10_calculator/rdsap/cert_to_inputs.py | 68 ++++++++++--- domain/sap10_ml/tests/_fixtures.py | 4 + .../rdsap/test_cert_to_inputs.py | 99 +++++++++++++++++++ 7 files changed, 167 insertions(+), 16 deletions(-) diff --git a/datatypes/epc/domain/epc_property_data.py b/datatypes/epc/domain/epc_property_data.py index 6649ade1..d5c11b1c 100644 --- a/datatypes/epc/domain/epc_property_data.py +++ b/datatypes/epc/domain/epc_property_data.py @@ -140,6 +140,10 @@ class SapHeating: cylinder_size: Optional[Union[int, str]] = ( None # int code from API; str (e.g. "Normal (90-130 litres)") from site notes ) + # RdSAP 10 §10.5 Table 28 — the lodged measured cylinder volume in + # litres, present only when `cylinder_size` is the "Exact" descriptor + # (gov-API code 6, field `cylinder_size_measured`). None otherwise. + cylinder_volume_measured_l: Optional[int] = None water_heating_code: Optional[int] = None # TODO: make enum? water_heating_fuel: Optional[int] = None # TODO: make enum? immersion_heating_type: Optional[Union[int, str]] = None # TODO: make enum? diff --git a/datatypes/epc/domain/mapper.py b/datatypes/epc/domain/mapper.py index b47db127..1512ea44 100644 --- a/datatypes/epc/domain/mapper.py +++ b/datatypes/epc/domain/mapper.py @@ -1331,6 +1331,7 @@ class EpcPropertyDataMapper: has_fixed_air_conditioning=schema.sap_heating.has_fixed_air_conditioning == "true", cylinder_size=schema.sap_heating.cylinder_size, + cylinder_volume_measured_l=schema.sap_heating.cylinder_size_measured, water_heating_code=schema.sap_heating.water_heating_code, water_heating_fuel=schema.sap_heating.water_heating_fuel, immersion_heating_type=schema.sap_heating.immersion_heating_type, @@ -1622,6 +1623,7 @@ class EpcPropertyDataMapper: has_fixed_air_conditioning=schema.sap_heating.has_fixed_air_conditioning == "true", cylinder_size=schema.sap_heating.cylinder_size, + cylinder_volume_measured_l=schema.sap_heating.cylinder_size_measured, water_heating_code=schema.sap_heating.water_heating_code, water_heating_fuel=schema.sap_heating.water_heating_fuel, immersion_heating_type=schema.sap_heating.immersion_heating_type, diff --git a/datatypes/epc/schema/rdsap_schema_21_0_0.py b/datatypes/epc/schema/rdsap_schema_21_0_0.py index 8fceb878..8235569c 100644 --- a/datatypes/epc/schema/rdsap_schema_21_0_0.py +++ b/datatypes/epc/schema/rdsap_schema_21_0_0.py @@ -76,6 +76,9 @@ class SapHeating: secondary_fuel_type: Optional[int] = None secondary_heating_type: Optional[int] = None cylinder_insulation_thickness: Optional[int] = None + # RdSAP 10 §10.5 Table 28 — measured cylinder volume (litres), lodged + # only when `cylinder_size` is the "Exact" descriptor (code 6). + cylinder_size_measured: Optional[int] = None @dataclass diff --git a/datatypes/epc/schema/rdsap_schema_21_0_1.py b/datatypes/epc/schema/rdsap_schema_21_0_1.py index c12bf31c..ef30581e 100644 --- a/datatypes/epc/schema/rdsap_schema_21_0_1.py +++ b/datatypes/epc/schema/rdsap_schema_21_0_1.py @@ -81,6 +81,9 @@ class SapHeating: secondary_fuel_type: Optional[int] = None secondary_heating_type: Optional[int] = None cylinder_insulation_thickness: Optional[int] = None + # RdSAP 10 §10.5 Table 28 — measured cylinder volume (litres), lodged + # only when `cylinder_size` is the "Exact" descriptor (code 6). + cylinder_size_measured: Optional[int] = None @dataclass diff --git a/domain/sap10_calculator/rdsap/cert_to_inputs.py b/domain/sap10_calculator/rdsap/cert_to_inputs.py index 0ef51687..9a90c0fe 100644 --- a/domain/sap10_calculator/rdsap/cert_to_inputs.py +++ b/domain/sap10_calculator/rdsap/cert_to_inputs.py @@ -5082,11 +5082,55 @@ _TABLE_3A_COMBI_LOSS_MAIN_HEATING_CATEGORIES: Final[frozenset[int]] = frozenset( # code 3 → Medium (160 litres) (certs 0350, 0380, 2225, 2636, # 3800, 9285) # code 4 → Large (210 litres) (cert 9418) -# Codes 5 / 6 (Inaccessible / Exact) not yet observed. +# code 5 → Inaccessible (context-dependent — see Table 28 below, +# resolved by `_cylinder_inaccessible_volume_l`) +# code 6 → Exact (the lodged measured volume in litres, +# `cylinder_volume_measured_l`; 20 API certs) _CYLINDER_SIZE_CODE_TO_LITRES: Final[dict[int, float]] = { 2: 110.0, 3: 160.0, 4: 210.0 } +# RdSAP 10 §10.5 Table 28 (PDF p.55) — the "Inaccessible" descriptor's +# size-to-use depends on the heating context, and the "Exact" descriptor +# lodges its measured volume separately. +_CYLINDER_SIZE_INACCESSIBLE: Final[int] = 5 +_CYLINDER_SIZE_EXACT: Final[int] = 6 +_CYLINDER_INACCESSIBLE_DUAL_IMMERSION_L: Final[float] = 210.0 +_CYLINDER_INACCESSIBLE_SOLID_FUEL_L: Final[float] = 160.0 +_CYLINDER_INACCESSIBLE_DEFAULT_L: Final[float] = 110.0 + + +def _cylinder_inaccessible_volume_l(epc: EpcPropertyData) -> float: + """RdSAP 10 §10.5 Table 28 (PDF p.55) — size to use for an + "Inaccessible" cylinder (code 5): 210 L for off-peak electric DUAL + immersion, 160 L from a solid-fuel boiler, otherwise 110 L.""" + if _immersion_is_single(epc) is False and _is_off_peak_meter( + epc.sap_energy_source.meter_type, fuel_is_electric=True + ): + return _CYLINDER_INACCESSIBLE_DUAL_IMMERSION_L + main = _first_main_heating(epc) + if main is not None and main.sap_main_heating_code in _TABLE_4A_SOLID_FUEL_BOILER_CODES: + return _CYLINDER_INACCESSIBLE_SOLID_FUEL_L + return _CYLINDER_INACCESSIBLE_DEFAULT_L + + +def _cylinder_volume_l_from_code(epc: EpcPropertyData) -> Optional[float]: + """RdSAP 10 §10.5 Table 28 — resolve the HW cylinder volume (litres) + from the lodged `cylinder_size` descriptor code. Codes 2/3/4 → + 110/160/210; code 5 (Inaccessible) → context-dependent; code 6 (Exact) + → the lodged measured volume. Returns None when no size code is lodged + (or code 6 lodges no measured volume). Does NOT gate on + `has_hot_water_cylinder` — callers apply that guard.""" + size_code = _int_or_none(epc.sap_heating.cylinder_size) + if size_code is None: + return None + if size_code == _CYLINDER_SIZE_EXACT: + measured = _int_or_none(epc.sap_heating.cylinder_volume_measured_l) + return float(measured) if measured is not None else None + if size_code == _CYLINDER_SIZE_INACCESSIBLE: + return _cylinder_inaccessible_volume_l(epc) + return _CYLINDER_SIZE_CODE_TO_LITRES.get(size_code) + # RdSAP `immersion_heating_type` lodgement codes. Code 1 = DUAL immersion, # code 2 = SINGLE. Confirmed against RdSAP 10 §10.5 (PDF p.54 — an # immersion is "assumed dual" on a dual/off-peak meter) cross-checked @@ -5484,10 +5528,7 @@ def _heat_pump_cylinder_meets_pcdb_criteria( for the cohort this criterion is always "unknown" → returns False. """ sh = epc.sap_heating - size_code = _int_or_none(sh.cylinder_size) - if size_code is None: - return False - cert_volume_l = _CYLINDER_SIZE_CODE_TO_LITRES.get(size_code) + cert_volume_l = _cylinder_volume_l_from_code(epc) if cert_volume_l is None: return False # Volume criterion. @@ -5995,15 +6036,13 @@ def _orientation_from_summary_string(raw: Optional[str]) -> Optional[Orientation def _hot_water_cylinder_volume_l(epc: EpcPropertyData) -> Optional[float]: """Resolve the HW cylinder volume (litres) from the cert's - `cylinder_size` code via RdSAP 10 §10.5 Table 28. Returns None - when no cylinder is lodged or the size code falls outside the - cohort-observed range (codes 2-4 → Normal / Medium / Large).""" + `cylinder_size` code via RdSAP 10 §10.5 Table 28 — Normal/Medium/Large + (codes 2/3/4), Inaccessible (5, context-dependent) and Exact (6, lodged + measured volume). Returns None when no cylinder is lodged or no size + code resolves.""" if not epc.has_hot_water_cylinder: return None - size_code = _int_or_none(epc.sap_heating.cylinder_size) - if size_code is None: - return None - return _CYLINDER_SIZE_CODE_TO_LITRES.get(size_code) + return _cylinder_volume_l_from_code(epc) def _immersion_is_single(epc: EpcPropertyData) -> Optional[bool]: @@ -6286,10 +6325,7 @@ def _cylinder_storage_loss_override( if not epc.has_hot_water_cylinder: return None sh = epc.sap_heating - size_code = _int_or_none(sh.cylinder_size) - if size_code is None: - return None - volume_l = _CYLINDER_SIZE_CODE_TO_LITRES.get(size_code) + volume_l = _cylinder_volume_l_from_code(epc) if volume_l is None: return None insulation_label = _cylinder_storage_loss_insulation_label( diff --git a/domain/sap10_ml/tests/_fixtures.py b/domain/sap10_ml/tests/_fixtures.py index 040466b5..2c7c3b68 100644 --- a/domain/sap10_ml/tests/_fixtures.py +++ b/domain/sap10_ml/tests/_fixtures.py @@ -96,6 +96,8 @@ def make_sap_heating( water_heating_code: Optional[int] = 901, water_heating_fuel: Optional[int] = 26, cylinder_size: Optional[Union[int, str]] = None, + cylinder_volume_measured_l: Optional[int] = None, + immersion_heating_type: Optional[Union[int, str]] = None, cylinder_insulation_type: Optional[int] = None, cylinder_insulation_thickness_mm: Optional[int] = None, cylinder_thermostat: Optional[str] = None, @@ -115,6 +117,8 @@ def make_sap_heating( water_heating_code=water_heating_code, water_heating_fuel=water_heating_fuel, cylinder_size=cylinder_size, + cylinder_volume_measured_l=cylinder_volume_measured_l, + immersion_heating_type=immersion_heating_type, cylinder_insulation_type=cylinder_insulation_type, cylinder_insulation_thickness_mm=cylinder_insulation_thickness_mm, cylinder_thermostat=cylinder_thermostat, diff --git a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py index 8a055253..19288401 100644 --- a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py +++ b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py @@ -421,6 +421,105 @@ def test_cylinder_thermostat_assumed_present_per_sap_9_4_9() -> None: assert _cylinder_thermostat_present(epc_gas, gas_boiler) is False +def _cylinder_epc( + *, cylinder_size: int, cylinder_volume_measured_l: Optional[int] = None, + immersion_heating_type: Optional[int] = None, main: Optional[MainHeatingDetail] = None, +) -> EpcPropertyData: + """A minimal cylinder-bearing epc for Table 28 size-code resolution.""" + return make_minimal_sap10_epc( + total_floor_area_m2=_TYPICAL_TFA_M2, habitable_rooms_count=4, + country_code="ENG", has_hot_water_cylinder=True, + sap_building_parts=[make_building_part(construction_age_band="D")], + sap_heating=make_sap_heating( + main_heating_details=[main] if main is not None else None, + cylinder_size=cylinder_size, + cylinder_volume_measured_l=cylinder_volume_measured_l, + immersion_heating_type=immersion_heating_type, + ), + ) + + +def test_cylinder_size_exact_code_6_uses_lodged_measured_volume() -> None: + # Arrange — RdSAP 10 §10.5 Table 28 (PDF p.55): the "Exact" cylinder-size + # descriptor (gov-API code 6) lodges the measured volume in litres via + # `cylinder_size_measured`; SAP uses the actual size when present. The + # map previously held only codes 2/3/4 → code 6 fell through to None, + # skipping the Table-13 high-rate fraction (20 certs in the API sample). + from domain.sap10_calculator.rdsap.cert_to_inputs import ( + _hot_water_cylinder_volume_l, # pyright: ignore[reportPrivateUsage] + ) + epc = _cylinder_epc(cylinder_size=6, cylinder_volume_measured_l=150) + + # Act + volume_l = _hot_water_cylinder_volume_l(epc) + + # Assert — the lodged 150 L is used verbatim, not a descriptor default. + assert volume_l is not None and abs(volume_l - 150.0) <= 1e-9 + + +def test_cylinder_size_inaccessible_code_5_solid_fuel_boiler_uses_160l() -> None: + # Arrange — RdSAP 10 §10.5 Table 28: an "Inaccessible" cylinder (code 5) + # heated "from a solid fuel boiler" uses 160 litres. + from domain.sap10_calculator.rdsap.cert_to_inputs import ( + _hot_water_cylinder_volume_l, # pyright: ignore[reportPrivateUsage] + ) + solid_fuel = MainHeatingDetail( + has_fghrs=False, main_fuel_type=5, heat_emitter_type=1, + emitter_temperature=1, main_heating_control=2106, + main_heating_category=1, sap_main_heating_code=153, # solid-fuel boiler + ) + epc = _cylinder_epc(cylinder_size=5, main=solid_fuel) + + # Act + volume_l = _hot_water_cylinder_volume_l(epc) + + # Assert + assert volume_l is not None and abs(volume_l - 160.0) <= 1e-9 + + +def test_cylinder_size_inaccessible_code_5_otherwise_uses_110l() -> None: + # Arrange — RdSAP 10 §10.5 Table 28: an "Inaccessible" cylinder (code 5) + # that is neither off-peak dual immersion nor solid-fuel uses 110 litres. + from domain.sap10_calculator.rdsap.cert_to_inputs import ( + _hot_water_cylinder_volume_l, # pyright: ignore[reportPrivateUsage] + ) + gas_boiler = MainHeatingDetail( + has_fghrs=False, main_fuel_type=26, heat_emitter_type=1, + emitter_temperature=1, main_heating_control=2106, + main_heating_category=2, sap_main_heating_code=102, + ) + epc = _cylinder_epc(cylinder_size=5, main=gas_boiler) + + # Act + volume_l = _hot_water_cylinder_volume_l(epc) + + # Assert + assert volume_l is not None and abs(volume_l - 110.0) <= 1e-9 + + +def test_cylinder_size_inaccessible_code_5_off_peak_dual_immersion_uses_210l() -> None: + # Arrange — RdSAP 10 §10.5 Table 28: an "Inaccessible" cylinder (code 5) + # heated by an off-peak electric DUAL immersion (immersion_heating_type=1) + # uses 210 litres. + from dataclasses import replace + + from domain.sap10_calculator.rdsap.cert_to_inputs import ( + _hot_water_cylinder_volume_l, # pyright: ignore[reportPrivateUsage] + ) + base = _cylinder_epc(cylinder_size=5, immersion_heating_type=1) # 1 = dual + # Off-peak (dual / Economy-7) meter, not the fixture's "Single" default. + epc = replace( + base, + sap_energy_source=replace(base.sap_energy_source, meter_type="1"), + ) + + # Act + volume_l = _hot_water_cylinder_volume_l(epc) + + # Assert + assert volume_l is not None and abs(volume_l - 210.0) <= 1e-9 + + def test_heat_network_distribution_electricity_per_sap_10_2_appendix_c_3_2() -> None: # Arrange — heat-network main (Table 4a code 301 = community heating, # category 6). SAP 10.2 Appendix C §C3.2 (PDF p.51): distribution From aab75cf902231f6c3c1123a3e135f6ffdb5c4d32 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Thu, 11 Jun 2026 10:55:19 +0000 Subject: [PATCH 54/87] fix(walls): reconcile gov-API wall_construction enum with the calc code-space MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The gov-EPC API `wall_construction` enum diverges from the calculator's internal WALL_* code-space (confirmed by the description-vs-code audit across the corpus): API 1-5 align (granite/sandstone/solid-brick/cavity/timber), but API 6=basement, 8=system built, 9=cob — whereas the calc constants are WALL_SYSTEM_BUILT=6, WALL_COB=7, WALL_PARK_HOME=8, WALL_CURTAIN=9. Codes 8 and 9 therefore fell OUT of u_wall's `known_types` and resolved only via the `walls[].description` fallback, with two failure modes: - System built (API 8): a cert lodging no description silently defaulted to cavity (1.5) instead of the system-built U (RdSAP 10 Table 6, e.g. band E as-built 1.7). Latent in the corpus (all 43 carry a description) but a silent mis-bill waiting to happen. - Cob (API 9): a LIVE bug — calc WALL_CURTAIN=9 (set by the Summary path's "CW" mapping, paired with a curtain_wall_age) intercepts code 9 in the `construction == WALL_CURTAIN` branch, billing the cob wall at the curtain default 2.0 regardless of description. Fix, split by where each can be disambiguated safely: - System built: `u_wall` gains `_GOV_API_WALL_CODE_TO_TYPE = {8: WALL_ SYSTEM_BUILT}`, resolving code 8 directly (calc WALL_PARK_HOME=8 is never dispatched, so no collision; gov 6=basement is left to the basement machinery — cannot remap 8→6). - Cob: translated at the API mapper (`_api_wall_construction_code`, 9 → WALL_COB=7) where the source is unambiguously the gov enum — the gov API has no curtain code, so an API 9 is always cob. Applied to main + alt walls across the from_rdsap_schema_* builders. The Summary path's "CW"→9 curtain mapping is untouched. Worksheet harness UNAFFECTED (47/47, 0 divergers — Summary path unchanged). API gauge 65.1% -> 65.3% within-0.5 (mean|err| 1.075 -> 1.059): the n=1 cob cert now computes cob instead of curtain. 3 AAA tests (u_wall system-built without description; mapper cob 9->7; aligned/system/basement pass-through). pyright net-zero. Co-Authored-By: Claude Opus 4.8 --- datatypes/epc/domain/mapper.py | 51 ++++++++++++++----- .../domain/tests/test_from_rdsap_schema.py | 30 +++++++++++ domain/sap10_ml/rdsap_uvalues.py | 28 ++++++++++ domain/sap10_ml/tests/test_rdsap_uvalues.py | 23 +++++++++ 4 files changed, 120 insertions(+), 12 deletions(-) diff --git a/datatypes/epc/domain/mapper.py b/datatypes/epc/domain/mapper.py index 1512ea44..f87c04d5 100644 --- a/datatypes/epc/domain/mapper.py +++ b/datatypes/epc/domain/mapper.py @@ -2,7 +2,7 @@ import re from dataclasses import replace from datetime import date from decimal import ROUND_HALF_UP, Decimal -from typing import Any, Dict, Final, List, Optional, Sequence, Union, cast +from typing import Any, Dict, Final, List, Optional, Sequence, TypeVar, Union, cast from datatypes.epc.schema.helpers import from_dict from datatypes.epc.domain.epc_property_data import ( @@ -539,7 +539,7 @@ class EpcPropertyDataMapper: SapBuildingPart( identifier=BuildingPartIdentifier.from_api_string(bp.identifier), construction_age_band=bp.construction_age_band, - wall_construction=bp.wall_construction, + wall_construction=_api_wall_construction_code(bp.wall_construction), wall_insulation_type=bp.wall_insulation_type, wall_thickness_measured=bp.wall_thickness_measured == "Y", party_wall_construction=_api_party_wall_construction_int( @@ -691,7 +691,7 @@ class EpcPropertyDataMapper: SapBuildingPart( identifier=BuildingPartIdentifier.from_api_string(bp.identifier), construction_age_band=bp.construction_age_band, - wall_construction=bp.wall_construction, + wall_construction=_api_wall_construction_code(bp.wall_construction), wall_insulation_type=bp.wall_insulation_type, wall_thickness_measured=bp.wall_thickness_measured == "Y", party_wall_construction=_api_party_wall_construction_int( @@ -830,7 +830,7 @@ class EpcPropertyDataMapper: SapBuildingPart( identifier=BuildingPartIdentifier.from_api_string(bp.identifier), construction_age_band=bp.construction_age_band, - wall_construction=bp.wall_construction, + wall_construction=_api_wall_construction_code(bp.wall_construction), wall_insulation_type=bp.wall_insulation_type, wall_thickness_measured=bp.wall_thickness_measured == "Y", party_wall_construction=_api_party_wall_construction_int( @@ -995,7 +995,7 @@ class EpcPropertyDataMapper: SapBuildingPart( identifier=BuildingPartIdentifier.from_api_string(bp.identifier), construction_age_band=bp.construction_age_band, - wall_construction=bp.wall_construction, + wall_construction=_api_wall_construction_code(bp.wall_construction), wall_insulation_type=bp.wall_insulation_type, wall_thickness_measured=bp.wall_thickness_measured == "Y", party_wall_construction=_api_party_wall_construction_int( @@ -1177,7 +1177,7 @@ class EpcPropertyDataMapper: SapBuildingPart( identifier=BuildingPartIdentifier.from_api_string(bp.identifier), construction_age_band=bp.construction_age_band, - wall_construction=bp.wall_construction, + wall_construction=_api_wall_construction_code(bp.wall_construction), wall_insulation_type=bp.wall_insulation_type, wall_thickness_measured=bp.wall_thickness_measured == "Y", party_wall_construction=_api_party_wall_construction_int( @@ -1402,7 +1402,7 @@ class EpcPropertyDataMapper: SapBuildingPart( identifier=BuildingPartIdentifier.from_api_string(bp.identifier), construction_age_band=bp.construction_age_band, - wall_construction=bp.wall_construction, + wall_construction=_api_wall_construction_code(bp.wall_construction), wall_insulation_type=bp.wall_insulation_type, wall_thickness_measured=bp.wall_thickness_measured == "Y", party_wall_construction=_api_party_wall_construction_int( @@ -1467,7 +1467,7 @@ class EpcPropertyDataMapper: SapAlternativeWall( wall_area=bp.sap_alternative_wall_1.wall_area, wall_dry_lined=bp.sap_alternative_wall_1.wall_dry_lined, - wall_construction=bp.sap_alternative_wall_1.wall_construction, + wall_construction=_api_wall_construction_code(bp.sap_alternative_wall_1.wall_construction), wall_insulation_type=bp.sap_alternative_wall_1.wall_insulation_type, wall_thickness_measured=bp.sap_alternative_wall_1.wall_thickness_measured, wall_insulation_thickness=bp.sap_alternative_wall_1.wall_insulation_thickness, @@ -1480,7 +1480,7 @@ class EpcPropertyDataMapper: SapAlternativeWall( wall_area=bp.sap_alternative_wall_2.wall_area, wall_dry_lined=bp.sap_alternative_wall_2.wall_dry_lined, - wall_construction=bp.sap_alternative_wall_2.wall_construction, + wall_construction=_api_wall_construction_code(bp.sap_alternative_wall_2.wall_construction), wall_insulation_type=bp.sap_alternative_wall_2.wall_insulation_type, wall_thickness_measured=bp.sap_alternative_wall_2.wall_thickness_measured, wall_insulation_thickness=bp.sap_alternative_wall_2.wall_insulation_thickness, @@ -1693,7 +1693,7 @@ class EpcPropertyDataMapper: SapBuildingPart( identifier=BuildingPartIdentifier.from_api_string(bp.identifier), construction_age_band=bp.construction_age_band, - wall_construction=bp.wall_construction, + wall_construction=_api_wall_construction_code(bp.wall_construction), wall_insulation_type=bp.wall_insulation_type, wall_thickness_measured=bp.wall_thickness_measured == "Y", party_wall_construction=_api_party_wall_construction_int( @@ -1758,7 +1758,7 @@ class EpcPropertyDataMapper: SapAlternativeWall( wall_area=bp.sap_alternative_wall_1.wall_area, wall_dry_lined=bp.sap_alternative_wall_1.wall_dry_lined, - wall_construction=bp.sap_alternative_wall_1.wall_construction, + wall_construction=_api_wall_construction_code(bp.sap_alternative_wall_1.wall_construction), wall_insulation_type=bp.sap_alternative_wall_1.wall_insulation_type, wall_thickness_measured=bp.sap_alternative_wall_1.wall_thickness_measured, wall_insulation_thickness=bp.sap_alternative_wall_1.wall_insulation_thickness, @@ -1771,7 +1771,7 @@ class EpcPropertyDataMapper: SapAlternativeWall( wall_area=bp.sap_alternative_wall_2.wall_area, wall_dry_lined=bp.sap_alternative_wall_2.wall_dry_lined, - wall_construction=bp.sap_alternative_wall_2.wall_construction, + wall_construction=_api_wall_construction_code(bp.sap_alternative_wall_2.wall_construction), wall_insulation_type=bp.sap_alternative_wall_2.wall_insulation_type, wall_thickness_measured=bp.sap_alternative_wall_2.wall_thickness_measured, wall_insulation_thickness=bp.sap_alternative_wall_2.wall_insulation_thickness, @@ -2294,6 +2294,33 @@ _ELMHURST_WALL_CODE_TO_SAP10: Dict[str, int] = { } +_WallConstructionCode = TypeVar("_WallConstructionCode") + + +def _api_wall_construction_code( + code: _WallConstructionCode, +) -> _WallConstructionCode: + """Translate the gov-EPC API `wall_construction` enum to the + calculator's WALL_* code-space where the two DIVERGE. + + The gov enum (confirmed by the description-vs-code audit) is + 1=granite, 2=sandstone, 3=solid brick, 4=cavity, 5=timber frame (all + ALIGN with the WALL_* constants), 6=basement, 8=system built, 9=cob. + Only code 9 needs remapping HERE: it collides with the calculator's + `WALL_CURTAIN`=9 (which the Summary path's "CW" mapping legitimately + uses, paired with a `curtain_wall_age`). The gov API has no curtain + code, so an API `wall_construction` of 9 is unambiguously cob → remap to + `WALL_COB`=7 at this boundary, before it can hit the curtain dispatch in + `u_wall`. Gov 8 (system built) is left as-is — `u_wall` resolves it and + calc `WALL_PARK_HOME`=8 is never dispatched; gov 6 (basement) is left to + the basement machinery. Non-int values pass through unchanged; the input + type is preserved so each call site's typed `wall_construction` field + stays satisfied.""" + if code == 9: + return cast(_WallConstructionCode, 7) + return code + + # Elmhurst wall-insulation-type codes mapped to the SAP10 integer enum # documented at domain.sap10_ml.rdsap_uvalues.WALL_INSULATION_FILLED_CAVITY. # Two-letter dual codes ("FE", "FI") encode a cavity-wall that received a diff --git a/datatypes/epc/domain/tests/test_from_rdsap_schema.py b/datatypes/epc/domain/tests/test_from_rdsap_schema.py index b882a4c0..793c64db 100644 --- a/datatypes/epc/domain/tests/test_from_rdsap_schema.py +++ b/datatypes/epc/domain/tests/test_from_rdsap_schema.py @@ -704,6 +704,36 @@ class TestFromRdSapSchema21_0_1: # --------------------------------------------------------------------------- +class TestApiWallConstructionCode: + """The gov-EPC API `wall_construction` enum diverges from the + calculator's WALL_* code-space at code 9: gov 9 = "Cob" but calc + `WALL_CURTAIN` = 9 (set by the Summary path's "CW" mapping). The gov + API has no curtain code, so an API code 9 must be remapped to + `WALL_COB` = 7 before it reaches `u_wall`'s curtain dispatch (which + would otherwise bill the wall at the curtain default 2.0 W/m²K).""" + + def test_gov_api_cob_code_9_remaps_to_wall_cob_7(self) -> None: + # Arrange + from datatypes.epc.domain.mapper import _api_wall_construction_code # pyright: ignore[reportPrivateUsage] + + # Act + result = _api_wall_construction_code(9) + + # Assert — 9 (gov cob) → 7 (WALL_COB), dodging WALL_CURTAIN=9. + assert result == 7 + + def test_aligned_and_system_built_codes_pass_through_unchanged(self) -> None: + # Arrange — codes 1-5 already align; gov 8 (system built) is left + # for u_wall to resolve (calc WALL_PARK_HOME=8 is never dispatched); + # gov 6 (basement) is left to the basement machinery. + from datatypes.epc.domain.mapper import _api_wall_construction_code # pyright: ignore[reportPrivateUsage] + + # Act / Assert + for code in (1, 2, 3, 4, 5, 6, 8): + assert _api_wall_construction_code(code) == code + assert _api_wall_construction_code(None) is None + + class TestApiResolveWallInsulationThickness: """`wall_insulation_thickness == "measured"` resolves to the separate `wall_insulation_thickness_measured` field (previously dropped by diff --git a/domain/sap10_ml/rdsap_uvalues.py b/domain/sap10_ml/rdsap_uvalues.py index 1f37b222..ea1188db 100644 --- a/domain/sap10_ml/rdsap_uvalues.py +++ b/domain/sap10_ml/rdsap_uvalues.py @@ -459,6 +459,29 @@ _DEFAULT_WALL_BY_AGE: Final[dict[str, int]] = { "L": WALL_CAVITY, "M": WALL_CAVITY, } +# Gov-EPC API `wall_construction` codes that DIVERGE from the calculator's +# internal WALL_* code-space. The gov enum (confirmed by the description-vs- +# code audit across the corpus) is 1=granite, 2=sandstone, 3=solid brick, +# 4=cavity, 5=timber frame (all ALIGN with the WALL_* constants), but +# 6=basement, 8=system built, 9=cob — whereas the calc constants are +# WALL_SYSTEM_BUILT=6, WALL_COB=7, WALL_PARK_HOME=8, WALL_CURTAIN=9. +# +# Code 8 (system built) falls OUT of `known_types` and previously resolved +# only via the `walls[].description` fallback; a cert lodging no description +# silently defaulted to cavity. Translate it here so the int code alone +# resolves. `WALL_PARK_HOME` (calc 8) is never dispatched, so reusing API +# code 8 for system built is collision-free at this layer. +# +# Code 9 (cob) is NOT handled here: calc `WALL_CURTAIN`=9 (set by the +# Summary path's "CW" mapping, with a `curtain_wall_age`) intercepts it in +# the `construction == WALL_CURTAIN` branch above. The gov-API cob code is +# therefore translated to `WALL_COB` upstream in the API mapper (where the +# source is unambiguously the gov enum), so it never reaches this collision. +# Code 6 (basement) is left to the mapper's basement machinery. +_GOV_API_WALL_CODE_TO_TYPE: Final[dict[int, int]] = { + 8: WALL_SYSTEM_BUILT, # gov API "System built" +} + # Surveyor-text -> wall-construction code, evaluated in priority order so that # "sandstone" beats "stone", "solid brick" beats "brick", etc. Used only as a @@ -585,6 +608,11 @@ def u_wall( } if construction in known_types: wall_type = construction + elif construction in _GOV_API_WALL_CODE_TO_TYPE: + # A gov-API construction code (system built / cob) that diverges from + # the WALL_* code-space — resolve it directly, independent of the + # surveyor description (RdSAP 10 §5.7 Table 6 rows still apply). + wall_type = _GOV_API_WALL_CODE_TO_TYPE[construction] else: wall_type = _wall_type_from_description(description) or _DEFAULT_WALL_BY_AGE.get(band, WALL_CAVITY) # RdSAP 10 §5.7 Table 13 + §5.8 (PDF p.41-42) — uninsulated solid diff --git a/domain/sap10_ml/tests/test_rdsap_uvalues.py b/domain/sap10_ml/tests/test_rdsap_uvalues.py index 5783af45..1018ce30 100644 --- a/domain/sap10_ml/tests/test_rdsap_uvalues.py +++ b/domain/sap10_ml/tests/test_rdsap_uvalues.py @@ -451,6 +451,29 @@ def test_u_wall_falls_back_to_age_band_default_when_construction_unknown() -> No assert result == pytest.approx(0.60, abs=0.001) +def test_u_wall_gov_api_system_built_code_8_resolves_without_description() -> None: + # Arrange — the gov-EPC API `wall_construction` enum diverges from the + # calculator's internal WALL_* code-space: API 8 = "System built" (calc + # WALL_SYSTEM_BUILT = 6; calc 8 = park home). The 43-cert system-built + # cohort currently resolves only via the `walls[].description` fallback; + # with no description, code 8 silently defaulted to cavity (1.5) instead + # of the system-built U (band E as-built = 1.7). + + # Act — code 8, NO description. + result = u_wall( + country=Country.ENG, age_band="E", construction=8, + insulation_thickness_mm=0, description=None, + ) + reference = u_wall( + country=Country.ENG, age_band="E", construction=WALL_SYSTEM_BUILT, + insulation_thickness_mm=0, + ) + + # Assert — code 8 is system-built (1.7), not the cavity default (1.5). + assert abs(result - reference) <= 1e-9 + assert abs(result - 1.7) <= 1e-9 + + def test_u_wall_falls_back_to_mid_range_default_when_everything_unknown() -> None: # Arrange — no signal at all. From 42e0bb31223b22e8cf8272bc194b18144d2e286d Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Thu, 11 Jun 2026 12:12:25 +0000 Subject: [PATCH 55/87] fix(thermal-mass): gov-API system built (wall code 8) is masonry, not park home MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The §5.16 Table 22 thermal-mass-parameter (TMP) "always low-mass" set was {timber 5, cob 7, park home 8}. But wall_construction code 8 is OVERLOADED by the same gov-API/calc code-space divergence as the wall-U fix: the Summary path's "PH" mapping uses 8 for park home, while the gov-EPC API enum uses 8 for SYSTEM BUILD (Summary system build = code 6). So every API system-built cert was mis-rated as low-mass 100 kJ/m²K instead of masonry 250 (Table 22 lists system build as masonry — PDF p.48, line "System build 250..."). A too-low TMP shortens the §7 time constant tau = Cm/(3.6·H), over-cutting the temperature reduction so mean internal temperature is UNDER-stated → space-heating demand under-stated → SAP over-rated. This was the cause of the uninsulated system-built over-rate cluster (n=9 gas-boiler certs at signed +2.39 vs cavity +0.43 / solid-brick +0.08 at the same bands — a system-built- specific anomaly with a spec-correct wall U). Fix: drop 8 from the always-low set and gate it on `property_type` — code 8 is the low-mass park-home value only when the dwelling really is a park home, otherwise it is gov-API system build and keeps masonry 250. Disambiguated by the same `property_type == "park home"` signal used elsewhere in the cascade. Worksheet harness UNAFFECTED (47/47, 0 divergers): the Summary path uses code 6 for system build and code 8 only for genuine park homes (which stay low-mass via the property_type gate). API gauge 65.3% -> 67.1% within-0.5 (mean|err| 1.059 -> 1.024, signed +0.050 -> -0.002). The uninsulated system-built cluster collapses +2.82 -> +0.28 signed (0/11 -> 7/11 within 0.5). 2 AAA tests (parametrised code-8 system-built -> 250; park-home property -> 100). pyright net-zero. Co-Authored-By: Claude Opus 4.8 --- .../sap10_calculator/rdsap/cert_to_inputs.py | 20 +++++++-- .../rdsap/test_cert_to_inputs.py | 41 +++++++++++++++++-- 2 files changed, 55 insertions(+), 6 deletions(-) diff --git a/domain/sap10_calculator/rdsap/cert_to_inputs.py b/domain/sap10_calculator/rdsap/cert_to_inputs.py index 9a90c0fe..fad11fae 100644 --- a/domain/sap10_calculator/rdsap/cert_to_inputs.py +++ b/domain/sap10_calculator/rdsap/cert_to_inputs.py @@ -234,8 +234,15 @@ _PENCE_TO_GBP: Final[float] = 0.01 _DEFAULT_THERMAL_MASS_PARAMETER_KJ_PER_M2_K: Final[float] = 250.0 _TMP_LOW_KJ_PER_M2_K: Final[float] = 100.0 # `wall_construction` int codes (domain/sap10_ml/rdsap_uvalues.py): -# 5 = timber frame, 7 = cob, 8 = park home — Table 22's "three types". -_TMP_ALWAYS_LOW_WALL_CONSTRUCTION_CODES: Final[frozenset[int]] = frozenset({5, 7, 8}) +# 5 = timber frame, 7 = cob — always Table 22 low-mass. Park home is the +# third "always low-mass" type, but its wall code (8) is OVERLOADED: the +# Summary path's "PH" mapping uses 8 for park home, whereas the gov-API +# enum uses 8 for SYSTEM BUILT (a masonry type, Summary system build = code +# 6). Code 8 therefore takes the low-mass value only when the dwelling is +# actually a park home (`property_type`); otherwise it is system built and +# keeps the masonry 250 — see `_thermal_mass_parameter_kj_per_m2_k`. +_TMP_ALWAYS_LOW_WALL_CONSTRUCTION_CODES: Final[frozenset[int]] = frozenset({5, 7}) +_TMP_PARK_HOME_OR_SYSTEM_BUILT_WALL_CODE: Final[int] = 8 # `wall_insulation_type` int codes that are INTERNAL insulation # (Table 22 "masonry … with internal insulation"): 3 = internal wall # insulation, 7 = filled cavity + internal. External (1), filled cavity @@ -3871,7 +3878,14 @@ def _thermal_mass_parameter_kj_per_m2_k(epc: EpcPropertyData) -> float: if not parts: return _DEFAULT_THERMAL_MASS_PARAMETER_KJ_PER_M2_K main: SapBuildingPart = parts[0] - if _int_or_none(main.wall_construction) in _TMP_ALWAYS_LOW_WALL_CONSTRUCTION_CODES: + wall_code: Optional[int] = _int_or_none(main.wall_construction) + if wall_code in _TMP_ALWAYS_LOW_WALL_CONSTRUCTION_CODES: + return _TMP_LOW_KJ_PER_M2_K + # Wall code 8 is a park home (low-mass) ONLY when the dwelling really is + # one; on the gov-API path code 8 is system built (masonry). Disambiguate + # by property_type so an API system build is not mis-rated as low-mass. + is_park_home: bool = (epc.property_type or "").strip().lower() == "park home" + if wall_code == _TMP_PARK_HOME_OR_SYSTEM_BUILT_WALL_CODE and is_park_home: return _TMP_LOW_KJ_PER_M2_K if _int_or_none(main.wall_insulation_type) in _TMP_INTERNAL_WALL_INSULATION_CODES: return _TMP_LOW_KJ_PER_M2_K diff --git a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py index 19288401..c088628d 100644 --- a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py +++ b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py @@ -148,12 +148,17 @@ def _typical_semi_detached_epc(): @pytest.mark.parametrize( "wall_construction, wall_insulation_type, expected_tmp", [ - # RdSAP 10 §5.16 Table 22 (PDF p.48) — timber frame (5), cob (7), - # park home (8) are always low-mass, regardless of insulation. + # RdSAP 10 §5.16 Table 22 (PDF p.48) — timber frame (5), cob (7) + # are always low-mass, regardless of insulation. (5, 4, 100.0), # timber frame, as-built (5, 2, 100.0), # timber frame, filled cavity — still 100 (7, 4, 100.0), # cob - (8, 1, 100.0), # park home, external insulation — still 100 + # Wall code 8 with no park-home property is gov-API SYSTEM BUILT + # (calc 8 = park home only on the Summary path) → masonry. Table 22 + # lists system build as masonry (250 as-built); the park-home + # low-mass case is covered separately below (needs property_type). + (8, 1, 250.0), # system built (gov-API code 8), external insulation + (8, 4, 250.0), # system built (gov-API code 8), as-built # Masonry WITH internal insulation (ins 3 = internal, 7 = # filled cavity + internal) → low-mass 100. (3, 3, 100.0), # solid brick + internal @@ -203,6 +208,36 @@ def test_thermal_mass_parameter_follows_rdsap_table_22( assert abs(tmp - expected_tmp) <= 1e-9 +def test_thermal_mass_wall_code_8_is_park_home_only_when_property_type_says_so() -> None: + # Arrange — RdSAP 10 §5.16 Table 22 (PDF p.48): timber frame / cob / + # PARK HOME are low-mass (100); system build is masonry (250 as-built). + # Wall_construction code 8 is overloaded: the Summary path's "PH" + # mapping uses 8 for park home, but the gov-API enum uses 8 for SYSTEM + # BUILT (Summary system build is code 6). So code 8 may only take the + # park-home low-mass value when the dwelling really is a park home — + # otherwise it is gov-API system built (masonry). Same construction + # code, opposite thermal mass, disambiguated by `property_type`. + def _epc(property_type: Optional[str]) -> EpcPropertyData: + return make_minimal_sap10_epc( + total_floor_area_m2=_TYPICAL_TFA_M2, habitable_rooms_count=4, + country_code="ENG", property_type=property_type, + sap_building_parts=[ + make_building_part(wall_construction=8, wall_insulation_type=4), + ], + sap_heating=make_sap_heating( + main_heating_details=[_gas_boiler_detail(sap_main_heating_code=102)], + ), + ) + + # Act + tmp_park_home: float = _thermal_mass_parameter_kj_per_m2_k(_epc("Park home")) + tmp_system_built: float = _thermal_mass_parameter_kj_per_m2_k(_epc("House")) + + # Assert — park home → low-mass 100; system built (code 8) → masonry 250. + assert abs(tmp_park_home - 100.0) <= 1e-9 + assert abs(tmp_system_built - 250.0) <= 1e-9 + + def test_heat_network_main_applies_table12c_dlf_to_main_heating_efficiency() -> None: # Arrange — heat-network main heating (Table 4a code 301 = community # heating with CHP/boilers; main_heating_category=6). Cert age band From 6884ec9fdadb2a2ff39dc9f46292409bfa4877c3 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Thu, 11 Jun 2026 13:51:26 +0000 Subject: [PATCH 56/87] =?UTF-8?q?fix(fabric):=20honour=20the=20gov-EPC=20l?= =?UTF-8?q?odged=20per-element=20U-values=20(RdSAP=20=C2=A75.1)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The gov-EPC API surfaces the assessor's RdSAP-assessed per-element U-values as `roof_u_value` / `wall_u_value` / `floor_u_value` on each building part. These were undeclared on the RdSAP 21.0.0/21.0.1 schemas, so `from_dict` silently dropped them, and `heat_transmission` re-derived each U from the §5.6 /§5.7/§5.11 construction-default cascade. The gov OPEN data routinely redacts the backing insulation thickness, so that re-derivation mis-bills an insulated element as uninsulated. RdSAP 10 §5.1: a known element U-value (documentary evidence / the lodged RdSAP output) is used directly in place of the construction-default cascade. Per [[project_per_cert_mapper_validation_state]] the gov API carries RdSAP OUTPUT, so the lodged U reproduces the official's element heat loss exactly. Worst case in the 2026 sample: cert 7921-0052-0940-5007-0663, an age-C "Pitched, sloping ceiling" (rc=8) top-floor flat lodging roof_u_value=0.2 with no thickness. The cascade returned the uninsulated 2.30 W/m²K → SAP 56.9 vs lodged 80 (-23.09, the single largest error in the sample). The roof override alone recovers ~15 SAP; the wall override (lodged 0.34 vs cascade) closes the rest of this cohort. Override applies to the MAIN wall only (alt-wall sub-areas keep their own per-area U) and the part's floor=0. Fires only when the rare field is present (9 of 909 computed certs), so the Summary path — which never lodges these API fields — is untouched. API gauge: 67.1% → 67.7% within-0.5, mean|err| 1.024 → 0.992. Worksheet harness: 47/47, 0 divergers (unchanged). Co-Authored-By: Claude Opus 4.8 --- datatypes/epc/domain/epc_property_data.py | 15 +++ datatypes/epc/domain/mapper.py | 12 ++ datatypes/epc/schema/rdsap_schema_21_0_0.py | 10 ++ datatypes/epc/schema/rdsap_schema_21_0_1.py | 11 ++ .../worksheet/heat_transmission.py | 23 ++++ .../worksheet/test_heat_transmission.py | 106 ++++++++++++++++++ 6 files changed, 177 insertions(+) diff --git a/datatypes/epc/domain/epc_property_data.py b/datatypes/epc/domain/epc_property_data.py index d5c11b1c..49c9b119 100644 --- a/datatypes/epc/domain/epc_property_data.py +++ b/datatypes/epc/domain/epc_property_data.py @@ -509,10 +509,19 @@ class SapBuildingPart: # (λ = 0.04 / 0.03 / 0.025 W/m·K). Used by the documentary-evidence # R-value path when a measured wall thickness is lodged alongside it. wall_insulation_thermal_conductivity: Optional[Union[str, int]] = None + # RdSAP 10 §5.1 — the assessor's lodged main-wall U-value (W/m²K), surfaced + # by the gov-EPC API as `wall_u_value`. Authoritative when the open data + # redacts the backing insulation; overrides the §5.6/§5.7 construction- + # default cascade for the main wall (alt-wall sub-areas keep their own U). + wall_u_value: Optional[float] = None sap_alternative_wall_1: Optional[SapAlternativeWall] = None sap_alternative_wall_2: Optional[SapAlternativeWall] = None floor_heat_loss: Optional[int] = None + # RdSAP 10 §5.1 — the assessor's lodged ground-floor U-value (W/m²K), + # surfaced by the gov-EPC API as `floor_u_value`. Overrides the BS EN ISO + # 13370 / Table 19 ground-floor cascade when present. + floor_u_value: Optional[float] = None floor_insulation_thickness: Optional[str] = None flat_roof_insulation_thickness: Optional[Union[str, int]] = ( None # TODO: make enum/mapping? @@ -528,6 +537,12 @@ class SapBuildingPart: roof_construction: Optional[int] = None roof_construction_type: Optional[str] = None # str from site notes e.g. "PS Pitched, sloping ceiling" + # RdSAP 10 §5.1 — the assessor's lodged roof U-value (W/m²K). The gov-EPC + # API surfaces it as `roof_u_value`; it is the RdSAP-assessed output for + # the roof and overrides the §5.11 construction-default cascade in + # `heat_transmission` (the open data can redact the backing insulation + # thickness, so the cascade otherwise mis-derives an uninsulated U). + roof_u_value: Optional[float] = None roof_insulation_location: Optional[Union[int, str]] = ( None # TODO: make enum/mapping? ) diff --git a/datatypes/epc/domain/mapper.py b/datatypes/epc/domain/mapper.py index f87c04d5..6cdc328d 100644 --- a/datatypes/epc/domain/mapper.py +++ b/datatypes/epc/domain/mapper.py @@ -1459,6 +1459,12 @@ class EpcPropertyDataMapper: bp.construction_age_band, bp.sloping_ceiling_insulation_thickness, ), + # RdSAP 10 §5.1 — the assessor's lodged roof/wall/floor U + # overrides the §5.6/§5.7/§5.11 construction-default cascade + # (gov open data can redact the backing insulation). + roof_u_value=bp.roof_u_value, + wall_u_value=bp.wall_u_value, + floor_u_value=bp.floor_u_value, sap_room_in_roof=_api_build_room_in_roof( bp.sap_room_in_roof, is_flat=schema.property_type == 2, @@ -1750,6 +1756,12 @@ class EpcPropertyDataMapper: bp.construction_age_band, bp.sloping_ceiling_insulation_thickness, ), + # RdSAP 10 §5.1 — the assessor's lodged roof/wall/floor U + # overrides the §5.6/§5.7/§5.11 construction-default cascade + # (gov open data can redact the backing insulation). + roof_u_value=bp.roof_u_value, + wall_u_value=bp.wall_u_value, + floor_u_value=bp.floor_u_value, sap_room_in_roof=_api_build_room_in_roof( bp.sap_room_in_roof, is_flat=schema.property_type == 2, diff --git a/datatypes/epc/schema/rdsap_schema_21_0_0.py b/datatypes/epc/schema/rdsap_schema_21_0_0.py index 8235569c..e70d7b52 100644 --- a/datatypes/epc/schema/rdsap_schema_21_0_0.py +++ b/datatypes/epc/schema/rdsap_schema_21_0_0.py @@ -265,6 +265,16 @@ class SapBuildingPart: # the slope as uninsulated (Table 16 / Table 18 fallback). Consumed by # `_api_resolve_sloping_ceiling_thickness` → Table 17 column (1a). sloping_ceiling_insulation_thickness: Optional[Union[str, int]] = None + # Lodged roof U-value (W/m²K) — the assessor's RdSAP-assessed roof U, + # authoritative when the open data redacts the backing insulation + # thickness. Consumed by `heat_transmission` as a §5.1 documentary-evidence + # override. Previously undeclared → dropped by `from_dict`. + roof_u_value: Optional[float] = None + # Lodged main-wall / ground-floor U-values (W/m²K) — same §5.1 documentary- + # evidence override; authoritative when the open data redacts the backing + # insulation. Previously undeclared → dropped by `from_dict`. + wall_u_value: Optional[float] = None + floor_u_value: Optional[float] = None @dataclass diff --git a/datatypes/epc/schema/rdsap_schema_21_0_1.py b/datatypes/epc/schema/rdsap_schema_21_0_1.py index ef30581e..48843b05 100644 --- a/datatypes/epc/schema/rdsap_schema_21_0_1.py +++ b/datatypes/epc/schema/rdsap_schema_21_0_1.py @@ -303,6 +303,17 @@ class SapBuildingPart: # the slope as uninsulated (Table 16 / Table 18 fallback). Consumed by # `_api_resolve_sloping_ceiling_thickness` → Table 17 column (1a). sloping_ceiling_insulation_thickness: Optional[Union[str, int]] = None + # Lodged roof U-value (W/m²K) — the assessor's RdSAP-assessed roof U. The + # gov open data can redact the backing insulation thickness, so this is the + # authoritative per-element value; consumed by `heat_transmission` as a + # §5.1 documentary-evidence override. Previously undeclared → dropped by + # `from_dict` (cert 7921-0052-0940-5007-0663 lodges roof_u_value=0.2). + roof_u_value: Optional[float] = None + # Lodged main-wall / ground-floor U-values (W/m²K) — same §5.1 documentary- + # evidence override as roof_u_value; authoritative when the open data + # redacts the backing insulation. Previously undeclared → dropped. + wall_u_value: Optional[float] = None + floor_u_value: Optional[float] = None @dataclass diff --git a/domain/sap10_calculator/worksheet/heat_transmission.py b/domain/sap10_calculator/worksheet/heat_transmission.py index 5af20455..ebdb2b52 100644 --- a/domain/sap10_calculator/worksheet/heat_transmission.py +++ b/domain/sap10_calculator/worksheet/heat_transmission.py @@ -819,6 +819,14 @@ def heat_transmission_from_cert( # billed at the un-adjusted U. dry_lined=bool(part.wall_dry_lined), ) + # RdSAP 10 §5.1 — a lodged/known main-wall U-value (the assessor's + # RdSAP output, surfaced by the gov-EPC API as `wall_u_value`) + # overrides the §5.6/§5.7 construction-default cascade for the MAIN + # wall. Alt-wall sub-areas keep their own per-area U (handled below), + # so this only replaces the primary `uw`. + lodged_wall_u = getattr(part, "wall_u_value", None) + if lodged_wall_u is not None: + uw = lodged_wall_u # When the per-bp `roof_insulation_thickness` is explicitly lodged # as 0 (uninsulated — e.g. cert 001479 Ext2 PS sloping ceiling # age C from Slice 91's `_api_resolve_sloping_ceiling_thickness`, @@ -866,6 +874,15 @@ def heat_transmission_from_cert( # string triggers the col (3) age-band default in `u_roof`. is_pitched_sloping_ceiling = "sloping ceiling" in roof_type_lower ur = u_roof(country=country, age_band=age_band, insulation_thickness_mm=roof_thickness, description=effective_roof_description, is_flat_roof=is_flat_roof, is_sloping_ceiling=is_sloping_ceiling, is_pitched_sloping_ceiling=is_pitched_sloping_ceiling) + # RdSAP 10 §5.1 — a lodged/known roof U-value (the assessor's RdSAP + # output, surfaced by the gov-EPC API as `roof_u_value`) is used + # directly in place of the §5.11 construction-default cascade. The gov + # open data can redact the backing insulation thickness, so the + # cascade otherwise mis-derives an uninsulated U for an insulated roof + # (cert 7921-0052-0940-5007-0663: lodged 0.2 vs cascade 2.30 → -23 SAP). + lodged_roof_u = getattr(part, "roof_u_value", None) + if lodged_roof_u is not None: + ur = lodged_roof_u # Floor U-value routing (in priority order): # 1. Basement floor — Table 23 F-column override (whole floor=0). # 2. Exposed/semi-exposed upper floor — Table 20 lookup; no @@ -910,6 +927,12 @@ def heat_transmission_from_cert( wall_thickness_mm=part.wall_thickness_mm, description=effective_floor_description, ) + # RdSAP 10 §5.1 — a lodged/known ground-floor U-value (surfaced by the + # gov-EPC API as `floor_u_value`) overrides the BS EN ISO 13370 / + # Table 19 cascade for this part's floor=0. + lodged_floor_u = getattr(part, "floor_u_value", None) + if lodged_floor_u is not None: + uf = lodged_floor_u # RdSAP 10 Table 15 footnote * — flats/maisonettes with unknown # party-wall construction default to U=0.0 (both sides heated), # not the U=0.25 house default. Cert 0036-6325-1100-0063-1226 diff --git a/tests/domain/sap10_calculator/worksheet/test_heat_transmission.py b/tests/domain/sap10_calculator/worksheet/test_heat_transmission.py index b6a21d0a..3153f1bc 100644 --- a/tests/domain/sap10_calculator/worksheet/test_heat_transmission.py +++ b/tests/domain/sap10_calculator/worksheet/test_heat_transmission.py @@ -151,6 +151,112 @@ def test_roof_insulated_assumed_with_ni_thickness_uses_50mm_per_section_5_11_4() assert result.roof_w_per_k == pytest.approx(68.0, abs=2.0) +def test_lodged_roof_u_value_overrides_construction_default() -> None: + # Arrange — RdSAP 10 §5.1: where an element's U-value is known from the + # assessment (documentary evidence / the lodged RdSAP output) it is used + # directly in place of the §5.11 construction-default cascade. The gov-EPC + # API surfaces the assessor's roof U as `roof_u_value`; the gov open data + # can redact the backing insulation thickness, leaving the §5.11 cascade + # to mis-derive an uninsulated U. Cert 7921-0052-0940-5007-0663 lodges a + # "Pitched, sloping ceiling" (rc=8) age-C roof with no thickness — the + # cascade returns the uninsulated 2.30 W/m²K (→ -23 SAP) where the lodged + # roof_u_value is 0.2. Geometry: 100 m² plan → sloped roof area = + # 100 / cos(30°) = 115.47 m². roof_w_per_k must follow the lodged 0.2 + # (0.2 × 115.47 = 23.094 W/K), NOT the 2.30 × 115.47 = 265 W/K default. + main = make_building_part( + construction_age_band="C", + wall_construction=3, + wall_insulation_type=4, + party_wall_construction=1, + roof_construction=8, + floor_dimensions=[ + make_floor_dimension( + total_floor_area_m2=100.0, room_height_m=2.5, + party_wall_length_m=0.0, heat_loss_perimeter_m=40.0, floor=0, + ), + ], + ) + main.roof_construction_type = "Pitched, sloping ceiling" + main.roof_u_value = 0.2 + epc = make_minimal_sap10_epc( + total_floor_area_m2=100.0, + country_code="ENG", + sap_building_parts=[main], + ) + + # Act + result = heat_transmission_from_cert(epc) + + # Assert — 0.2 × (100 / cos(30°)) = 0.2 × 115.470 = 23.094 W/K. + assert abs(result.roof_w_per_k - 23.094) <= 1e-3 + + +def test_lodged_wall_u_value_overrides_construction_default() -> None: + # Arrange — RdSAP 10 §5.1: a lodged main-wall U-value (the gov-EPC API + # `wall_u_value`, the assessor's RdSAP output) overrides the §5.6/§5.7 + # construction-default cascade. Cohort certs 2021/7505 lodge solid-brick + # (rc=3) walls with the open data's insulation redacted, where the lodged + # wall_u_value (0.34) is far below the cascade's uninsulated default → the + # cascade under-rates by ~-2.6 SAP. Geometry chosen so the gross wall area + # (no openings) is 80 m²: net wall U×A must follow 0.34, NOT the default. + main = make_building_part( + construction_age_band="C", + wall_construction=3, + wall_insulation_type=4, + party_wall_construction=1, + roof_construction=4, + floor_dimensions=[ + make_floor_dimension( + total_floor_area_m2=100.0, room_height_m=2.5, + party_wall_length_m=0.0, heat_loss_perimeter_m=40.0, floor=0, + ), + ], + ) + main.wall_u_value = 0.34 + epc = make_minimal_sap10_epc( + total_floor_area_m2=100.0, + country_code="ENG", + sap_building_parts=[main], + ) + + # Act + result = heat_transmission_from_cert(epc) + + # Assert — gross wall = perimeter 40 m × height 2.5 m = 100 m²; no windows + # or doors in this minimal cert → net wall area 100 m². 0.34 × 100 = 34.0. + assert abs(result.walls_w_per_k - 34.0) <= 1e-6 + + +def test_lodged_floor_u_value_overrides_iso_13370_cascade() -> None: + # Arrange — RdSAP 10 §5.1: a lodged ground-floor U-value (the gov-EPC API + # `floor_u_value`) overrides the BS EN ISO 13370 / Table 19 cascade. + main = make_building_part( + construction_age_band="C", + wall_construction=3, + wall_insulation_type=4, + party_wall_construction=1, + roof_construction=4, + floor_dimensions=[ + make_floor_dimension( + total_floor_area_m2=100.0, room_height_m=2.5, + party_wall_length_m=0.0, heat_loss_perimeter_m=40.0, floor=0, + ), + ], + ) + main.floor_u_value = 0.12 + epc = make_minimal_sap10_epc( + total_floor_area_m2=100.0, + country_code="ENG", + sap_building_parts=[main], + ) + + # Act + result = heat_transmission_from_cert(epc) + + # Assert — 0.12 × 100 m² floor = 12.0 W/K. + assert abs(result.floor_w_per_k - 12.0) <= 1e-6 + + def test_exposed_timber_floor_age_b_uses_table_20_u_120_not_iso_13370() -> None: # Arrange — RdSAP10 §5.13 Table 20: a part whose lowest floor sits # over outside air (or unheated space) rather than soil takes its From a135d887218d6e18936c29645ec65015e5d24d84 Mon Sep 17 00:00:00 2001 From: Daniel Roth Date: Fri, 12 Jun 2026 16:04:19 +0000 Subject: [PATCH 57/87] Rename files in subfolders too --- .claude/settings.local.json | 30 ++++ scripts/rename_sharepoint_files.py | 79 +++++---- tests/scripts/__init__.py | 0 tests/scripts/test_rename_sharepoint_files.py | 161 ++++++++++++++++++ 4 files changed, 235 insertions(+), 35 deletions(-) create mode 100644 .claude/settings.local.json create mode 100644 tests/scripts/__init__.py create mode 100644 tests/scripts/test_rename_sharepoint_files.py diff --git a/.claude/settings.local.json b/.claude/settings.local.json new file mode 100644 index 00000000..ed0600e5 --- /dev/null +++ b/.claude/settings.local.json @@ -0,0 +1,30 @@ +{ + "permissions": { + "allow": [ + "Bash(python -m pytest backend/pashub_fetcher/tests/test_pashub_client.py::test_select_latest_core_files_falls_back_to_latest_when_no_osm_candidates -v)", + "Bash(python -m pyright backend/app/db/functions/magic_plan_functions.py)", + "Bash(npx pyright *)", + "Bash(python -m pytest backend/app/db/functions/tests/test_magic_plan_functions.py --no-header -q)", + "Bash(python -m pyright backend/magic_plan/tests/test_audit_script.py)", + "Bash(find /workspaces/model -name \"pyrightconfig.json\" 2>/dev/null | head -5 && which pyright || find /home -name \"pyright\" 2>/dev/null | head -3 && find /usr -name \"pyright\" 2>/dev/null | head -3)", + "Read(//home/**)", + "Read(//usr/**)", + "Bash(/home/vscode/.npm/_npx/110e52990071af13/node_modules/.bin/pyright backend/magic_plan/tests/test_audit_script.py)", + "Bash(python -m pytest backend/magic_plan/tests/test_audit_script.py::test_write_headers_two_rows_correct_labels_and_column_positions -x)", + "Bash(python -m pytest backend/magic_plan/tests/test_audit_script.py -x)", + "Bash(python -m pytest backend/magic_plan/tests/test_audit_script.py::test_apply_section_borders_sets_medium_right_border_on_boundary_columns -x)", + "Bash(python -m pytest backend/magic_plan/tests/test_audit_script.py)", + "Bash(/home/vscode/.npm/_npx/110e52990071af13/node_modules/.bin/pyright backend/magic_plan/audit_script.py)", + "Bash(mkdir -p /workspaces/model/infrastructure/solar)", + "Bash(mkdir -p /workspaces/model/tests/infrastructure/solar)", + "Bash(python -m pytest datatypes/magicplan/domain/tests/test_mapper.py::test_kitchen_window_has_ventilation -x)", + "Bash(python -m pytest datatypes/magicplan/domain/tests/test_mapper.py -x)", + "Bash(git add *)", + "Bash(git commit -m ' *)", + "Bash(python3 -c ' *)", + "Bash(python -m pytest datatypes/magicplan/domain/tests/test_mapper.py::test_toilet_door_has_ventilation_undercut -x)", + "Bash(python -m pytest tests/infrastructure/postgres/test_uploaded_file_table.py -x -q)", + "Bash(python -m pytest tests/infrastructure/postgres/test_uploaded_file_table.py tests/repositories/tasks/ tests/repositories/magic_plan/ tests/repositories/property/ -x -q)" + ] + } +} diff --git a/scripts/rename_sharepoint_files.py b/scripts/rename_sharepoint_files.py index 881b96ef..a7306d88 100644 --- a/scripts/rename_sharepoint_files.py +++ b/scripts/rename_sharepoint_files.py @@ -16,8 +16,8 @@ from utils.logger import setup_logger from utils.sharepoint.domna_sharepoint_client import DomnaSharepointClient from utils.sharepoint.domna_sites import DomnaSites -DRY_RUN: bool = True -CSV_PATH: str = "scripts/sero_address_list.csv" +DRY_RUN: bool = False +CSV_PATH: str = "scripts/sero_address_list_test.csv" BASE_PATH = ( "Osmosis-ACD Projects/Sero-Clarion Housing/" @@ -70,6 +70,47 @@ def build_canonical_filename( return f"{uprn}_{street_post}{ext}" +def process_folder( + sp_client: DomnaSharepointClient, + folder_path: str, + uprn: str, + address: str, + postcode: str, +) -> None: + try: + contents = sp_client.get_folders_in_path(folder_path) + except ValueError: + logger.warning(f"Missing folder for UPRN {uprn}: {folder_path}") + return + + for item in contents.get("value", []): + if "folder" in item: + process_folder( + sp_client, f"{folder_path}/{item['name']}", uprn, address, postcode + ) + elif "file" in item: + original_name: str = item["name"] + new_name = build_canonical_filename(uprn, address, postcode, original_name) + + if new_name is None: + continue + + if DRY_RUN: + logger.info( + f'[DRY RUN] Renaming: "{original_name}" → "{new_name}" (UPRN: {uprn})' + ) + else: + try: + sp_client.rename_file(item["id"], new_name) + logger.info( + f'Renamed: "{original_name}" → "{new_name}" (UPRN: {uprn})' + ) + except Exception as e: + logger.error( + f'Failed to rename "{original_name}" → "{new_name}" (UPRN: {uprn}): {e}' + ) + + def main() -> None: sp_client = DomnaSharepointClient(DomnaSites.SOCIAL_HOUSING_WAVE_3) @@ -89,39 +130,7 @@ def main() -> None: f"{BASE_PATH}/{address}, {postcode}" f"/{SharepointSubfolders.ASSESSMENT.value}/{ASSESSMENT_SUBFOLDER}" ) - - try: - contents = sp_client.get_folders_in_path(folder_path) - except ValueError: - logger.warning(f"Missing folder for UPRN {uprn}: {folder_path}") - continue - - for item in contents.get("value", []): - if "file" not in item: - continue - - original_name: str = item["name"] - new_name = build_canonical_filename( - uprn, address, postcode, original_name - ) - - if new_name is None: - continue - - if DRY_RUN: - logger.info( - f'[DRY RUN] Renaming: "{original_name}" → "{new_name}" (UPRN: {uprn})' - ) - else: - try: - sp_client.rename_file(item["id"], new_name) - logger.info( - f'Renamed: "{original_name}" → "{new_name}" (UPRN: {uprn})' - ) - except Exception as e: - logger.error( - f'Failed to rename "{original_name}" → "{new_name}" (UPRN: {uprn}): {e}' - ) + process_folder(sp_client, folder_path, uprn, address, postcode) if __name__ == "__main__": diff --git a/tests/scripts/__init__.py b/tests/scripts/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/tests/scripts/test_rename_sharepoint_files.py b/tests/scripts/test_rename_sharepoint_files.py new file mode 100644 index 00000000..4525fe84 --- /dev/null +++ b/tests/scripts/test_rename_sharepoint_files.py @@ -0,0 +1,161 @@ +from typing import Any +from unittest.mock import MagicMock, call, patch + +import pytest + +import scripts.rename_sharepoint_files as module +from scripts.rename_sharepoint_files import build_canonical_filename, process_folder + + +def _make_file(name: str, item_id: str = "id-1") -> dict[str, Any]: + return {"name": name, "id": item_id, "file": {}} + + +def _make_folder(name: str) -> dict[str, Any]: + return {"name": name, "folder": {}} + + +def _make_package(name: str) -> dict[str, Any]: + return {"name": name, "package": {}} + + +# --------------------------------------------------------------------------- +# build_canonical_filename +# --------------------------------------------------------------------------- + + +def test_already_canonical_returns_none() -> None: + assert build_canonical_filename("100", "1 High St", "AB1 2CD", "100_High St AB1 2CD_Report.pdf") is None + + +def test_strips_address_prefix_and_adds_uprn() -> None: + result = build_canonical_filename("100", "1 High St", "AB1 2CD", "1 High St AB1 2CD - Survey.pdf") + assert result == "100_1 High St AB1 2CD_Survey.pdf" + + +def test_no_prefix_still_canonical() -> None: + result = build_canonical_filename("100", "1 High St", "AB1 2CD", "Survey.pdf") + assert result == "100_1 High St AB1 2CD_Survey.pdf" + + +# --------------------------------------------------------------------------- +# process_folder — files only at root level +# --------------------------------------------------------------------------- + + +def test_renames_top_level_files(caplog: pytest.LogCaptureFixture) -> None: + sp = MagicMock() + sp.get_folders_in_path.return_value = { + "value": [ + _make_file("Survey.pdf", "id-1"), + _make_file("Report.docx", "id-2"), + ] + } + + with patch.object(module, "DRY_RUN", False): + process_folder(sp, "some/path", "100", "1 High St", "AB1 2CD") + + assert sp.rename_file.call_count == 2 + sp.rename_file.assert_any_call("id-1", "100_1 High St AB1 2CD_Survey.pdf") + sp.rename_file.assert_any_call("id-2", "100_1 High St AB1 2CD_Report.docx") + + +# --------------------------------------------------------------------------- +# process_folder — recursive two-level hierarchy +# --------------------------------------------------------------------------- + + +def test_recurses_into_subfolders_and_renames_all_files() -> None: + sp = MagicMock() + + root_contents: dict[str, Any] = { + "value": [ + _make_file("Root.pdf", "root-file"), + _make_folder("SubA"), + ] + } + suba_contents: dict[str, Any] = { + "value": [ + _make_file("Sub.pdf", "sub-file"), + ] + } + + sp.get_folders_in_path.side_effect = lambda path: ( + root_contents if path == "base/path" else suba_contents + ) + + with patch.object(module, "DRY_RUN", False): + process_folder(sp, "base/path", "200", "2 Main Rd", "XY9 8ZW") + + assert sp.rename_file.call_count == 2 + sp.rename_file.assert_any_call("root-file", "200_2 Main Rd XY9 8ZW_Root.pdf") + sp.rename_file.assert_any_call("sub-file", "200_2 Main Rd XY9 8ZW_Sub.pdf") + + sp.get_folders_in_path.assert_any_call("base/path/SubA") + + +# --------------------------------------------------------------------------- +# process_folder — non-file, non-folder items are skipped +# --------------------------------------------------------------------------- + + +def test_ignores_package_items() -> None: + sp = MagicMock() + sp.get_folders_in_path.return_value = { + "value": [_make_package("Notebook")] + } + + with patch.object(module, "DRY_RUN", False): + process_folder(sp, "some/path", "300", "3 Oak Ave", "ZZ1 1ZZ") + + sp.rename_file.assert_not_called() + assert sp.get_folders_in_path.call_count == 1 + + +# --------------------------------------------------------------------------- +# process_folder — missing folder +# --------------------------------------------------------------------------- + + +def test_missing_folder_logs_warning_and_returns(caplog: pytest.LogCaptureFixture) -> None: + sp = MagicMock() + sp.get_folders_in_path.side_effect = ValueError("not found") + + with patch.object(module, "DRY_RUN", False): + process_folder(sp, "missing/path", "400", "4 Elm St", "AA2 2BB") + + sp.rename_file.assert_not_called() + assert any("Missing folder" in r.message and "400" in r.message for r in caplog.records) + + +# --------------------------------------------------------------------------- +# process_folder — dry run +# --------------------------------------------------------------------------- + + +def test_dry_run_logs_without_renaming(caplog: pytest.LogCaptureFixture) -> None: + sp = MagicMock() + sp.get_folders_in_path.return_value = {"value": [_make_file("Doc.pdf", "id-x")]} + + with patch.object(module, "DRY_RUN", True): + process_folder(sp, "some/path", "500", "5 Pine Ln", "BB3 3CC") + + sp.rename_file.assert_not_called() + assert any("[DRY RUN]" in r.message for r in caplog.records) + + +# --------------------------------------------------------------------------- +# process_folder — already-canonical files are skipped +# --------------------------------------------------------------------------- + + +def test_skips_already_canonical_files() -> None: + sp = MagicMock() + sp.get_folders_in_path.return_value = { + "value": [_make_file("500_Pine Ln BB3 3CC_Doc.pdf", "id-y")] + } + + with patch.object(module, "DRY_RUN", False): + process_folder(sp, "some/path", "500", "5 Pine Ln", "BB3 3CC") + + sp.rename_file.assert_not_called() From 2bfbad5ced1c0b2de059e99fcc63a0cfb6696cea Mon Sep 17 00:00:00 2001 From: Daniel Roth Date: Fri, 12 Jun 2026 16:11:48 +0000 Subject: [PATCH 58/87] add local claude settings to gitignore --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index 285309dd..e80431de 100644 --- a/.gitignore +++ b/.gitignore @@ -303,3 +303,4 @@ backlog/* # Local Claude config files .claude/*modelling_cohort.csv +.claude/settings.local.json From 92ba2b92993646badad1d7c9acf3566fc3469e6c Mon Sep 17 00:00:00 2001 From: Daniel Roth Date: Fri, 12 Jun 2026 16:15:16 +0000 Subject: [PATCH 59/87] remove local file from branch --- .claude/settings.local.json | 30 ------------------------------ 1 file changed, 30 deletions(-) delete mode 100644 .claude/settings.local.json diff --git a/.claude/settings.local.json b/.claude/settings.local.json deleted file mode 100644 index ed0600e5..00000000 --- a/.claude/settings.local.json +++ /dev/null @@ -1,30 +0,0 @@ -{ - "permissions": { - "allow": [ - "Bash(python -m pytest backend/pashub_fetcher/tests/test_pashub_client.py::test_select_latest_core_files_falls_back_to_latest_when_no_osm_candidates -v)", - "Bash(python -m pyright backend/app/db/functions/magic_plan_functions.py)", - "Bash(npx pyright *)", - "Bash(python -m pytest backend/app/db/functions/tests/test_magic_plan_functions.py --no-header -q)", - "Bash(python -m pyright backend/magic_plan/tests/test_audit_script.py)", - "Bash(find /workspaces/model -name \"pyrightconfig.json\" 2>/dev/null | head -5 && which pyright || find /home -name \"pyright\" 2>/dev/null | head -3 && find /usr -name \"pyright\" 2>/dev/null | head -3)", - "Read(//home/**)", - "Read(//usr/**)", - "Bash(/home/vscode/.npm/_npx/110e52990071af13/node_modules/.bin/pyright backend/magic_plan/tests/test_audit_script.py)", - "Bash(python -m pytest backend/magic_plan/tests/test_audit_script.py::test_write_headers_two_rows_correct_labels_and_column_positions -x)", - "Bash(python -m pytest backend/magic_plan/tests/test_audit_script.py -x)", - "Bash(python -m pytest backend/magic_plan/tests/test_audit_script.py::test_apply_section_borders_sets_medium_right_border_on_boundary_columns -x)", - "Bash(python -m pytest backend/magic_plan/tests/test_audit_script.py)", - "Bash(/home/vscode/.npm/_npx/110e52990071af13/node_modules/.bin/pyright backend/magic_plan/audit_script.py)", - "Bash(mkdir -p /workspaces/model/infrastructure/solar)", - "Bash(mkdir -p /workspaces/model/tests/infrastructure/solar)", - "Bash(python -m pytest datatypes/magicplan/domain/tests/test_mapper.py::test_kitchen_window_has_ventilation -x)", - "Bash(python -m pytest datatypes/magicplan/domain/tests/test_mapper.py -x)", - "Bash(git add *)", - "Bash(git commit -m ' *)", - "Bash(python3 -c ' *)", - "Bash(python -m pytest datatypes/magicplan/domain/tests/test_mapper.py::test_toilet_door_has_ventilation_undercut -x)", - "Bash(python -m pytest tests/infrastructure/postgres/test_uploaded_file_table.py -x -q)", - "Bash(python -m pytest tests/infrastructure/postgres/test_uploaded_file_table.py tests/repositories/tasks/ tests/repositories/magic_plan/ tests/repositories/property/ -x -q)" - ] - } -} From 4fb9b853dc789035251541c2c740924b01f59c58 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Sat, 13 Jun 2026 23:15:32 +0000 Subject: [PATCH 60/87] fix(ventilation): apply Table 4g note 3 in-use factor to index-less MEV SFP MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The no-PCDB MEV fan-electricity path fed the SAP 10.2 Table 4g default SFP (0.8 W/(l/s)) directly as SFPav. But Table 4g note 3 (PDF p.176) is explicit: the default SFP values "are to be multiplied by the appropriate in-use factor for default data from the PCDB" — PCDB Table 329 system_type 10 ("default data, used when SFP is taken from Table 4g rather than the PCDB"), IUF 2.5 (duct-agnostic per note 2). Table 4h, which previously held these factors, is retired ("no longer used – data now stored in the PCDB"). Omitting the IUF under-billed the index-less MEV fan electricity by 2.5x (SFPav 0.8 instead of 0.8 x 2.5 = 2.0), so cost was too low and the cohort over-rated. This is distinct from the with-index path, which already applies the tested-product system_type-2 "no scheme" IUF (~1.45) per fan. Index-less gas-house MEV cohort: +1.37 median -> -0.18 (12% -> 92% within 0.5), no overshoot — the missing IUF was exactly the over-rate. API gauge 67.7% -> 68.4% within-0.5 (mean|err| 0.992 -> 0.986, signed +0.031 -> +0.006). Worksheet harness 47/47, 0 divergers (Summary-path MEV certs carry a PCDB index or are natural, so unaffected). Co-Authored-By: Claude Opus 4.8 --- .../sap10_calculator/rdsap/cert_to_inputs.py | 33 +++++++++- .../rdsap/test_cert_to_inputs.py | 65 ++++++++++++++++--- 2 files changed, 86 insertions(+), 12 deletions(-) diff --git a/domain/sap10_calculator/rdsap/cert_to_inputs.py b/domain/sap10_calculator/rdsap/cert_to_inputs.py index 698c5e54..d293c193 100644 --- a/domain/sap10_calculator/rdsap/cert_to_inputs.py +++ b/domain/sap10_calculator/rdsap/cert_to_inputs.py @@ -566,6 +566,11 @@ def _table_4f_additive_components(epc: EpcPropertyData) -> float: _MEV_KITCHEN_FAN_CONFIG_CODES: Final[frozenset[int]] = frozenset({1, 3, 5}) # PCDB Table 329 / 322 system_type=2 = decentralised MEV. _MEV_DECENTRALISED_SYSTEM_TYPE: Final[int] = 2 +# PCDB Table 329 system_type=10 = "default data" (PCDF Spec §A.20) — the +# in-use-factor row used when the SFP is taken from SAP 10.2 Table 4g +# rather than a specific PCDB product. Table 4g note 3 (PDF p.176) +# requires the default SFP to be multiplied by this IUF (2.5, duct-agnostic). +_MEV_DEFAULT_DATA_SYSTEM_TYPE: Final[int] = 10 # Elmhurst "Duct Type" cascade integer: 1=Flexible, 2=Rigid (per # `_ELMHURST_DUCT_TYPE_TO_INT` in datatypes.epc.domain.mapper). _MV_DUCT_TYPE_FLEXIBLE: Final[int] = 1 @@ -578,11 +583,25 @@ _MEV_THROUGH_WALL_CONFIG_CODES: Final[frozenset[int]] = frozenset({5, 6}) # SAP 10.2 Table 4g (PDF p.176) / §2.6.3 note 1 — default specific fan # power (W per litre/sec) for an MEV system whose fan(s) are not in the -# PCDB. Used directly as the IUF-adjusted SFPav in the (230a) formula -# (SFPav × 1.22 × V) when no Table 322 record resolves. +# PCDB. This is the RAW SFP: Table 4g note 3 requires it to be multiplied +# by the "default data" in-use factor (Table 329 system_type 10) before +# use as the SFPav in the (230a) formula (SFPav × 1.22 × V). _TABLE_4G_DEFAULT_MEV_SFP_W_PER_L_PER_S: Final[float] = 0.8 +def _mev_default_data_iuf() -> float: + """SAP 10.2 Table 4g note 3 (PDF p.176) in-use factor for default MEV + data — PCDB Table 329 system_type 10 (IUF 2.5, identical across rigid/ + flexible/no-duct columns per Table 4g note 2 "applies to both rigid and + flexible ducting"). Falls back to 1.0 if the Table 329 record is + unavailable (ETL bootstrap), preserving the pre-fix raw-SFP behaviour + rather than zeroing fan electricity.""" + record = mv_in_use_factors_record(_MEV_DEFAULT_DATA_SYSTEM_TYPE) + if record is None or record.sfp_iuf_rigid_no_scheme is None: + return 1.0 + return record.sfp_iuf_rigid_no_scheme + + def _mev_decentralised_kwh_per_yr_from_cert(epc: EpcPropertyData) -> float: """Compose the SAP 10.2 §5 Table 4f line (230a) MEV decentralised annual electricity contribution from PCDB Tables 322 (per-fan SFP @@ -635,8 +654,16 @@ def _mev_decentralised_kwh_per_yr_from_cert(epc: EpcPropertyData) -> float: != MechanicalVentilationKind.EXTRACT_OR_PIV_OUTSIDE.name ): return 0.0 + # SAP 10.2 Table 4g note 3 (PDF p.176): the default SFP "[is] to be + # multiplied by the appropriate in-use factor for default data from + # the PCDB" (Table 329 system_type 10, IUF 2.5). Omitting it + # under-billed the index-less MEV fan electricity by 2.5x → +1.3 SAP + # over-rate on the no-PCDB MEV cohort (mostly gas houses). Distinct + # from the with-index path below, which applies the tested-product + # system_type-2 "no scheme" IUF (~1.45) per fan. + sfp_av = _TABLE_4G_DEFAULT_MEV_SFP_W_PER_L_PER_S * _mev_default_data_iuf() return mev_decentralised_kwh_per_yr( - sfp_av_w_per_l_per_s=_TABLE_4G_DEFAULT_MEV_SFP_W_PER_L_PER_S, + sfp_av_w_per_l_per_s=sfp_av, dwelling_volume_m3=dimensions_from_cert(epc).volume_m3, ) iuf_record = mv_in_use_factors_record(_MEV_DECENTRALISED_SYSTEM_TYPE) diff --git a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py index c088628d..ed9d43ab 100644 --- a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py +++ b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py @@ -59,6 +59,10 @@ from domain.sap10_calculator.rdsap.cert_to_inputs import ( _heat_network_factor_fuel_code, # pyright: ignore[reportPrivateUsage] _heat_network_standing_charge_gbp, # pyright: ignore[reportPrivateUsage] _main_fuel_code, # pyright: ignore[reportPrivateUsage] + _mev_decentralised_kwh_per_yr_from_cert, # pyright: ignore[reportPrivateUsage] + _mev_default_data_iuf, # pyright: ignore[reportPrivateUsage] + _TABLE_4G_DEFAULT_MEV_SFP_W_PER_L_PER_S, # pyright: ignore[reportPrivateUsage] + dimensions_from_cert, _table_12_factor_fuel_code, # pyright: ignore[reportPrivateUsage] _is_electric_main, # pyright: ignore[reportPrivateUsage] _is_heat_network_electric_main, # pyright: ignore[reportPrivateUsage] @@ -90,6 +94,7 @@ from domain.sap10_calculator.rdsap.cert_to_inputs import ( ventilation_from_cert, ) from domain.sap10_calculator.tables.pcdb import GasOilBoilerRecord, gas_oil_boiler_record +from domain.sap10_calculator.worksheet.mev import mev_decentralised_kwh_per_yr from tests.domain.sap10_calculator.worksheet import _elmhurst_worksheet_000477 as _w000477 from domain.sap10_calculator.worksheet.water_heating import ( combi_loss_monthly_kwh_table_3b_row_1_instantaneous, @@ -1436,12 +1441,12 @@ def test_corridor_flat_assumes_draught_lobby_present_zeroing_line_13() -> None: def test_index_less_mev_uses_table_4g_default_sfp_for_fan_electricity() -> None: # Arrange — an MEV system with NO PCDB record (index absent / not in - # Table 322). SAP 10.2 §2.6.3 / Table 4g note 1 prescribes a default - # specific fan power of 0.8 W/(l/s), used directly as the SFPav in the - # §5 Table 4f line (230a) `SFPav × 1.22 × V`. Without it the cascade - # billed ZERO fan electricity for index-less MEV (the +2.2 SAP - # over-rate on the index-less MEV cohort, mostly gas houses). A natural - # dwelling contributes nothing. + # Table 322). SAP 10.2 §2.6.3 / Table 4g gives a default specific fan + # power of 0.8 W/(l/s); Table 4g note 3 (PDF p.176) requires multiplying + # it by the default-data in-use factor (Table 329 system_type 10, IUF + # 2.5) before use as the SFPav in the §5 Table 4f line (230a) + # `SFPav × 1.22 × V`. So the effective SFPav is 0.8 × 2.5 = 2.0. A + # natural dwelling contributes nothing. from domain.sap10_calculator.rdsap.cert_to_inputs import ( _TABLE_4G_DEFAULT_MEV_SFP_W_PER_L_PER_S, # pyright: ignore[reportPrivateUsage] _mev_decentralised_kwh_per_yr_from_cert, # pyright: ignore[reportPrivateUsage] @@ -1468,10 +1473,10 @@ def test_index_less_mev_uses_table_4g_default_sfp_for_fan_electricity() -> None: mev_kwh = _mev_decentralised_kwh_per_yr_from_cert(mev_no_index) natural_kwh = _mev_decentralised_kwh_per_yr_from_cert(natural) - # Assert — default SFP 0.8 × 1.22 × V for the index-less MEV; zero for - # the naturally-ventilated dwelling. + # Assert — IUF-adjusted SFPav (0.8 × 2.5) × 1.22 × V for the index-less + # MEV; zero for the naturally-ventilated dwelling. volume = dimensions_from_cert(mev_no_index).volume_m3 - expected = _TABLE_4G_DEFAULT_MEV_SFP_W_PER_L_PER_S * 1.22 * volume + expected = _TABLE_4G_DEFAULT_MEV_SFP_W_PER_L_PER_S * 2.5 * 1.22 * volume assert abs(mev_kwh - expected) <= 1e-6 assert mev_kwh > 0.0 assert natural_kwh == 0.0 @@ -7306,3 +7311,45 @@ def test_non_heat_network_main_returns_none_so_caller_uses_fuel_standing() -> No # Act / Assert assert _heat_network_standing_charge_gbp(epc, main) is None + + +def test_mev_default_data_iuf_is_table_329_system_type_10_value() -> None: + # Arrange / Act — SAP 10.2 Table 4g note 3 (PDF p.176) directs the + # default SFP to "the appropriate in-use factor for default data from + # the PCDB" = Table 329 system_type 10, which lodges IUF 2.5 (identical + # across rigid/flexible/no-duct columns). + iuf = _mev_default_data_iuf() + + # Assert + assert abs(iuf - 2.5) <= 1e-9 + + +def test_index_less_mev_applies_table_4g_note_3_default_data_iuf() -> None: + # Arrange — an MEV (mechanical extract) dwelling with NO PCDB index. + # SAP 10.2 Table 4g gives the default SFP 0.8 W/(l/s), and note 3 + # requires multiplying it by the default-data in-use factor (2.5) before + # use as SFPav in the (230a) fan-electricity formula. Before this fix + # the raw 0.8 was used directly, under-billing fan electricity by 2.5x + # and over-rating the index-less MEV cohort by ~+1.3 SAP. + epc = make_minimal_sap10_epc( + total_floor_area_m2=_TYPICAL_TFA_M2, + habitable_rooms_count=4, + country_code="ENG", + sap_building_parts=[make_building_part()], + sap_ventilation=SapVentilation( + mechanical_ventilation_kind="EXTRACT_OR_PIV_OUTSIDE", + ), + ) + + # Act + fan_kwh = _mev_decentralised_kwh_per_yr_from_cert(epc) + volume_m3 = dimensions_from_cert(epc).volume_m3 + expected = mev_decentralised_kwh_per_yr( + sfp_av_w_per_l_per_s=_TABLE_4G_DEFAULT_MEV_SFP_W_PER_L_PER_S * 2.5, + dwelling_volume_m3=volume_m3, + ) + + # Assert — fan electricity follows the IUF-adjusted SFPav (2.0), i.e. + # 2.5x the raw-0.8 value, not the raw default. + assert fan_kwh > 0.0 + assert abs(fan_kwh - expected) <= 1e-9 From 5317175dd3983ddf80525883771f85216c2eb871 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Sat, 13 Jun 2026 23:31:02 +0000 Subject: [PATCH 61/87] fix(water-heating): count electric showers in Noutlets for mixer demand (App J) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The mixer-shower hot-water demand (worksheet 42a) divided N_shower by the count of MIXER outlets only. But SAP 10.2 Appendix J step 1a is explicit: "Establish how many shower outlets are present in the dwelling, Noutlets (including in the count any instantaneous electric showers)" — and the electric-shower step (64a) uses that same Noutlets from step 1a. So a dwelling with both a mixer and an electric shower assigned the FULL N_shower to the mixer system AND billed the electric shower on top of it, double- counting shower demand → over-counted main HW → under-rated the dwelling. Fix: thread the electric-shower count into the mixer demand so the denominator is the total outlet count (mixer + electric), iterating the warm-water draw over the mixer outlets only (per step 1e). shower_types=1,2 cohort: -0.37 median -> +0.28 (crossed zero); API gauge 68.4% -> 69.0% within-0.5. Golden cert 0300-2747 (1 mixer + 1 electric) re-pinned: PE +0.93 -> -0.10, CO2 +0.25 -> +0.15 (both toward zero, confirming the double-count). Worksheet harness 47/47, 0 divergers (the Elmhurst fixtures have no electric showers). Co-Authored-By: Claude Opus 4.8 --- .../worksheet/water_heating.py | 22 +++++++++++---- .../rdsap/test_golden_fixtures.py | 10 +++++-- .../worksheet/test_water_heating.py | 28 +++++++++++++++++++ 3 files changed, 52 insertions(+), 8 deletions(-) diff --git a/domain/sap10_calculator/worksheet/water_heating.py b/domain/sap10_calculator/worksheet/water_heating.py index 0c8ebd5c..56eb0cd9 100644 --- a/domain/sap10_calculator/worksheet/water_heating.py +++ b/domain/sap10_calculator/worksheet/water_heating.py @@ -132,6 +132,7 @@ def hot_water_mixer_showers_monthly_l_per_day( has_bath: bool, mixer_shower_flow_rates_l_per_min: tuple[float, ...], cold_water_temps_c: tuple[float, ...], + n_electric_showers: int = 0, ) -> tuple[float, ...]: """SAP 10.2 §4 line (42a)m via Appendix J equations J1, J2, J3. @@ -145,14 +146,23 @@ def hot_water_mixer_showers_monthly_l_per_day( Each outlet's warm-water draw for a month is the flow rate × 6 min × Table J5 fbeh. The hot fraction is (41 − Tcold[m])/(52 − Tcold[m]). Per-outlet daily warm water is scaled by N_shower / N_outlets, then - summed across outlets to give (42a)m. + summed across the MIXER outlets to give (42a)m. - Instantaneous electric showers belong to worksheet (64a)m, not (42a)m - — those should be excluded from `mixer_shower_flow_rates_l_per_min`. + N_outlets is the dwelling's TOTAL shower-outlet count: SAP 10.2 + Appendix J step 1a is explicit that the count INCLUDES "any + instantaneous electric showers" (which then bill their own energy via + (64a)m). So `mixer_shower_flow_rates_l_per_min` lists only the mixer + outlets (iterated for the warm-water draw), but `n_electric_showers` + must be added to the denominator — otherwise a dwelling with both a + mixer and an electric shower assigns the FULL N_shower to the mixer + system AND bills the electric shower on top, double-counting shower + demand (over-counts main HW → under-rates the dwelling). """ - n_outlets = len(mixer_shower_flow_rates_l_per_min) - if n_outlets == 0: + n_mixer_outlets = len(mixer_shower_flow_rates_l_per_min) + if n_mixer_outlets == 0: return tuple(0.0 for _ in range(12)) + # Appendix J step 1a: Noutlets includes instantaneous electric showers. + n_outlets = n_mixer_outlets + n_electric_showers if has_bath: n_shower = 0.45 * n_occupants + 0.65 else: @@ -894,6 +904,8 @@ def water_heating_from_cert( has_bath=has_bath, mixer_shower_flow_rates_l_per_min=mixer_shower_flow_rates_l_per_min, cold_water_temps_c=cold_water_temps_c, + # SAP 10.2 Appendix J step 1a — Noutlets includes electric showers. + n_electric_showers=electric_shower_count if has_electric_shower else 0, ) has_shower = ( len(mixer_shower_flow_rates_l_per_min) > 0 diff --git a/tests/domain/sap10_calculator/rdsap/test_golden_fixtures.py b/tests/domain/sap10_calculator/rdsap/test_golden_fixtures.py index 63e3b83d..a01b25e8 100644 --- a/tests/domain/sap10_calculator/rdsap/test_golden_fixtures.py +++ b/tests/domain/sap10_calculator/rdsap/test_golden_fixtures.py @@ -224,8 +224,8 @@ _EXPECTATIONS: tuple[_GoldenExpectation, ...] = ( cert_number="0300-2747-7640-2526-2135", actual_sap=78, expected_sap_resid=+0, - expected_pe_resid_kwh_per_m2=+0.9264, - expected_co2_resid_tonnes_per_yr=+0.2495, + expected_pe_resid_kwh_per_m2=-0.1033, + expected_co2_resid_tonnes_per_yr=+0.1488, notes=( "Large semi-detached, TFA 526, age D, gas boiler PCDB-listed " "(no Table 4b code). Cert lodges open_flues_count=1 + " @@ -245,7 +245,11 @@ _EXPECTATIONS: tuple[_GoldenExpectation, ...] = ( "CO2 / PE factors through the cert's mains-gas " "`secondary_fuel_type` (mirroring the cost-side Slice 58 " "fix), closing PE +8.28 → +0.93 and CO2 −0.25 → +0.25 — " - "second-biggest cohort PE closure to date." + "second-biggest cohort PE closure to date. Slice S0380.xx " + "(SAP 10.2 Appendix J step 1a — Noutlets includes electric " + "showers) halved this cert's mixer-shower HW (1 mixer + 1 " + "electric → /2 not /1), removing the double-counted shower " + "demand: PE +0.93 → -0.10, CO2 +0.25 → +0.15 (both toward 0)." ), ), _GoldenExpectation( diff --git a/tests/domain/sap10_calculator/worksheet/test_water_heating.py b/tests/domain/sap10_calculator/worksheet/test_water_heating.py index e8e85b1e..81086c46 100644 --- a/tests/domain/sap10_calculator/worksheet/test_water_heating.py +++ b/tests/domain/sap10_calculator/worksheet/test_water_heating.py @@ -344,6 +344,34 @@ def test_hot_water_mixer_showers_two_outlets_distributes_shower_count_evenly() - assert t == pytest.approx(s, abs=1e-9), f"month {m+1}" +def test_hot_water_mixer_showers_counts_electric_showers_in_noutlets() -> None: + # Arrange — SAP 10.2 Appendix J step 1a (PDF p.~): "Establish how many + # shower outlets are present in the dwelling, Noutlets (INCLUDING in the + # count any instantaneous electric showers)". The mixer (42a) demand + # divides N_shower by this TOTAL outlet count, so a coexisting electric + # shower reduces each mixer outlet's share. Adding one electric shower to + # a single-mixer dwelling makes Noutlets=2 → the mixer demand halves + # (vs. counting only the mixer outlet, which over-counts main HW and + # under-rates the dwelling). + mixer_only = hot_water_mixer_showers_monthly_l_per_day( + n_occupants=2.0, has_bath=True, + mixer_shower_flow_rates_l_per_min=(8.0,), + cold_water_temps_c=TABLE_J1_TCOLD_FROM_MAINS_C, + ) + + # Act — same single mixer outlet, but one electric shower also present. + with_electric = hot_water_mixer_showers_monthly_l_per_day( + n_occupants=2.0, has_bath=True, + mixer_shower_flow_rates_l_per_min=(8.0,), + cold_water_temps_c=TABLE_J1_TCOLD_FROM_MAINS_C, + n_electric_showers=1, + ) + + # Assert — N_shower / 2 instead of / 1 → exactly half the mixer demand. + for m, (only, both) in enumerate(zip(mixer_only, with_electric)): + assert abs(both - only / 2.0) <= 1e-9, f"month {m+1}" + + @pytest.mark.parametrize("fixture", ALL_FIXTURES, ids=[fixture_id(f) for f in ALL_FIXTURES]) def test_total_hot_water_monthly_matches_elmhurst_line_44(fixture: ModuleType) -> None: """SAP10.2 §4 line (44)m via Appendix J equation J13: From fbe1cb54adb7b7baff5fc8de782f56adb729e09f Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Sat, 13 Jun 2026 23:40:05 +0000 Subject: [PATCH 62/87] test(epc): end-to-end SAP-accuracy gauge over the RdSAP-21.0.1 corpus MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds a committed integration test driving the full API path — raw gov-EPC response → from_api_response → cert_to_inputs → calculate_sap_from_inputs — across all 1000 certs in the in-repo RdSAP-21.0.1 corpus, and pins the aggregate accuracy of our continuous SAP (plus CO2 and primary energy) against each cert's lodged figures. Mirrors scripts/eval_api_sap_accuracy.py but runs in CI off the committed corpus (~2s, no /tmp sample needed). Scoped to RdSAP-21.0.1 — the SAP 10.2-era schema whose lodged rating uses the same methodology we compute (a fair target). Pre-SAP10 schemas (17.x-20.0.0) lodge SAP 2012 ratings and are out of scope (guarded for mapping only by test_mapper_corpus.py). Current: SAP within-0.5 = 65.0%, MAE = 1.174 (tight floor/ceiling — the optimised gauge). CO2 MAE = 0.27 t/yr (bias +0.17) and PE MAE = 14.6 kWh/m2/yr (bias +8.9) are reported + loosely guarded: cost is well-calibrated but CO2/PE both run ~+5-10% high (uniform across fuels — a systematic CO2/PE-factor or scope gap, not yet investigated). Thresholds ratchet as slices tighten each metric. Co-Authored-By: Claude Opus 4.8 --- .../epc_client/test_sap_accuracy_corpus.py | 139 ++++++++++++++++++ 1 file changed, 139 insertions(+) create mode 100644 tests/infrastructure/epc_client/test_sap_accuracy_corpus.py diff --git a/tests/infrastructure/epc_client/test_sap_accuracy_corpus.py b/tests/infrastructure/epc_client/test_sap_accuracy_corpus.py new file mode 100644 index 00000000..8fa8b382 --- /dev/null +++ b/tests/infrastructure/epc_client/test_sap_accuracy_corpus.py @@ -0,0 +1,139 @@ +"""End-to-end SAP-accuracy gauge over the committed RdSAP-21.0.1 corpus. + +Drives the full API path — raw gov-EPC response → ``from_api_response`` → +``cert_to_inputs`` → ``calculate_sap_from_inputs`` — across all 1000 certs in +``backend/epc_api/json_samples/RdSAP-Schema-21.0.1/corpus.jsonl`` and pins the +aggregate accuracy of our continuous SAP (and CO2 / PE) against each cert's +lodged figures. This is the committed regression guard for the headline +"% within 0.5 SAP of the lodged rating" gauge that the per-cert mapper work +optimises (mirrors scripts/eval_api_sap_accuracy.py, but on the in-repo +corpus so it runs in CI without the /tmp sample). + +SCOPE — RdSAP-21.0.1 ONLY. It is the RdSAP 10 / SAP 10.2-era schema, so its +lodged ``energy_rating_current`` was produced by the same SAP methodology we +compute, making it a fair accuracy target. The pre-SAP10 schemas (17.x-20.0.0) +lodge SAP 2012 ratings — a different underlying calculation — so they are NOT +expected to match and are excluded here (their mapper coverage is guarded by +test_mapper_corpus.py instead). + +The asserted thresholds are deterministic floors/ceilings over the fixed +corpus: tighten them whenever a slice improves the gauge (ratchet, never +loosen). Run ``pytest -s`` to see the live metrics line. +""" + +from __future__ import annotations + +import json +from pathlib import Path +from typing import Any + +import pytest + +from datatypes.epc.domain.mapper import EpcPropertyDataMapper +from domain.sap10_calculator.calculator import calculate_sap_from_inputs +from domain.sap10_calculator.rdsap.cert_to_inputs import ( + SAP_10_2_SPEC_PRICES, + cert_to_inputs, +) + +_CORPUS = Path( + "backend/epc_api/json_samples/RdSAP-Schema-21.0.1/corpus.jsonl" +) + +# Measured floors/ceilings over the fixed corpus at HEAD (1000 certs, 0 skips). +# Current: SAP within-0.5 = 65.0%, SAP MAE = 1.174. +# CO2 MAE = 0.27 t/yr (signed +0.17 — a systematic over-estimate, see below). +# PE MAE = 14.6 kWh/m2/yr (signed +8.9). +# +# The SAP (cost) gauge is the optimised target — its floor/ceiling are TIGHT. +# CO2 and PE are reported + LOOSELY guarded: cost is well-calibrated but CO2 +# and PE both run ~+5-10% high (a real systematic gap, not yet investigated — +# uniform across fuels, so a CO2/PE-factor or scope issue, NOT the energy or +# cost). Their ceilings catch "got worse", not "isn't perfect". +# RATCHET any of these up when a slice tightens the corresponding metric. +_MIN_WITHIN_HALF_SAP = 0.62 +_MAX_SAP_MAE = 1.25 +_MAX_CO2_MAE_TONNES = 0.35 # t CO2 / yr vs co2_emissions_current +_MAX_PE_PER_M2_MAE = 16.0 # kWh / m2 / yr vs energy_consumption_current + + +def _load_corpus() -> list[dict[str, Any]]: + if not _CORPUS.exists(): + return [] + return [ + json.loads(line) + for line in _CORPUS.read_text().splitlines() + if line.strip() + ] + + +def test_api_path_sap_accuracy_on_rdsap_21_0_1_corpus( + capsys: pytest.CaptureFixture[str], +) -> None: + # Arrange — the full in-repo 21.0.1 corpus. + corpus = _load_corpus() + if not corpus: + pytest.skip(f"no corpus at {_CORPUS}") + + sap_abs_errs: list[float] = [] + co2_signed_errs_t: list[float] = [] # our − lodged, tonnes/yr + pe_signed_errs: list[float] = [] # our − lodged, kWh/m²/yr + skipped = 0 + + # Act — run the API → EpcPropertyData → calculator pipeline per cert. + for doc in corpus: + lodged_sap = doc.get("energy_rating_current") + if lodged_sap is None: + skipped += 1 + continue + try: + epc = EpcPropertyDataMapper.from_api_response(doc) + result = calculate_sap_from_inputs( + cert_to_inputs(epc, prices=SAP_10_2_SPEC_PRICES) + ) + except Exception: + # A mapper / calculator raise is a coverage gap tracked elsewhere + # (eval_api_sap_accuracy.py); here we gauge the certs that compute. + skipped += 1 + continue + + sap_abs_errs.append(abs(result.sap_score_continuous - lodged_sap)) + + lodged_co2_t = doc.get("co2_emissions_current") # tonnes/yr + if lodged_co2_t is not None: + co2_signed_errs_t.append(result.co2_kg_per_yr / 1000.0 - lodged_co2_t) + lodged_pe_per_m2 = doc.get("energy_consumption_current") # kWh/m²/yr (primary) + if lodged_pe_per_m2 is not None: + pe_signed_errs.append(result.primary_energy_kwh_per_m2 - lodged_pe_per_m2) + + n = len(sap_abs_errs) + within_half = sum(1 for e in sap_abs_errs if e < 0.5) / n + sap_mae = sum(sap_abs_errs) / n + co2_mae = sum(abs(e) for e in co2_signed_errs_t) / len(co2_signed_errs_t) + co2_bias = sum(co2_signed_errs_t) / len(co2_signed_errs_t) + pe_mae = sum(abs(e) for e in pe_signed_errs) / len(pe_signed_errs) + pe_bias = sum(pe_signed_errs) / len(pe_signed_errs) + + with capsys.disabled(): + print( + f"\n[RdSAP-21.0.1 corpus | {n} computed / {skipped} skipped]" + f"\n SAP within-0.5 = {within_half:.1%} MAE = {sap_mae:.3f}" + f"\n CO2 MAE = {co2_mae:.2f} t/yr (bias {co2_bias:+.2f} t/yr)" + f"\n PE MAE = {pe_mae:.1f} kWh/m2/yr (bias {pe_bias:+.1f})" + ) + + # Assert — SAP (cost) is the optimised gauge: tight floor/ceiling. CO2/PE + # are loose "don't regress" guards (see module + threshold notes). + assert within_half >= _MIN_WITHIN_HALF_SAP, ( + f"SAP within-0.5 {within_half:.1%} fell below floor " + f"{_MIN_WITHIN_HALF_SAP:.0%}" + ) + assert sap_mae <= _MAX_SAP_MAE, ( + f"SAP MAE {sap_mae:.3f} exceeded ceiling {_MAX_SAP_MAE}" + ) + assert co2_mae <= _MAX_CO2_MAE_TONNES, ( + f"CO2 MAE {co2_mae:.2f} t/yr exceeded ceiling {_MAX_CO2_MAE_TONNES}" + ) + assert pe_mae <= _MAX_PE_PER_M2_MAE, ( + f"PE MAE {pe_mae:.1f} kWh/m2/yr exceeded ceiling {_MAX_PE_PER_M2_MAE}" + ) From dfcd7af57cb01fde318da7a3c4a19c433e421846 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Sun, 14 Jun 2026 01:54:51 +0000 Subject: [PATCH 63/87] fix(heat-network): apply Table 4c(3) flat-rate charging factor to demand SAP 10.2 Table 4c(3) (PDF p.169) "Factor for controls and charging method" multiplies a heat network's heat requirement by 1.05-1.10 for FLAT-RATE charging (note d: household pays a fixed amount regardless of heat used, so no incentive to economise), and by 1.0 for charging linked to use. The worksheet folds it into the heat-network requirement alongside the Table 12c distribution loss factor: (307) space = (98c) x (302) x (305) x (306) (310) DHW = (64) x (305a) x (306) Our cascade applied (306) DLF but never (305)/(305a), so every flat-rate community-heating cert under-counted demand -> over-rated SAP. Folded the factor into the 1/DLF efficiency override at the space-heating (206) and DHW (water-inherits-from-main) sites. Space column adds +0.05 for no thermostatic control (2301/2302); DHW column is 1.05 flat-rate / 1.0 linked-to-use. Corpus (RdSAP-21.0.1, 1000 certs): community cluster median +0.32 -> -0.19, within-0.5 38% -> 62% (control 2307 +0.83 -> -0.19; 2306 unchanged at factor 1.0 as spec requires). Overall gauge 65.0% -> 65.9%, MAE 1.174 -> 1.160. Ratcheted the corpus-test floor 0.62 -> 0.63 / MAE ceiling 1.25 -> 1.22. Also records (corpus-test comment + scripts/decompose_co2_pe_error.py) the disproof of the prior "CO2/PE +5% is a factor/scope bug" lead: factors are spec-exact, scope identical, and the bias is per-cert demand fidelity (corr(SAP-err, PE-diff) = -0.54), not a one-slice factor fix. Co-Authored-By: Claude Opus 4.8 --- .../sap10_calculator/rdsap/cert_to_inputs.py | 67 +++++++++- scripts/decompose_co2_pe_error.py | 124 ++++++++++++++++++ .../rdsap/test_cert_to_inputs.py | 64 +++++++++ .../epc_client/test_sap_accuracy_corpus.py | 32 +++-- 4 files changed, 274 insertions(+), 13 deletions(-) create mode 100644 scripts/decompose_co2_pe_error.py diff --git a/domain/sap10_calculator/rdsap/cert_to_inputs.py b/domain/sap10_calculator/rdsap/cert_to_inputs.py index d293c193..0851e92b 100644 --- a/domain/sap10_calculator/rdsap/cert_to_inputs.py +++ b/domain/sap10_calculator/rdsap/cert_to_inputs.py @@ -1070,6 +1070,56 @@ def _heat_network_dlf(age_band: Optional[str]) -> float: raise UnmappedSapCode("heat_network_age_band", age_band) +# SAP 10.2 Table 4c(3) (PDF p.169) — "Factor for controls and charging +# method" for HEAT NETWORKS, by Table 4e control code. The factor multiplies +# the heat-network heat requirement on top of the Table 12c distribution loss +# factor: worksheet (307) space = (98c) × (302) × (305) × (306), and +# (310) DHW = (64) × (305a) × (306). "Flat rate charging" (note d: the +# household pays a fixed amount regardless of heat used) carries a demand +# penalty — there is no incentive to economise; "charging linked to use of +# heat" does not. The SPACE column adds a further +0.05 when there is also +# no thermostatic room-temperature control (2301/2302). +_HEAT_NETWORK_SPACE_CHARGING_FACTOR_BY_CODE: Final[dict[int, float]] = { + # Flat rate charging, no thermostatic room control → 1.10 + 2301: 1.10, 2302: 1.10, + # Flat rate charging, with some thermostatic control → 1.05 + 2303: 1.05, 2304: 1.05, 2305: 1.05, 2307: 1.05, 2311: 1.05, 2313: 1.05, + # Charging linked to use of heat, thermostat but no TRVs → 1.05 + 2308: 1.05, 2309: 1.05, + # Charging linked to use of heat, with TRVs → 1.00 + 2306: 1.00, 2310: 1.00, 2312: 1.00, 2314: 1.00, +} +_HEAT_NETWORK_DHW_CHARGING_FACTOR_BY_CODE: Final[dict[int, float]] = { + # Flat rate charging → 1.05 (all controls) + 2301: 1.05, 2302: 1.05, 2303: 1.05, 2304: 1.05, + 2305: 1.05, 2307: 1.05, 2311: 1.05, 2313: 1.05, + # Charging linked to use of heat → 1.00 + 2306: 1.00, 2308: 1.00, 2309: 1.00, 2310: 1.00, 2312: 1.00, 2314: 1.00, +} + + +def _heat_network_space_charging_factor(main: Optional[MainHeatingDetail]) -> float: + """SAP 10.2 Table 4c(3) (PDF p.169) worksheet (305) — heat-network + space-heating factor for controls and charging method. Returns 1.0 when + the control is absent (no penalty); every Table 4e Group-3 code + (2301-2314) is covered, so an unmapped key only arises off the + heat-network path and is harmless at 1.0.""" + code = main.main_heating_control if main is not None else None + if not isinstance(code, int): + return 1.0 + return _HEAT_NETWORK_SPACE_CHARGING_FACTOR_BY_CODE.get(code, 1.0) + + +def _heat_network_dhw_charging_factor(main: Optional[MainHeatingDetail]) -> float: + """SAP 10.2 Table 4c(3) (PDF p.169) worksheet (305a) — heat-network + water-heating factor for charging method (flat rate → 1.05, linked to + use → 1.0). Returns 1.0 when the control is absent.""" + code = main.main_heating_control if main is not None else None + if not isinstance(code, int): + return 1.0 + return _HEAT_NETWORK_DHW_CHARGING_FACTOR_BY_CODE.get(code, 1.0) + + def _dwelling_age_band(epc: EpcPropertyData) -> Optional[str]: """The dwelling's construction age band, read from the first building part that lodges one. @@ -1822,7 +1872,13 @@ def _main_heating_detail_efficiency( eff = seasonal_efficiency(main_code, main_category, main_fuel) if _is_heat_network_main(main): primary_age = _dwelling_age_band(epc) - eff = 1.0 / _heat_network_dlf(primary_age) + # Worksheet (307): heat required = demand × (305) × (306 DLF), so the + # delivered-per-fuel efficiency carries 1 / ((305) charging factor × + # DLF). The Table 4c(3) flat-rate charging penalty raises demand. + eff = 1.0 / ( + _heat_network_dlf(primary_age) + * _heat_network_space_charging_factor(main) + ) return eff @@ -7074,8 +7130,13 @@ def cert_to_inputs( # HW from main on a heat-network cert: the DHW also incurs the # network's distribution losses. Same 1/DLF override as for # space heating so the delivered HW kWh reflects q_useful × DLF - # = q_generated, matching the per-kWh-generated unit price. - water_eff = 1.0 / _heat_network_dlf(primary_age) + # = q_generated, matching the per-kWh-generated unit price. Worksheet + # (310): heat required = (64) × (305a) × (306 DLF), so the DHW + # efficiency also carries 1 / Table 4c(3) (305a) charging factor. + water_eff = 1.0 / ( + _heat_network_dlf(primary_age) + * _heat_network_dhw_charging_factor(main) + ) elif epc.sap_heating.water_heating_code in _WATER_HEAT_NETWORK_ONLY_CODES: # HW-only heat network (whc 950/951/952): the Table 4a plant # efficiency is already in `water_eff`; apply the Table 12c diff --git a/scripts/decompose_co2_pe_error.py b/scripts/decompose_co2_pe_error.py new file mode 100644 index 00000000..a025a738 --- /dev/null +++ b/scripts/decompose_co2_pe_error.py @@ -0,0 +1,124 @@ +"""Decompose the API-path CO2/PE over-estimate against lodged EPC figures. + +The corpus integration test surfaced a systematic +5-10% over-estimate on +CO2 and PE while cost/SAP is well-calibrated. This script profiles the +structure of that bias: multiplicative vs additive, per-component shares, +and segmentation by fuel / PV / heating type — to localise the cause. +""" +from __future__ import annotations + +import json +import statistics as stats +from collections import defaultdict +from pathlib import Path +from typing import Any + +from datatypes.epc.domain.mapper import EpcPropertyDataMapper +from domain.sap10_calculator.calculator import calculate_sap_from_inputs +from domain.sap10_calculator.rdsap.cert_to_inputs import ( + SAP_10_2_SPEC_PRICES, + cert_to_inputs, +) + +CORPUS = Path("backend/epc_api/json_samples/RdSAP-Schema-21.0.1/corpus.jsonl") + + +def main() -> None: + docs = [ + json.loads(line) + for line in CORPUS.read_text().splitlines() + if line.strip() + ] + rows: list[dict[str, Any]] = [] + for doc in docs: + lodged_sap = doc.get("energy_rating_current") + lodged_co2 = doc.get("co2_emissions_current") # t/yr + lodged_pe = doc.get("energy_consumption_current") # kWh/m2/yr + if lodged_pe is None or lodged_co2 is None: + continue + try: + epc = EpcPropertyDataMapper.from_api_response(doc) + res = calculate_sap_from_inputs( + cert_to_inputs(epc, prices=SAP_10_2_SPEC_PRICES) + ) + except Exception: + continue + tfa = res.intermediate["tfa_m2"] + im = res.intermediate + rows.append({ + "cert": doc.get("rrn") or "", + "sap_err": res.sap_score_continuous - (lodged_sap or 0), + "our_pe": res.primary_energy_kwh_per_m2, + "lodged_pe": lodged_pe, + "pe_diff": res.primary_energy_kwh_per_m2 - lodged_pe, + "pe_ratio": res.primary_energy_kwh_per_m2 / lodged_pe if lodged_pe else 0, + "our_co2": res.co2_kg_per_yr / 1000.0, + "lodged_co2": lodged_co2, + "co2_diff": res.co2_kg_per_yr / 1000.0 - lodged_co2, + "co2_ratio": (res.co2_kg_per_yr / 1000.0) / lodged_co2 if lodged_co2 else 0, + # PE components per m2 + "space_pe": im["space_heating_pe_kwh_per_m2"], + "hw_pe": im["hot_water_pe_kwh_per_m2"], + "other_pe": im["other_pe_kwh_per_m2"], + "pv_pe": im["pv_pe_offset_kwh_per_m2"], + "mains_gas": (doc.get("sap_energy_source") or {}).get("mains_gas"), + "has_pv": bool((doc.get("sap_energy_source") or {}).get("photovoltaic_supply")), + "tfa": tfa, + }) + + n = len(rows) + print(f"decomposed {n} certs\n" + "=" * 70) + + def summ(key: str) -> str: + vals = [r[key] for r in rows] + return (f"median={stats.median(vals):+.3f} mean={stats.mean(vals):+.3f} " + f"p10={sorted(vals)[n//10]:+.3f} p90={sorted(vals)[n*9//10]:+.3f}") + + print(f"PE diff (kWh/m2): {summ('pe_diff')}") + print(f"PE ratio (our/lod): {summ('pe_ratio')}") + print(f"CO2 diff (t/yr) : {summ('co2_diff')}") + print(f"CO2 ratio (our/lod): {summ('co2_ratio')}") + print("=" * 70) + + # multiplicative vs additive: correlate pe_diff with lodged_pe magnitude + lod = [r["lodged_pe"] for r in rows] + dif = [r["pe_diff"] for r in rows] + mean_lod, mean_dif = stats.mean(lod), stats.mean(dif) + cov = sum((l - mean_lod) * (d - mean_dif) for l, d in zip(lod, dif)) / n + var = sum((l - mean_lod) ** 2 for l in lod) / n + slope = cov / var + print(f"PE: regress pe_diff ~ lodged_pe -> slope={slope:+.4f} intercept={mean_dif - slope*mean_lod:+.3f}") + print(" (slope~0 => additive constant; slope>0 => multiplicative)") + print("=" * 70) + + # segment + def seg(name: str, pred: Any) -> None: + s = [r for r in rows if pred(r)] + if not s: + return + print(f" {name:28s} n={len(s):4d} PEdiff_med={stats.median(r['pe_diff'] for r in s):+6.2f} " + f"PEratio_med={stats.median(r['pe_ratio'] for r in s):.3f} " + f"CO2diff_med={stats.median(r['co2_diff'] for r in s):+.3f} " + f"SAPerr_med={stats.median(r['sap_err'] for r in s):+.2f}") + + print("SEGMENTS:") + seg("mains_gas=Y", lambda r: r["mains_gas"] in (True, 1, "Y")) + seg("mains_gas=N", lambda r: r["mains_gas"] not in (True, 1, "Y")) + seg("ALL", lambda r: True) + print("=" * 70) + print("PE COMPONENT MEANS (kWh/m2):") + for k in ("space_pe", "hw_pe", "other_pe", "pv_pe", "our_pe", "lodged_pe"): + print(f" {k:12s} mean={stats.mean(r[k] for r in rows):7.2f} " + f"median={stats.median(r[k] for r in rows):7.2f}") + print("=" * 70) + # Well-calibrated-SAP certs: energy is right, so PE diff isolates factor/scope. + cal = [r for r in rows if abs(r["sap_err"]) < 0.3] + print(f"WELL-CALIBRATED-SAP (|sap_err|<0.3) n={len(cal)}:") + print(f" PE diff median={stats.median(r['pe_diff'] for r in cal):+.2f} " + f"ratio={stats.median(r['pe_ratio'] for r in cal):.3f}") + print(f" CO2 diff median={stats.median(r['co2_diff'] for r in cal):+.3f} " + f"ratio={stats.median(r['co2_ratio'] for r in cal):.3f}") + + +if __name__ == "__main__": + main() diff --git a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py index ed9d43ab..326034d0 100644 --- a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py +++ b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py @@ -277,6 +277,70 @@ def test_heat_network_main_applies_table12c_dlf_to_main_heating_efficiency() -> assert inputs.main_heating_efficiency == pytest.approx(1.0 / 1.41, abs=0.005) +def test_heat_network_flat_rate_charging_applies_table_4c3_space_factor() -> None: + # Arrange — community heat-network main (Table 4a code 301, cat 6), age + # band E (Table 12c DLF = 1.41). Control 2307 = "Flat rate charging, + # TRVs". SAP 10.2 Table 4c(3) (PDF p.169) multiplies the heat-network + # heat requirement by 1.05 for flat-rate charging (worksheet (305) → + # (307) = (98c) × (302) × (305) × (306)). Flat-rate billing gives no + # incentive to economise, so demand rises. The factor folds into the + # delivered-per-fuel efficiency alongside the DLF: 1 / (DLF × 1.05). + main = MainHeatingDetail( + has_fghrs=False, + main_fuel_type=20, # mains gas (community) + heat_emitter_type=1, + emitter_temperature=1, + main_heating_control=2307, # Flat rate charging, TRVs + main_heating_category=6, + sap_main_heating_code=301, + ) + part = make_building_part(construction_age_band="E") + epc = make_minimal_sap10_epc( + total_floor_area_m2=_TYPICAL_TFA_M2, + habitable_rooms_count=4, + country_code="ENG", + sap_building_parts=[part], + sap_heating=make_sap_heating(main_heating_details=[main]), + ) + + # Act + inputs = cert_to_inputs(epc) + + # Assert — efficiency = 1 / (DLF 1.41 × Table 4c(3) factor 1.05). + assert abs(inputs.main_heating_efficiency - 1.0 / (1.41 * 1.05)) <= 1e-9 + + +def test_heat_network_charging_linked_to_use_has_no_table_4c3_penalty() -> None: + # Arrange — same community heat-network main, but control 2306 = + # "Charging system linked to use of heating, programmer and TRVs". SAP + # 10.2 Table 4c(3) (PDF p.169) gives factor 1.0 (charges track usage, so + # no demand penalty), leaving the efficiency at the bare 1/DLF — i.e. the + # 2307 flat-rate penalty above must NOT fire here. + main = MainHeatingDetail( + has_fghrs=False, + main_fuel_type=20, + heat_emitter_type=1, + emitter_temperature=1, + main_heating_control=2306, # Charging linked to use, programmer + TRVs + main_heating_category=6, + sap_main_heating_code=301, + ) + part = make_building_part(construction_age_band="E") + epc = make_minimal_sap10_epc( + total_floor_area_m2=_TYPICAL_TFA_M2, + habitable_rooms_count=4, + country_code="ENG", + sap_building_parts=[part], + sap_heating=make_sap_heating(main_heating_details=[main]), + ) + + # Act + inputs = cert_to_inputs(epc) + + # Assert — efficiency = 1 / DLF 1.41, no Table 4c(3) multiplier. + assert abs(inputs.main_heating_efficiency - 1.0 / 1.41) <= 1e-9 + + def test_heat_network_dlf_uses_first_non_empty_building_part_age_band() -> None: # Arrange — the GOV.UK API lodges a junk empty leading building part # (all fields absent) before the real Main Dwelling. The dwelling age diff --git a/tests/infrastructure/epc_client/test_sap_accuracy_corpus.py b/tests/infrastructure/epc_client/test_sap_accuracy_corpus.py index 8fa8b382..27bdcd52 100644 --- a/tests/infrastructure/epc_client/test_sap_accuracy_corpus.py +++ b/tests/infrastructure/epc_client/test_sap_accuracy_corpus.py @@ -41,18 +41,30 @@ _CORPUS = Path( ) # Measured floors/ceilings over the fixed corpus at HEAD (1000 certs, 0 skips). -# Current: SAP within-0.5 = 65.0%, SAP MAE = 1.174. -# CO2 MAE = 0.27 t/yr (signed +0.17 — a systematic over-estimate, see below). -# PE MAE = 14.6 kWh/m2/yr (signed +8.9). +# Current: SAP within-0.5 = 65.9%, SAP MAE = 1.160 (heat-network Table 4c(3) +# flat-rate charging factor, this slice: community cluster 38% -> 62% within-0.5). +# CO2 MAE = 0.28 t/yr (signed +0.17 — a systematic over-estimate, see below). +# PE MAE = 14.7 kWh/m2/yr (signed +9.2). # # The SAP (cost) gauge is the optimised target — its floor/ceiling are TIGHT. -# CO2 and PE are reported + LOOSELY guarded: cost is well-calibrated but CO2 -# and PE both run ~+5-10% high (a real systematic gap, not yet investigated — -# uniform across fuels, so a CO2/PE-factor or scope issue, NOT the energy or -# cost). Their ceilings catch "got worse", not "isn't perfect". -# RATCHET any of these up when a slice tightens the corresponding metric. -_MIN_WITHIN_HALF_SAP = 0.62 -_MAX_SAP_MAE = 1.25 +# CO2 and PE are reported + LOOSELY guarded: both run ~+5% high vs the lodged +# figures. INVESTIGATED (see scripts/decompose_co2_pe_error.py): this is NOT a +# CO2/PE-factor or scope bug. Table 12 factors are spec-exact (SAP 10.2 p.188: +# mains gas PEF 1.130 / CO2 0.210, electricity 1.501 / 0.136) and worksheet +# (286)/(272) scope is identical to ours. Two real causes: +# 1. Per-cert mapper/demand fidelity. corr(SAP-err, PE-diff) = -0.54: when we +# over-estimate demand the cert under-rates on SAP AND over-estimates PE. +# Golden cert 0300-2747 (matched data) hits register PE to -0.10 kWh/m2, +# proving the engine is correct — the corpus bias is the aggregate of the +# same unclosed mapper gaps the SAP gauge chases, seen through a more +# sensitive (linear, no standing-charge/deflator damping) lens. +# 2. SAP cost is genuinely calibrated, NOT energy hiding behind it: a +5% gas +# space-heating bump moves SAP -0.6 (would show as a -0.6 corpus bias if +# energy were 5% high; actual SAP bias is +0.145). +# So closing demand over-estimates lifts BOTH the SAP gauge and PE/CO2; there is +# no one-slice factor fix. RATCHET any ceiling up when a slice tightens it. +_MIN_WITHIN_HALF_SAP = 0.63 +_MAX_SAP_MAE = 1.22 _MAX_CO2_MAE_TONNES = 0.35 # t CO2 / yr vs co2_emissions_current _MAX_PE_PER_M2_MAE = 16.0 # kWh / m2 / yr vs energy_consumption_current From bec62b91677c10f4390012c353b42df1f980ee55 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Sun, 14 Jun 2026 02:12:39 +0000 Subject: [PATCH 64/87] fix(storage-heaters): Table 12a code-408 integrated-storage high-rate fraction MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit SAP 10.2 Table 12a Grid 1 (PDF p.191): electric storage heater SAP code 408 is an "Integrated (storage + direct-acting) system" with a 0.20 space-heating high-rate fraction on a 7-hour tariff — NOT the 0.00 of "other storage heaters". `_table_12a_system_for_main` returned None for all storage codes (an explicit TODO), so code 408 fell back to the 100%-low-rate path and billed space heating at the bare 7-hour low rate (5.50 p/kWh) — under-costing → over-rating. Mapped cat-7 storage: 408 -> INTEGRATED_STORAGE_DIRECT (0.20), others -> OTHER_STORAGE_HEATERS (0.00, unchanged behaviour). The enum + fraction rows already existed; this only wires the dispatch, so the split flows self-consistently to both the §10a cost and the Appendix-M1 D_PV high-rate fraction. Corpus: sap408 over-raters +14.6/+12.9/+12.7 -> +7.1/+5.1/+3.4 (two crossed into within-0.5). Gauge 65.9% -> 66.1%, MAE 1.160 -> 1.128. Floor 0.63 -> 0.64 / MAE ceiling 1.22 -> 1.18. Worksheet harness 47/47 0 diverge. The residual +3..+7 is the "all other uses" 0.90 high-rate fraction (lighting/pumps/HW still billed 100%-low on the off-peak legacy path) — the next slice. Co-Authored-By: Claude Opus 4.8 --- .../sap10_calculator/rdsap/cert_to_inputs.py | 13 +++- .../rdsap/test_cert_to_inputs.py | 72 +++++++++++++++++++ .../epc_client/test_sap_accuracy_corpus.py | 12 ++-- 3 files changed, 91 insertions(+), 6 deletions(-) diff --git a/domain/sap10_calculator/rdsap/cert_to_inputs.py b/domain/sap10_calculator/rdsap/cert_to_inputs.py index 0851e92b..d35be43b 100644 --- a/domain/sap10_calculator/rdsap/cert_to_inputs.py +++ b/domain/sap10_calculator/rdsap/cert_to_inputs.py @@ -2408,7 +2408,8 @@ def _table_12a_system_for_main( Coverage as fixtures land: - ASHP / GSHP (codes 211-224, 521-524, PCDB index) — wired - - Storage heaters (401-409) — TODO + - Storage heaters (cat 7): 408 → INTEGRATED_STORAGE_DIRECT (0.20), + all others → OTHER_STORAGE_HEATERS (0.00) — wired - Underfloor heating (421-422) — TODO - Direct-acting electric (191) / CPSU (192) / electric storage boiler (193, 195) — TODO @@ -2451,6 +2452,16 @@ def _table_12a_system_for_main( 211 <= code <= 217 or 221 <= code <= 227 or 521 <= code <= 524 ): return Table12aSystem.ASHP_OTHER + # Electric STORAGE heaters (RdSAP main_heating_category 7) — SAP 10.2 + # Table 12a Grid 1 (PDF p.191). Code 408 is an "Integrated (storage + + # direct-acting) system" → 0.20 SH high-rate fraction at 7-hour; every + # other storage code is "Other storage heaters" → 0.00 (charged wholly + # off-peak, the same 100%-low-rate the None fallback already gave). + # Gated on `_is_electric_main` belt-and-braces (all callers pre-gate). + if main.main_heating_category == 7 and _is_electric_main(main): + if code == 408: + return Table12aSystem.INTEGRATED_STORAGE_DIRECT + return Table12aSystem.OTHER_STORAGE_HEATERS return None diff --git a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py index 326034d0..9d7971cf 100644 --- a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py +++ b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py @@ -624,6 +624,78 @@ def test_cylinder_size_inaccessible_code_5_off_peak_dual_immersion_uses_210l() - assert volume_l is not None and abs(volume_l - 210.0) <= 1e-9 +def test_integrated_storage_heater_408_bills_table_12a_grid1_high_rate_fraction() -> None: + # Arrange — electric storage heaters (cat 7), SAP code 408, on an + # off-peak 7-hour (dual / Economy-7) meter. SAP 10.2 Table 12a Grid 1 + # (PDF p.191): code 408 is an "Integrated (storage + direct-acting) + # system" with a 0.20 space-heating high-rate fraction at 7-hour (NOT + # the 0.00 of "other storage heaters"). The scalar SH rate is therefore + # the blend 0.20 × high (15.29) + 0.80 × low (5.50) = 7.458 p/kWh. + from dataclasses import replace + + main = MainHeatingDetail( + has_fghrs=False, + main_fuel_type=29, # electricity + heat_emitter_type=1, + emitter_temperature=1, + main_heating_control=2401, + main_heating_category=7, # electric storage heaters + sap_main_heating_code=408, + ) + base = make_minimal_sap10_epc( + total_floor_area_m2=_TYPICAL_TFA_M2, + habitable_rooms_count=4, + country_code="ENG", + sap_building_parts=[make_building_part(construction_age_band="E")], + sap_heating=make_sap_heating(main_heating_details=[main]), + ) + epc = replace( + base, + sap_energy_source=replace(base.sap_energy_source, meter_type="1"), + ) + + # Act + inputs = cert_to_inputs(epc) + + # Assert — blended scalar rate 0.20×15.29 + 0.80×5.50 = 7.458 p/kWh. + assert abs(inputs.space_heating_fuel_cost_gbp_per_kwh - 0.07458) <= 1e-9 + + +def test_non_integrated_storage_heater_bills_100_percent_low_rate() -> None: + # Arrange — same off-peak storage cert but SAP code 401 ("other storage + # heaters"): Table 12a Grid 1 gives a 0.00 high-rate fraction → the heat + # is charged wholly at the 7-hour low rate (5.50 p/kWh). Guards that the + # 408 integrated-system 0.20 fraction does NOT leak to other codes. + from dataclasses import replace + + main = MainHeatingDetail( + has_fghrs=False, + main_fuel_type=29, + heat_emitter_type=1, + emitter_temperature=1, + main_heating_control=2401, + main_heating_category=7, + sap_main_heating_code=401, + ) + base = make_minimal_sap10_epc( + total_floor_area_m2=_TYPICAL_TFA_M2, + habitable_rooms_count=4, + country_code="ENG", + sap_building_parts=[make_building_part(construction_age_band="E")], + sap_heating=make_sap_heating(main_heating_details=[main]), + ) + epc = replace( + base, + sap_energy_source=replace(base.sap_energy_source, meter_type="1"), + ) + + # Act + inputs = cert_to_inputs(epc) + + # Assert — 100% low rate, 5.50 p/kWh. + assert abs(inputs.space_heating_fuel_cost_gbp_per_kwh - 0.0550) <= 1e-9 + + def test_heat_network_distribution_electricity_per_sap_10_2_appendix_c_3_2() -> None: # Arrange — heat-network main (Table 4a code 301 = community heating, # category 6). SAP 10.2 Appendix C §C3.2 (PDF p.51): distribution diff --git a/tests/infrastructure/epc_client/test_sap_accuracy_corpus.py b/tests/infrastructure/epc_client/test_sap_accuracy_corpus.py index 27bdcd52..e6c3be1e 100644 --- a/tests/infrastructure/epc_client/test_sap_accuracy_corpus.py +++ b/tests/infrastructure/epc_client/test_sap_accuracy_corpus.py @@ -41,10 +41,12 @@ _CORPUS = Path( ) # Measured floors/ceilings over the fixed corpus at HEAD (1000 certs, 0 skips). -# Current: SAP within-0.5 = 65.9%, SAP MAE = 1.160 (heat-network Table 4c(3) -# flat-rate charging factor, this slice: community cluster 38% -> 62% within-0.5). +# Current: SAP within-0.5 = 66.1%, SAP MAE = 1.128 (Table 12a Grid 1 +# integrated-storage code-408 0.20 high-rate fraction, this slice: sap408 +# over-rate +14.6/+12.9/+12.7 -> +7.1/+5.1/+3.4; prior slice was the +# heat-network Table 4c(3) flat-rate charging factor). # CO2 MAE = 0.28 t/yr (signed +0.17 — a systematic over-estimate, see below). -# PE MAE = 14.7 kWh/m2/yr (signed +9.2). +# PE MAE = 14.7 kWh/m2/yr (signed +9.1). # # The SAP (cost) gauge is the optimised target — its floor/ceiling are TIGHT. # CO2 and PE are reported + LOOSELY guarded: both run ~+5% high vs the lodged @@ -63,8 +65,8 @@ _CORPUS = Path( # energy were 5% high; actual SAP bias is +0.145). # So closing demand over-estimates lifts BOTH the SAP gauge and PE/CO2; there is # no one-slice factor fix. RATCHET any ceiling up when a slice tightens it. -_MIN_WITHIN_HALF_SAP = 0.63 -_MAX_SAP_MAE = 1.22 +_MIN_WITHIN_HALF_SAP = 0.64 +_MAX_SAP_MAE = 1.18 _MAX_CO2_MAE_TONNES = 0.35 # t CO2 / yr vs co2_emissions_current _MAX_PE_PER_M2_MAE = 16.0 # kWh / m2 / yr vs energy_consumption_current From 94275d07cc422fe5b6963f9f28b781e1f780cea9 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Sun, 14 Jun 2026 08:20:34 +0000 Subject: [PATCH 65/87] fix(hot-water): default present-but-unsized cylinder to Table 28 Normal 110 L MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit RdSAP 10 §10.5 (PDF p.55): "If the actual size is not determined, the size of a hot-water cylinder is taken as according to Table 28." When a cylinder is present (has_hot_water_cylinder) but no size descriptor resolves — the gov API lodges cylinder_size=0, or Exact with no measured volume — `_hot_water_ cylinder_volume_l` returned None, silently dropping BOTH the cylinder's storage loss and the Table 13 electric-DHW high-rate fraction, under-costing and over-rating the dwelling. Default such cylinders to the Table 28 baseline "Normal" 110 L (the value §10.7 also instantiates as the first-row default). The context-dependent Inaccessible 210/160 values are deliberately NOT applied here — they are tied to the explicit "Inaccessible" descriptor (code 5) the assessor lodges, not to an unpopulated size field. Scope: 7 of 301 cylinder certs in the corpus (2%). Correctness fix — closes a real spec gap; marginal on the headline (within-0.5 66.1% unchanged, MAE 1.128 -> 1.124) because these certs' residual is dominated by a separate HW- demand gap, not the cylinder. Worksheet harness 47/47 0 diverge (Summary certs lodge a real size, so the fallback never fires). 1 AAA test, pyright net-zero. Co-Authored-By: Claude Opus 4.8 --- .../sap10_calculator/rdsap/cert_to_inputs.py | 25 ++++++++++++++++--- .../rdsap/test_cert_to_inputs.py | 19 ++++++++++++++ .../epc_client/test_sap_accuracy_corpus.py | 9 ++++--- 3 files changed, 46 insertions(+), 7 deletions(-) diff --git a/domain/sap10_calculator/rdsap/cert_to_inputs.py b/domain/sap10_calculator/rdsap/cert_to_inputs.py index d35be43b..9c6cc278 100644 --- a/domain/sap10_calculator/rdsap/cert_to_inputs.py +++ b/domain/sap10_calculator/rdsap/cert_to_inputs.py @@ -5223,6 +5223,15 @@ _CYLINDER_SIZE_EXACT: Final[int] = 6 _CYLINDER_INACCESSIBLE_DUAL_IMMERSION_L: Final[float] = 210.0 _CYLINDER_INACCESSIBLE_SOLID_FUEL_L: Final[float] = 160.0 _CYLINDER_INACCESSIBLE_DEFAULT_L: Final[float] = 110.0 +# RdSAP 10 §10.5 (PDF p.55): "If the actual size is not determined, the size +# of a hot-water cylinder is taken as according to Table 28." For a cylinder +# present but with no size descriptor lodged (size code 0 / absent), the +# baseline Table 28 default is the "Normal" row (110 L) — the same value +# §10.7 instantiates as the first-row default. The context-dependent +# Inaccessible 210/160 values are NOT applied here: they are tied to the +# explicit "Inaccessible" descriptor (code 5) the assessor lodges +# deliberately, not to a merely-unpopulated size field. +_CYLINDER_SIZE_NOT_DETERMINED_L: Final[float] = 110.0 def _cylinder_inaccessible_volume_l(epc: EpcPropertyData) -> float: @@ -6163,11 +6172,21 @@ def _hot_water_cylinder_volume_l(epc: EpcPropertyData) -> Optional[float]: """Resolve the HW cylinder volume (litres) from the cert's `cylinder_size` code via RdSAP 10 §10.5 Table 28 — Normal/Medium/Large (codes 2/3/4), Inaccessible (5, context-dependent) and Exact (6, lodged - measured volume). Returns None when no cylinder is lodged or no size - code resolves.""" + measured volume). Returns None only when no cylinder is lodged. + + RdSAP 10 §10.5 (PDF p.55): "If the actual size is not determined, the + size of a hot-water cylinder is taken as according to Table 28." When a + cylinder IS present but no size descriptor resolves (size code 0 / + absent, or Exact with no measured volume), fall back to the Table 28 + baseline "Normal" default (110 L). Without this the cylinder resolved + to None, silently dropping BOTH its storage loss and the Table 13 + high-rate fraction, over-rating unsized-cylinder electric dwellings.""" if not epc.has_hot_water_cylinder: return None - return _cylinder_volume_l_from_code(epc) + volume_l = _cylinder_volume_l_from_code(epc) + if volume_l is not None: + return volume_l + return _CYLINDER_SIZE_NOT_DETERMINED_L def _immersion_is_single(epc: EpcPropertyData) -> Optional[bool]: diff --git a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py index 9d7971cf..a862c238 100644 --- a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py +++ b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py @@ -561,6 +561,25 @@ def test_cylinder_size_exact_code_6_uses_lodged_measured_volume() -> None: assert volume_l is not None and abs(volume_l - 150.0) <= 1e-9 +def test_cylinder_present_but_size_not_determined_defaults_to_normal_110l() -> None: + # Arrange — RdSAP 10 §10.5 (PDF p.55): "If the actual size is not + # determined, the size of a hot-water cylinder is taken as according to + # Table 28." A cylinder IS present but no size descriptor resolves (gov + # API lodges `cylinder_size=0`) → the Table 28 baseline "Normal" default + # of 110 L, NOT None (which silently dropped the cylinder's storage loss + # AND the Table 13 high-rate fraction, over-rating the dwelling). + from domain.sap10_calculator.rdsap.cert_to_inputs import ( + _hot_water_cylinder_volume_l, # pyright: ignore[reportPrivateUsage] + ) + epc = _cylinder_epc(cylinder_size=0) + + # Act + volume_l = _hot_water_cylinder_volume_l(epc) + + # Assert + assert volume_l is not None and abs(volume_l - 110.0) <= 1e-9 + + def test_cylinder_size_inaccessible_code_5_solid_fuel_boiler_uses_160l() -> None: # Arrange — RdSAP 10 §10.5 Table 28: an "Inaccessible" cylinder (code 5) # heated "from a solid fuel boiler" uses 160 litres. diff --git a/tests/infrastructure/epc_client/test_sap_accuracy_corpus.py b/tests/infrastructure/epc_client/test_sap_accuracy_corpus.py index e6c3be1e..378b6b99 100644 --- a/tests/infrastructure/epc_client/test_sap_accuracy_corpus.py +++ b/tests/infrastructure/epc_client/test_sap_accuracy_corpus.py @@ -41,10 +41,11 @@ _CORPUS = Path( ) # Measured floors/ceilings over the fixed corpus at HEAD (1000 certs, 0 skips). -# Current: SAP within-0.5 = 66.1%, SAP MAE = 1.128 (Table 12a Grid 1 -# integrated-storage code-408 0.20 high-rate fraction, this slice: sap408 -# over-rate +14.6/+12.9/+12.7 -> +7.1/+5.1/+3.4; prior slice was the -# heat-network Table 4c(3) flat-rate charging factor). +# Current: SAP within-0.5 = 66.1%, SAP MAE = 1.124 (RdSAP 10 §10.5 present- +# but-unsized cylinder -> Table 28 Normal 110 L default, this slice — a +# correctness fix: 7 certs that silently dropped storage loss + Table 13; +# marginal on the headline. Prior slices: Table 12a code-408 0.20 storage +# high-rate fraction; heat-network Table 4c(3) flat-rate charging factor). # CO2 MAE = 0.28 t/yr (signed +0.17 — a systematic over-estimate, see below). # PE MAE = 14.7 kWh/m2/yr (signed +9.1). # From 9ee3821138720fb0ee76c0e9a5f6a870b1d1dec1 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Sun, 14 Jun 2026 08:48:38 +0000 Subject: [PATCH 66/87] fix(pv): zero exported PV when dwelling is not export-capable MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit SAP 10.2 Appendix M1 (PDF p.94): "EPV,ex,m = 0 if the PV system is not connected to an export-capable meter." The cascade computed the β-split export stream regardless of `is_dwelling_export_capable`, so a non-export- capable dwelling was credited the full PV export — in the §10a COST it credits at the Table 32 import rate (13.19 p/kWh), which dominates the rating. On 7 Wybourn Terrace S2 5BJ the PE (144 vs lodged 151) and CO2 (27 vs 29) already matched, yet the phantom export cost credit pulled SAP from ~73 to 92.1 (+19). Zero `epv_exported_monthly_kwh` after the Appendix-G4 diverter adjustment when not export-capable; the onsite (EPV,dw) consumption and the diverter HW reduction are unchanged. Not-export-capable PV cohort (corpus, 4 certs): 7 Wybourn +19.1 -> +6.5, 4 Lime Ave +11.1 -> +0.4, 8 Hatherleigh +7.6 -> -0.2, Flat 5 ~-0.4. Gauge 66.1% -> 66.9%, MAE 1.124 -> 1.039. Floor 0.64 -> 0.65 / ceiling 1.18 -> 1.08. Worksheet harness 47/47 0 diverge (Summary certs carry export-capable meters). 1 AAA test, pyright net-zero. Found by auditing the worst over-rater without a worksheet: PE/CO2-match + cost-miss localised it to the PV export credit. Co-Authored-By: Claude Opus 4.8 --- .../sap10_calculator/rdsap/cert_to_inputs.py | 16 ++++++++++ .../rdsap/test_cert_to_inputs.py | 32 +++++++++++++++++++ .../epc_client/test_sap_accuracy_corpus.py | 15 +++++---- 3 files changed, 56 insertions(+), 7 deletions(-) diff --git a/domain/sap10_calculator/rdsap/cert_to_inputs.py b/domain/sap10_calculator/rdsap/cert_to_inputs.py index 9c6cc278..12f5b17e 100644 --- a/domain/sap10_calculator/rdsap/cert_to_inputs.py +++ b/domain/sap10_calculator/rdsap/cert_to_inputs.py @@ -7612,6 +7612,22 @@ def cert_to_inputs( epv_exported_monthly_kwh=adjusted_export_monthly_kwh, ) + # SAP 10.2 Appendix M1 (PDF p.94): "EPV,ex,m = 0 if the PV system is not + # connected to an export-capable meter." A non-export-capable dwelling + # earns no export payment — only the onsite β consumption (EPV,dw) + # offsets demand. Zero the exported stream so the §10a cost, CO2 and PE + # export credits all drop out; the dwelling (onsite) portion and any + # diverter HW reduction above are unchanged. (Without this the cascade + # credited the full export — e.g. cert at 7 Wybourn Terrace S2 5BJ + # over-rated +19 SAP: PE/CO2 matched the lodged figures but the export + # cost credit alone pulled the rating from ~73 to 92.) + if not epc.sap_energy_source.is_dwelling_export_capable: + pv_split = PhotovoltaicSplit( + beta_monthly=pv_split.beta_monthly, + epv_dwelling_monthly_kwh=pv_split.epv_dwelling_monthly_kwh, + epv_exported_monthly_kwh=(0.0,) * 12, + ) + # SAP 10.2 §12.4.4 overrides — when summer immersion applies (back- # boiler combo + cylinder + WHC from main heating), the HW cost / # CO2 / PE factors are kWh-weighted blends of the winter boiler fuel diff --git a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py index a862c238..dbb817bb 100644 --- a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py +++ b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py @@ -1226,6 +1226,38 @@ def test_pv_export_credit_input_reports_rdsap10_table_32_rate() -> None: assert abs(inputs.pv_export_credit_gbp_per_kwh - 0.1319) <= 1e-6 +def test_pv_not_export_capable_zeroes_exported_kwh() -> None: + # Arrange — two identical PV dwellings (same array), one connected to an + # export-capable meter and one not. SAP 10.2 Appendix M1 (PDF p.94): + # "EPV,ex,m = 0 if the PV system is not connected to an export-capable + # meter." A non-export-capable dwelling earns no export — only the + # onsite β consumption offsets demand — so its exported kWh must be 0, + # while the export-capable twin exports a positive amount. The onsite + # (dwelling) portion is identical for both. + capable = _typical_semi_detached_epc() + capable.sap_energy_source.photovoltaic_arrays = [ + PhotovoltaicArray(peak_power=3.0, pitch=2, orientation=5, overshading=1), + ] + capable.sap_energy_source.is_dwelling_export_capable = True + not_capable = _typical_semi_detached_epc() + not_capable.sap_energy_source.photovoltaic_arrays = [ + PhotovoltaicArray(peak_power=3.0, pitch=2, orientation=5, overshading=1), + ] + not_capable.sap_energy_source.is_dwelling_export_capable = False + + # Act + cap_inputs = cert_to_inputs(capable) + nocap_inputs = cert_to_inputs(not_capable) + + # Assert — exported kWh zeroed when not export-capable; onsite unchanged. + assert (cap_inputs.pv_exported_kwh_per_yr or 0.0) > 0.0 + assert (nocap_inputs.pv_exported_kwh_per_yr or 0.0) == 0.0 + assert abs( + (cap_inputs.pv_dwelling_kwh_per_yr or 0.0) + - (nocap_inputs.pv_dwelling_kwh_per_yr or 0.0) + ) <= 1e-9 + + def test_pv_generation_differentiates_arrays_by_orientation() -> None: # Arrange — two single-array PVs at the same kWp / pitch / overshading, # one facing South and one facing North. Per SAP10.2 Appendix M diff --git a/tests/infrastructure/epc_client/test_sap_accuracy_corpus.py b/tests/infrastructure/epc_client/test_sap_accuracy_corpus.py index 378b6b99..13e9f6e6 100644 --- a/tests/infrastructure/epc_client/test_sap_accuracy_corpus.py +++ b/tests/infrastructure/epc_client/test_sap_accuracy_corpus.py @@ -41,11 +41,12 @@ _CORPUS = Path( ) # Measured floors/ceilings over the fixed corpus at HEAD (1000 certs, 0 skips). -# Current: SAP within-0.5 = 66.1%, SAP MAE = 1.124 (RdSAP 10 §10.5 present- -# but-unsized cylinder -> Table 28 Normal 110 L default, this slice — a -# correctness fix: 7 certs that silently dropped storage loss + Table 13; -# marginal on the headline. Prior slices: Table 12a code-408 0.20 storage -# high-rate fraction; heat-network Table 4c(3) flat-rate charging factor). +# Current: SAP within-0.5 = 66.9%, SAP MAE = 1.039 (SAP 10.2 Appendix M1 +# EPV,ex=0 for non-export-capable PV, this slice: 7 Wybourn +19.1 -> +6.5, +# 4 Lime Ave +11.1 -> +0.4, 8 Hatherleigh +7.6 -> -0.2 — PE/CO2 matched +# lodged but the export cost credit alone was over-rating). Prior slices: +# unsized-cylinder Table 28 110 L; Table 12a code-408 0.20 storage fraction; +# heat-network Table 4c(3) flat-rate charging factor. # CO2 MAE = 0.28 t/yr (signed +0.17 — a systematic over-estimate, see below). # PE MAE = 14.7 kWh/m2/yr (signed +9.1). # @@ -66,8 +67,8 @@ _CORPUS = Path( # energy were 5% high; actual SAP bias is +0.145). # So closing demand over-estimates lifts BOTH the SAP gauge and PE/CO2; there is # no one-slice factor fix. RATCHET any ceiling up when a slice tightens it. -_MIN_WITHIN_HALF_SAP = 0.64 -_MAX_SAP_MAE = 1.18 +_MIN_WITHIN_HALF_SAP = 0.65 +_MAX_SAP_MAE = 1.08 _MAX_CO2_MAE_TONNES = 0.35 # t CO2 / yr vs co2_emissions_current _MAX_PE_PER_M2_MAE = 16.0 # kWh / m2 / yr vs energy_consumption_current From e7177a8bd4f318808ce8f0811d3a03b0790a221d Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Sun, 14 Jun 2026 09:16:22 +0000 Subject: [PATCH 67/87] fix(electric-heaters): code-699 "electric heaters assumed" bills Table 12a direct-acting split MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A "No system present: electric heaters assumed" lodging carries SAP Table 4a code 699 (electric room heaters) but RdSAP main_heating_category 1, NOT 10. `_table_12a_system_for_main` keyed the direct-acting-electric routing on category==10 only, so the category-1 form fell through to None and `_space_heating_fuel_cost_gbp_per_kwh` billed space heating 100% at the off-peak LOW rate — as if direct-acting room heaters charged overnight like storage. Per RdSAP 10 §12 Rule 3 (PDF p.62) electric room heaters (691-694, 699) route to the 10-hour tariff, and SAP 10.2 Table 12a Grid 1 (PDF p.191) gives the "other direct-acting electric" row a 0.50 high-rate fraction at 10-hour (1.00 at 7-hour). Route those SAP codes — the same set §12 Rule 3 already uses — to OTHER_DIRECT_ACTING_ELECTRIC alongside the category-10 gate. Found via the PE/CO2-vs-cost split on the worst over-rater in the /tmp sample: cert 2958 PE +0% / CO2 -1% (energy correct) but SAP +32.2 — a pure cost-side bug. Space rate 7.50 -> 11.09 p/kWh; cert 2958 +32.2 -> +14.7. The committed corpus gauge is unchanged (its 3 non-category-10 code-699 certs are all on Single meters -> STANDARD tariff, so this split never applies to them); the win is on the unbiased /tmp population's single worst cert. Co-Authored-By: Claude Opus 4.8 --- .../sap10_calculator/rdsap/cert_to_inputs.py | 38 +++++++++++++---- .../rdsap/test_cert_to_inputs.py | 41 +++++++++++++++++++ 2 files changed, 70 insertions(+), 9 deletions(-) diff --git a/domain/sap10_calculator/rdsap/cert_to_inputs.py b/domain/sap10_calculator/rdsap/cert_to_inputs.py index 12f5b17e..d60daefd 100644 --- a/domain/sap10_calculator/rdsap/cert_to_inputs.py +++ b/domain/sap10_calculator/rdsap/cert_to_inputs.py @@ -2393,6 +2393,15 @@ _TARIFF_HIGH_LOW_FUEL_CODES_TABLE_12: Final[dict[Tariff, tuple[int, int]]] = { } +# SAP Table 4a electric room-heater codes (panel/convector/radiant 691, +# fan 692, portable 693, water-/oil-filled 694, "electric heaters assumed" +# 699) — the same set RdSAP 10 §12 Rule 3 (PDF p.62) routes to the 10-hour +# tariff. They are direct-acting electric for the Table 12a Grid 1 SH split. +_ELECTRIC_ROOM_HEATER_SAP_CODES: Final[frozenset[int]] = frozenset( + {691, 692, 693, 694, 699} +) + + def _table_12a_system_for_main( main: Optional[MainHeatingDetail], ) -> Optional[Table12aSystem]: @@ -2421,15 +2430,26 @@ def _table_12a_system_for_main( main.main_heating_index_number is not None and heat_pump_record(main.main_heating_index_number) is not None ) - # Electric room heaters (RdSAP main_heating_category 10) are direct- - # acting electric → SAP 10.2 Table 12a Grid 1 (PDF p.191) "Other - # systems including direct-acting electric" row (7-hour high-rate - # fraction 1.00, 10-hour 0.50). Distinct from electric STORAGE - # heaters (category 7), which charge off-peak and correctly fall - # through to None here (→ 100% low rate). Gated on `_is_electric_main` - # so a non-electric room heater (gas / solid-fuel cat 10) is excluded; - # all callers already pre-gate on electric, this is belt-and-braces. - if main.main_heating_category == 10 and _is_electric_main(main): + # Electric room heaters are direct-acting electric → SAP 10.2 Table + # 12a Grid 1 (PDF p.191) "Other systems including direct-acting + # electric" row (7-hour high-rate fraction 1.00, 10-hour 0.50). + # Identified EITHER by RdSAP main_heating_category 10 OR by a Table 4a + # electric room-heater SAP code (691-694 panel/fan/portable/water- + # filled, 699 "electric heaters assumed" — the SAME set RdSAP 10 §12 + # Rule 3 (PDF p.62) routes to the 10-hour tariff). The "No system + # present: electric heaters assumed" lodging (code 699) carries + # main_heating_category 1, NOT 10, so the category-only gate missed it + # and it fell through to None → 100% off-peak LOW rate, billing + # direct-acting heaters as if they charged overnight like storage + # (cert 2958 +32.2 SAP, the worst over-rate in the sample). Distinct + # from electric STORAGE heaters (category 7), which DO charge off-peak + # and correctly fall through to None here (→ 100% low rate). Gated on + # `_is_electric_main` so a non-electric room heater (gas / solid-fuel + # cat 10) is excluded; all callers already pre-gate on electric. + if _is_electric_main(main) and ( + main.main_heating_category == 10 + or (code is not None and code in _ELECTRIC_ROOM_HEATER_SAP_CODES) + ): return Table12aSystem.OTHER_DIRECT_ACTING_ELECTRIC # A PCDB Table 362 record IS a heat pump by definition (the Appendix-N # efficiency cascade keys off it), whether or not a Table-4a SAP code diff --git a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py index dbb817bb..4531c08d 100644 --- a/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py +++ b/tests/domain/sap10_calculator/rdsap/test_cert_to_inputs.py @@ -715,6 +715,47 @@ def test_non_integrated_storage_heater_bills_100_percent_low_rate() -> None: assert abs(inputs.space_heating_fuel_cost_gbp_per_kwh - 0.0550) <= 1e-9 +def test_no_system_electric_heaters_assumed_code_699_bills_direct_acting_split() -> None: + # Arrange — "No system present: electric heaters assumed" lodges SAP + # Table 4a code 699 (electric room heaters) but RdSAP main_heating_ + # category 1, NOT 10, on a Dual (Economy-7) meter. RdSAP 10 §12 Rule 3 + # (PDF p.62) routes electric room heaters (691-694, 699) to the 10-hour + # tariff, and SAP 10.2 Table 12a Grid 1 (PDF p.191) gives the "other + # direct-acting electric" row a 0.50 high-rate fraction at 10-hour. + # The scalar SH rate is therefore the blend 0.50 × high (14.68) + 0.50 + # × low (7.50) = 11.09 p/kWh — NOT the 7.50 p/kWh of 100% off-peak that + # the category-10-only gate produced when it missed this category-1 + # lodging. + from dataclasses import replace + + main = MainHeatingDetail( + has_fghrs=False, + main_fuel_type=29, # electricity + heat_emitter_type=0, + emitter_temperature=1, + main_heating_control=2699, + main_heating_category=1, # NOT 10 — the "electric heaters assumed" form + sap_main_heating_code=699, + ) + base = make_minimal_sap10_epc( + total_floor_area_m2=_TYPICAL_TFA_M2, + habitable_rooms_count=4, + country_code="ENG", + sap_building_parts=[make_building_part(construction_age_band="E")], + sap_heating=make_sap_heating(main_heating_details=[main]), + ) + epc = replace( + base, + sap_energy_source=replace(base.sap_energy_source, meter_type="1"), + ) + + # Act + inputs = cert_to_inputs(epc) + + # Assert — blended 0.50×14.68 + 0.50×7.50 = 11.09 p/kWh. + assert abs(inputs.space_heating_fuel_cost_gbp_per_kwh - 0.1109) <= 1e-9 + + def test_heat_network_distribution_electricity_per_sap_10_2_appendix_c_3_2() -> None: # Arrange — heat-network main (Table 4a code 301 = community heating, # category 6). SAP 10.2 Appendix C §C3.2 (PDF p.51): distribution From ac77624d67dd43aae9e3d56654e1e90b42869e02 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Sun, 14 Jun 2026 09:51:34 +0000 Subject: [PATCH 68/87] test(pv-battery): pin SAP cost-neutrality on export-capable standard tariff MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit End-to-end API-path regression pin for the battery behaviour validated by the user-simulated Elmhurst worksheet pair (cert 001431 "simulated case 35/36", 5 kWh, export-capable, mains-gas, standard tariff). The official SAP rating ("10a. Fuel costs - using Table 12 prices") values PV used-in- dwelling and PV exported identically at 13.19 p/kWh (export code 60 == import code 30, ADR-0010), so a battery only redistributes PV between two equally-priced lines: worksheet PV credit (252) = -455.6458 and SAP (258) = 88.0859 are IDENTICAL with/without the battery (ΔSAP = 0). Two tests over the committed RdSAP-21.0.1 corpus: - standard tariff (meter 2): toggling the battery holds continuous SAP EXACTLY constant, while at least one cert's primary energy DOES respond (proving the App-M1 §3c β-split is wired, not a dropped battery). - off-peak tariff (meter != 2): the battery STRICTLY raises SAP, because self-consumed PV displaces high-rate import (15.29) above the 13.19 export credit — confirming the standard-tariff neutrality is a price coincidence, not a no-op. Guards table_32 export price (code 60) and the battery β-split against silent regression. Complements the unit-level β tests in test_photovoltaic.py. Co-Authored-By: Claude Opus 4.8 --- .../test_pv_battery_sap_neutrality.py | 165 ++++++++++++++++++ 1 file changed, 165 insertions(+) create mode 100644 tests/infrastructure/epc_client/test_pv_battery_sap_neutrality.py diff --git a/tests/infrastructure/epc_client/test_pv_battery_sap_neutrality.py b/tests/infrastructure/epc_client/test_pv_battery_sap_neutrality.py new file mode 100644 index 00000000..7a2c1add --- /dev/null +++ b/tests/infrastructure/epc_client/test_pv_battery_sap_neutrality.py @@ -0,0 +1,165 @@ +"""Regression pin: a PV battery is SAP-cost-NEUTRAL on an export-capable +STANDARD-tariff dwelling, but still moves primary energy / CO2 and DOES +help the rating on an off-peak tariff. + +SPEC EVIDENCE — the user-simulated Elmhurst P960 worksheet pair at +`sap worksheets/golden fixture debugging/simulated case 35 (without +battery)` and `simulated case 36 (with battery)` (cert 001431, 5 kWh, +export-capable, mains-gas, standard tariff). The official SAP rating uses +"10a. Fuel costs - using Table 12 prices", where PV used-in-dwelling and +PV exported are BOTH priced at 13.19 p/kWh (export code 60 == import code +30, ADR-0010). A battery only redistributes PV between those two equally- +priced lines (worksheet split 1128/2326 -> 1951/1504 kWh), so the PV +credit (252) = -455.6458 and the SAP (258) = 88.0859 are IDENTICAL with +and without the battery: ΔSAP = exactly 0. The battery DOES move the EPC's +consumer £-bill (a SEPARATE "10a ... using BEDF prices (595)" block at +import 27.67 / export 5.81, £696 -> £515/yr) — but that block never feeds +(258). It also moves CO2 (272) / PE (286), since used vs exported carry +different factors. + +This guards two things against silent regression: + 1. The export price == import price (13.19) that makes the credit split- + invariant on standard tariff — if someone changes table_32 code 60, + the standard-tariff ΔSAP would stop being 0 and Test A fails. + 2. The Appendix M1 §3c battery β-split is actually wired — if a refactor + dropped the battery, PE would stop responding (Test A's PE assertion) + AND the off-peak benefit would vanish (Test B). + +The β-coefficient formula itself is unit-tested in +tests/domain/sap10_calculator/worksheet/test_photovoltaic.py; this is the +end-to-end API-path (`from_api_response` -> cert_to_inputs -> calculator) +counterpart over the committed RdSAP-21.0.1 corpus. +""" + +from __future__ import annotations + +import json +from copy import deepcopy +from pathlib import Path +from typing import Any, cast + +import pytest + +from datatypes.epc.domain.mapper import EpcPropertyDataMapper +from domain.sap10_calculator.calculator import calculate_sap_from_inputs +from domain.sap10_calculator.rdsap.cert_to_inputs import ( + SAP_10_2_SPEC_PRICES, + cert_to_inputs, +) + +_CORPUS = Path("backend/epc_api/json_samples/RdSAP-Schema-21.0.1/corpus.jsonl") + +# RdSAP meter_type 2 = Single (standard tariff). Anything else (1 Dual / 4 +# Dual-24h / 5 18-hour) is an off-peak tariff where self-consumed PV +# displaces a higher unit rate than the export credit — see _rdsap_tariff. +_STANDARD_METER_TYPE = 2 + + +def _load_corpus() -> list[dict[str, Any]]: + if not _CORPUS.exists(): + return [] + return [ + json.loads(line) + for line in _CORPUS.read_text().splitlines() + if line.strip() + ] + + +def _energy_source(doc: dict[str, Any]) -> dict[str, Any]: + es = doc.get("sap_energy_source") + return cast("dict[str, Any]", es) if isinstance(es, dict) else {} + + +def _has_real_pv_and_battery(doc: dict[str, Any]) -> bool: + es = _energy_source(doc) + if not es.get("is_dwelling_export_capable"): + return False + if (es.get("pv_battery_count") or 0) < 1: + return False + pv = es.get("photovoltaic_supply") + # A bare "no details" array / "none_or_no_details" dict carries no PV. + if isinstance(pv, dict) and "none_or_no_details" in pv: + return False + return True + + +def _sap_pe(doc: dict[str, Any]) -> tuple[float, float]: + epc = EpcPropertyDataMapper.from_api_response(doc) + result = calculate_sap_from_inputs( + cert_to_inputs(epc, prices=SAP_10_2_SPEC_PRICES) + ) + return result.sap_score_continuous, result.primary_energy_kwh_per_m2 + + +def _with_battery_zeroed(doc: dict[str, Any]) -> dict[str, Any]: + zeroed = deepcopy(doc) + zeroed["sap_energy_source"]["pv_battery_count"] = 0 + return zeroed + + +def test_battery_is_sap_cost_neutral_on_export_capable_standard_tariff() -> None: + # Arrange — every export-capable, standard-tariff (meter 2) corpus cert + # that carries both a real PV array and a battery (the worksheet case + # 35/36 profile: mains-gas, single-rate meter). + corpus = _load_corpus() + if not corpus: + pytest.skip(f"no corpus at {_CORPUS}") + standard_battery_certs = [ + doc + for doc in corpus + if _has_real_pv_and_battery(doc) + and _energy_source(doc).get("meter_type") == _STANDARD_METER_TYPE + ] + assert standard_battery_certs, ( + "no export-capable standard-tariff PV+battery cert in the corpus — " + "this regression pin has nothing to guard" + ) + + # Act / Assert — toggling the battery off leaves the continuous SAP + # EXACTLY unchanged (export price == import price 13.19 makes the + # onsite/export split cost-irrelevant), while at least one cert's + # primary energy DOES respond (proving the battery is modelled, not + # silently dropped). + pe_deltas: list[float] = [] + for doc in standard_battery_certs: + sap_on, pe_on = _sap_pe(doc) + sap_off, pe_off = _sap_pe(_with_battery_zeroed(doc)) + assert abs(sap_on - sap_off) <= 1e-9, ( + "battery changed SAP on a standard-tariff export-capable cert: " + f"{sap_on} vs {sap_off}" + ) + pe_deltas.append(pe_on - pe_off) + + assert min(pe_deltas) < -1e-6, ( + "no standard-tariff battery cert showed a primary-energy effect — " + "the App-M1 §3c battery β-split may have been dropped" + ) + + +def test_battery_improves_sap_on_export_capable_off_peak_tariff() -> None: + # Arrange — export-capable PV+battery certs on an OFF-PEAK tariff + # (meter_type != 2). Here self-consumed PV displaces high-rate import + # (e.g. 7-hour high 15.29 p/kWh) which exceeds the 13.19 export credit, + # so a battery (more self-consumption) lowers cost. + corpus = _load_corpus() + if not corpus: + pytest.skip(f"no corpus at {_CORPUS}") + off_peak_battery_certs = [ + doc + for doc in corpus + if _has_real_pv_and_battery(doc) + and _energy_source(doc).get("meter_type") != _STANDARD_METER_TYPE + ] + if not off_peak_battery_certs: + pytest.skip("no export-capable off-peak PV+battery cert in the corpus") + + # Act / Assert — the battery STRICTLY raises the SAP, confirming the + # standard-tariff neutrality above is a price coincidence (import == + # export) and not a dropped-battery no-op. + for doc in off_peak_battery_certs: + sap_on, _ = _sap_pe(doc) + sap_off, _ = _sap_pe(_with_battery_zeroed(doc)) + assert sap_on - sap_off > 1e-6, ( + "battery did not improve SAP on an off-peak export-capable cert: " + f"{sap_on} vs {sap_off}" + ) From 9830ea21100edd51d218f0ab8d64789ba20ab266 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Sun, 14 Jun 2026 10:35:38 +0000 Subject: [PATCH 69/87] fix(elmhurst): raise on unmapped fuel-fired secondary room-heater code MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Elmhurst Summary lodges only the secondary heating SAP code (Table 4a Category 10), never its fuel. `_elmhurst_secondary_fuel_from_sap_code` mapped the gas block (601-613 → mains gas) and solid block (631-634 → house coal) to their modal defaults, but returned None for any OTHER Category-10 code — and None makes the cascade SILENTLY bill the secondary as electricity (13.19 p/kWh). For a fuel-fired heater (e.g. 621-625 liquid-fuel oil/bioethanol) that is a large, invisible mis-price. Per the UnmappedElmhurstLabel strict-raise pattern (mirrors the wall_type / glazing label raises), a fuel-fired Category-10 code (601-699) outside the mapped gas/solid blocks now RAISES instead of guessing. Electric room heaters (691-699) keep returning None — electricity IS their fuel. The gas block 601-613 still resolves to the modal default mains gas: the Summary cannot distinguish mains gas from LPG/biogas, so an LPG or biogas live-effect fire (worksheet "simulated case 37" used biogas at 7.60 p/kWh vs our 3.48 p/kWh mains-gas default, a +7 SAP gap) is not recoverable from the Summary export — that is a data-availability limit, not a guess we can fix here. This commit closes the genuinely-silent-wrong path; the gas sub-fuel remains the documented modal default. Worksheet harness 47/47, 0 raised. 3 AAA tests, pyright net-zero, regression clean, corpus gauge unchanged (Elmhurst-path only; the API path lodges the secondary fuel explicitly). Co-Authored-By: Claude Opus 4.8 --- datatypes/epc/domain/mapper.py | 23 ++++++++- .../epc/domain/tests/test_from_site_notes.py | 49 +++++++++++++++++++ 2 files changed, 71 insertions(+), 1 deletion(-) diff --git a/datatypes/epc/domain/mapper.py b/datatypes/epc/domain/mapper.py index e744e9fb..00dace05 100644 --- a/datatypes/epc/domain/mapper.py +++ b/datatypes/epc/domain/mapper.py @@ -5347,7 +5347,7 @@ def _elmhurst_secondary_fuel_from_sap_code( column B for LPG. Cohort default is mains gas (`_ELMHURST_MAIN_FUEL_TO_SAP10["Mains gas"] = 26`). 621-625: Liquid fuel room heaters (oil / bioethanol). Cohort - not yet exercised; deferred until a fixture surfaces. + not yet exercised — NOT mapped, so they RAISE (see below). 631-634: Solid fuel room heaters (open fire, closed room heater with/without boiler). House coal is the modal default per Table 12 secondary rate (3.67 p/kWh). @@ -5358,6 +5358,21 @@ def _elmhurst_secondary_fuel_from_sap_code( in grate") path — pre-slice the cascade defaulted to electricity at 13.19 p/kWh, over-charging secondary by ~£340/yr and pushing SAP -15.81 below the worksheet's 63.87. + + RAISE-don't-guess: a Category-10 room-heater code (601-699) that is + fuel-fired but NOT in the mapped gas/solid blocks (e.g. 621-625 liquid + fuel) used to return None → the cascade silently billed it as + electricity (13.19 p/kWh), a large mis-price for an oil/LPG heater. + Per the `UnmappedElmhurstLabel` strict-raise pattern these now raise + so the gap surfaces instead of producing a wrong SAP. Electric room + heaters (691-699) keep returning None — electricity IS their fuel. + + NOTE the gas block 601-613 still resolves to the MODAL default mains + gas: the Summary lodges only the SAP code, never the gas sub-fuel, so + an LPG or biogas live-effect fire (worksheet "simulated case 37" used + biogas at 7.60 p/kWh) is indistinguishable here from mains gas — the + cohort default is correct for the common case and the rarer sub-fuels + are not recoverable from the Summary export. """ if sap_code is None: return None @@ -5365,6 +5380,12 @@ def _elmhurst_secondary_fuel_from_sap_code( return 26 # Mains gas, matching `_ELMHURST_MAIN_FUEL_TO_SAP10` if 631 <= sap_code <= 634: return 11 # House coal (Coal in `_ELMHURST_MAIN_FUEL_TO_SAP10`) + if 691 <= sap_code <= 699: + return None # Electric room heaters → cascade electricity default + if 601 <= sap_code <= 699: + # Fuel-fired Category-10 room heater we do not yet map — raise + # rather than let the cascade silently bill it as electricity. + raise UnmappedElmhurstLabel("secondary_heating.sap_code", str(sap_code)) return None diff --git a/datatypes/epc/domain/tests/test_from_site_notes.py b/datatypes/epc/domain/tests/test_from_site_notes.py index e1d0e2cf..c10bec19 100644 --- a/datatypes/epc/domain/tests/test_from_site_notes.py +++ b/datatypes/epc/domain/tests/test_from_site_notes.py @@ -713,3 +713,52 @@ class TestUnmeasurableWallThickness: def test_wall_thickness_mm_is_none(self, result: EpcPropertyData) -> None: assert result.sap_building_parts[0].wall_thickness_mm is None + + +class TestElmhurstSecondaryFuelFromSapCode: + """`_elmhurst_secondary_fuel_from_sap_code` must RAISE on an unmapped + fuel-fired Table 4a Category-10 room-heater code rather than return + None and let the cascade silently bill it as electricity (13.19 + p/kWh). The Summary lodges only the secondary SAP code, not its fuel. + """ + + def test_gas_room_heater_resolves_to_mains_gas_modal_default(self) -> None: + # Arrange — SAP code 605 (flush-fitting live-effect gas fire), in + # the 601-613 gas block. The Summary cannot distinguish mains gas + # from LPG/biogas, so the modal default is mains gas (26). + from datatypes.epc.domain.mapper import ( + _elmhurst_secondary_fuel_from_sap_code, # pyright: ignore[reportPrivateUsage] + ) + + # Act + fuel = _elmhurst_secondary_fuel_from_sap_code(605) + + # Assert + assert fuel == 26 + + def test_electric_room_heater_returns_none_for_electricity_default(self) -> None: + # Arrange — SAP code 693 (electric room heater); electricity IS its + # fuel, so None (→ cascade electricity default) is correct, NOT a + # silent mis-fuel. + from datatypes.epc.domain.mapper import ( + _elmhurst_secondary_fuel_from_sap_code, # pyright: ignore[reportPrivateUsage] + ) + + # Act + fuel = _elmhurst_secondary_fuel_from_sap_code(693) + + # Assert + assert fuel is None + + def test_unmapped_liquid_fuel_room_heater_raises(self) -> None: + # Arrange — SAP code 621 (liquid-fuel room heater) is fuel-fired but + # not in the mapped gas/solid blocks; returning None would silently + # bill it as electricity. It must raise instead. + from datatypes.epc.domain.mapper import ( + UnmappedElmhurstLabel, + _elmhurst_secondary_fuel_from_sap_code, # pyright: ignore[reportPrivateUsage] + ) + + # Act / Assert + with pytest.raises(UnmappedElmhurstLabel): + _elmhurst_secondary_fuel_from_sap_code(621) From 0fae84d2b62ba6188f839ec92f2d9f93c5a61118 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 15 Jun 2026 06:27:37 +0000 Subject: [PATCH 70/87] fix(elmhurst): map secondary room-heater SAP codes to Table 4a fuel category MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Completes `_elmhurst_secondary_fuel_from_sap_code` per SAP 10.2 §12 (PDF p.34: "Secondary heating systems and applicable fuel types are taken from the room heaters section of Table 4a") + RdSAP 10 §10.4.1. Each Table 4a room-heater code now resolves to its fuel CATEGORY's modal fuel: - gas room heaters 601-613 → mains gas (26 → Table 32 1, 3.48 p/kWh) - liquid room heaters 621-625 → heating oil (28 → Table 32 4, 5.44 p/kWh) - solid room heaters 631-636 → house coal (11 → Table 32 11, 3.67 p/kWh) - electric room htrs 691-694/699/701 → None (cascade electricity default) Previously only the gas (601-613→26) and solid (631-634→11) blocks were mapped; liquid heaters (621-625) and 635-636 fell through to None → silently billed as electricity (13.19 p/kWh), a large mis-price for an oil/solid heater. The prior slice raised on those; this maps them to the correct category fuel instead, and keeps the raise ONLY for codes inside the room-heater range (601-701) that are not a recognised Table 4a row. The specific sub-fuel within a category (mains gas vs LPG vs biogas) is a SEPARATE lodgement per §10.4.1 and is NOT exported in the Summary, so the gas block stays the modal mains gas — worksheet "simulated case 37" lodged its 605 live-effect fire on biogas (7.60 p/kWh), unrecoverable from the Summary code alone (this is the entire +7 SAP case-37 gap: secondary energy £131 + a separate biogas standing charge £70; every other line matches the worksheet exactly, incl. (206) main efficiency 61%). 5 AAA tests, harness 47/47 (0 raised), pyright net-zero, regression clean, corpus gauge unchanged (Elmhurst-path only). Co-Authored-By: Claude Opus 4.8 --- datatypes/epc/domain/mapper.py | 95 ++++++++++++------- .../epc/domain/tests/test_from_site_notes.py | 36 ++++++- 2 files changed, 90 insertions(+), 41 deletions(-) diff --git a/datatypes/epc/domain/mapper.py b/datatypes/epc/domain/mapper.py index 00dace05..7e3add21 100644 --- a/datatypes/epc/domain/mapper.py +++ b/datatypes/epc/domain/mapper.py @@ -5332,6 +5332,35 @@ def _elmhurst_pump_age_int(age_str: Optional[str]) -> Optional[int]: return 2 +# SAP 10.2 Table 4a "Room heaters" section — the secondary-heating SAP +# code → FUEL CATEGORY map. Per RdSAP 10 §10.4.1 + SAP 10.2 §12 (PDF +# p.34, "Secondary heating systems and applicable fuel types are taken +# from the room heaters section of Table 4a") each code carries an +# appliance type whose APPLICABLE FUELS are a category (gas / liquid / +# solid / electric); the SPECIFIC sub-fuel within that category (mains +# gas vs LPG vs biogas) is lodged SEPARATELY and is NOT recoverable from +# the Summary, which exports only the code. We therefore resolve each +# code to its category's MODAL fuel. Codes mirror the Table 4a efficiency +# rows in `domain.sap10_ml.sap_efficiencies` (601-613 gas, 621-625 +# liquid, 631-636 solid, 691-694/699/701 electric). +_ELMHURST_SECONDARY_GAS_CODES: Final[frozenset[int]] = frozenset( + {601, 602, 603, 604, 605, 606, 607, 609, 610, 611, 612, 613} +) +_ELMHURST_SECONDARY_LIQUID_CODES: Final[frozenset[int]] = frozenset( + {621, 622, 623, 624, 625} +) +_ELMHURST_SECONDARY_SOLID_CODES: Final[frozenset[int]] = frozenset( + {631, 632, 633, 634, 635, 636} +) +_ELMHURST_SECONDARY_ELECTRIC_CODES: Final[frozenset[int]] = frozenset( + {691, 692, 693, 694, 699, 701} +) +# Modal Table 32 / `_ELMHURST_MAIN_FUEL_TO_SAP10` fuel code per category. +_SECONDARY_FUEL_MAINS_GAS: Final[int] = 26 # Table 32 code 1, 3.48 p/kWh +_SECONDARY_FUEL_HEATING_OIL: Final[int] = 28 # → Table 32 code 4, 5.44 p/kWh +_SECONDARY_FUEL_HOUSE_COAL: Final[int] = 11 # Table 32 code 11, 3.67 p/kWh + + def _elmhurst_secondary_fuel_from_sap_code( sap_code: Optional[int], ) -> Optional[int]: @@ -5342,49 +5371,43 @@ def _elmhurst_secondary_fuel_from_sap_code( electricity when `secondary_fuel_type` is None — correct for the portable-electric default but wrong for fuel-fired room heaters. - SAP 10.2 Table 4a Category 10 ("Room heaters") code blocks: - 601-613: Gas (mains gas / LPG / biogas) — column A is mains gas; - column B for LPG. Cohort default is mains gas - (`_ELMHURST_MAIN_FUEL_TO_SAP10["Mains gas"] = 26`). - 621-625: Liquid fuel room heaters (oil / bioethanol). Cohort - not yet exercised — NOT mapped, so they RAISE (see below). - 631-634: Solid fuel room heaters (open fire, closed room - heater with/without boiler). House coal is the modal - default per Table 12 secondary rate (3.67 p/kWh). - 691-699: Electric room heaters. Cascade default (None) routes - to standard electricity (13.19 p/kWh). + Each code resolves to its SAP 10.2 Table 4a fuel CATEGORY's modal + fuel (per RdSAP 10 §10.4.1 — the specific sub-fuel is a separate + lodgement the Summary omits): + - gas room heaters (601-613) → mains gas (26) + - liquid room heaters (621-625) → heating oil (28) + - solid room heaters (631-636) → house coal (11) + - electric room heaters (691-699, → None (cascade electricity + 701) default; electricity IS the fuel) + + A code in the room-heater range (601-701) that is NOT a recognised + Table 4a row RAISES `UnmappedElmhurstLabel` rather than silently + mis-fuelling — per the strict-raise pattern. Codes outside the range + (e.g. None / no secondary) return None. Cohort cert 2102-3018-0205-7886-5204 surfaces the 631 ("Open fire - in grate") path — pre-slice the cascade defaulted to electricity - at 13.19 p/kWh, over-charging secondary by ~£340/yr and pushing - SAP -15.81 below the worksheet's 63.87. + in grate") path — pre-fix the cascade defaulted to electricity at + 13.19 p/kWh, over-charging secondary by ~£340/yr and pushing SAP + -15.81 below the worksheet's 63.87. - RAISE-don't-guess: a Category-10 room-heater code (601-699) that is - fuel-fired but NOT in the mapped gas/solid blocks (e.g. 621-625 liquid - fuel) used to return None → the cascade silently billed it as - electricity (13.19 p/kWh), a large mis-price for an oil/LPG heater. - Per the `UnmappedElmhurstLabel` strict-raise pattern these now raise - so the gap surfaces instead of producing a wrong SAP. Electric room - heaters (691-699) keep returning None — electricity IS their fuel. - - NOTE the gas block 601-613 still resolves to the MODAL default mains - gas: the Summary lodges only the SAP code, never the gas sub-fuel, so - an LPG or biogas live-effect fire (worksheet "simulated case 37" used - biogas at 7.60 p/kWh) is indistinguishable here from mains gas — the - cohort default is correct for the common case and the rarer sub-fuels - are not recoverable from the Summary export. + SUB-FUEL CAVEAT: the gas block 601-613 resolves to the modal mains + gas; an LPG or biogas live-effect fire (worksheet "simulated case 37" + lodged biogas at 7.60 p/kWh vs mains gas 3.48 p/kWh) is + indistinguishable here — the sub-fuel is not in the Summary export. """ if sap_code is None: return None - if 601 <= sap_code <= 613: - return 26 # Mains gas, matching `_ELMHURST_MAIN_FUEL_TO_SAP10` - if 631 <= sap_code <= 634: - return 11 # House coal (Coal in `_ELMHURST_MAIN_FUEL_TO_SAP10`) - if 691 <= sap_code <= 699: + if sap_code in _ELMHURST_SECONDARY_GAS_CODES: + return _SECONDARY_FUEL_MAINS_GAS + if sap_code in _ELMHURST_SECONDARY_LIQUID_CODES: + return _SECONDARY_FUEL_HEATING_OIL + if sap_code in _ELMHURST_SECONDARY_SOLID_CODES: + return _SECONDARY_FUEL_HOUSE_COAL + if sap_code in _ELMHURST_SECONDARY_ELECTRIC_CODES: return None # Electric room heaters → cascade electricity default - if 601 <= sap_code <= 699: - # Fuel-fired Category-10 room heater we do not yet map — raise - # rather than let the cascade silently bill it as electricity. + if 601 <= sap_code <= 701: + # A room-heater-range code we don't recognise — raise rather than + # let the cascade silently bill it as electricity. raise UnmappedElmhurstLabel("secondary_heating.sap_code", str(sap_code)) return None diff --git a/datatypes/epc/domain/tests/test_from_site_notes.py b/datatypes/epc/domain/tests/test_from_site_notes.py index c10bec19..289d415a 100644 --- a/datatypes/epc/domain/tests/test_from_site_notes.py +++ b/datatypes/epc/domain/tests/test_from_site_notes.py @@ -750,10 +750,36 @@ class TestElmhurstSecondaryFuelFromSapCode: # Assert assert fuel is None - def test_unmapped_liquid_fuel_room_heater_raises(self) -> None: - # Arrange — SAP code 621 (liquid-fuel room heater) is fuel-fired but - # not in the mapped gas/solid blocks; returning None would silently - # bill it as electricity. It must raise instead. + def test_liquid_fuel_room_heater_resolves_to_heating_oil(self) -> None: + # Arrange — SAP code 621 (liquid-fuel room heater) → its Table 4a + # category's modal fuel, heating oil (28 → Table 32 code 4), NOT a + # silent electricity fallback. + from datatypes.epc.domain.mapper import ( + _elmhurst_secondary_fuel_from_sap_code, # pyright: ignore[reportPrivateUsage] + ) + + # Act + fuel = _elmhurst_secondary_fuel_from_sap_code(621) + + # Assert + assert fuel == 28 + + def test_solid_fuel_room_heater_resolves_to_house_coal(self) -> None: + # Arrange — SAP code 631 (open fire in grate) → house coal (11). + from datatypes.epc.domain.mapper import ( + _elmhurst_secondary_fuel_from_sap_code, # pyright: ignore[reportPrivateUsage] + ) + + # Act + fuel = _elmhurst_secondary_fuel_from_sap_code(631) + + # Assert + assert fuel == 11 + + def test_unrecognised_room_heater_range_code_raises(self) -> None: + # Arrange — SAP code 620 sits in the room-heater range (601-701) but + # is not a recognised Table 4a row; rather than silently mis-fuel it + # must raise. from datatypes.epc.domain.mapper import ( UnmappedElmhurstLabel, _elmhurst_secondary_fuel_from_sap_code, # pyright: ignore[reportPrivateUsage] @@ -761,4 +787,4 @@ class TestElmhurstSecondaryFuelFromSapCode: # Act / Assert with pytest.raises(UnmappedElmhurstLabel): - _elmhurst_secondary_fuel_from_sap_code(621) + _elmhurst_secondary_fuel_from_sap_code(620) From 361abc1202d3ff3f009e0d5757287992c1cb9e50 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 15 Jun 2026 06:43:55 +0000 Subject: [PATCH 71/87] fix(mapper): handle 'ND' multiple_glazing_type on RdSAP-Schema-20.0.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `_synthesise_20_0_0_sap_windows` passed `schema.multiple_glazing_type` straight into `_api_cascade_glazing_type`, which raised UnmappedApiCode on the "ND" (Not Defined) string that the 20.0.0 corpus lodges alongside the 1-8 integer codes — failing the mapper-coverage guard on every ND-glazed 20.0.0 cert. Mirror the existing 18.0/19.0/17.x seams: route integer codes through the cascade, fall the "ND" string back to the DG-modal default (cascade code 2 → daylight g_L 0.80). Also corrects the 20.0.0 schema field type `int` → `Union[int, str]` to match the data (as 18.0 already does), which keeps the isinstance guard pyright-clean. Pre-existing failure (present before this branch's recent commits), not in the handover regression gate. Fixes all 15 RdSAP-Schema-20.0.0 ND certs; test_mapper_corpus 6002/6002 pass. pyright net-zero. Co-Authored-By: Claude Opus 4.8 --- datatypes/epc/domain/mapper.py | 18 ++++++++++++++++-- datatypes/epc/schema/rdsap_schema_20_0_0.py | 4 +++- 2 files changed, 19 insertions(+), 3 deletions(-) diff --git a/datatypes/epc/domain/mapper.py b/datatypes/epc/domain/mapper.py index 7e3add21..06771a7e 100644 --- a/datatypes/epc/domain/mapper.py +++ b/datatypes/epc/domain/mapper.py @@ -3319,14 +3319,28 @@ def _synthesise_reduced_field_windows( ] +# ADR-0028: multiple_glazing_type "ND" (Not Defined) → the DG-modal +# default (cascade code 2 → daylight g_L 0.80), as for 18.0/19.0/17.x. +_RDSAP20_ND_GLAZING_TYPE: int = 2 + + def _synthesise_20_0_0_sap_windows(schema: RdSapSchema20_0_0) -> List[SapWindow]: """ADR-0027/0028 seam: 20.0.0 glazed_type codes 1-8+ND are identical to 21.0.1's, so route multiple_glazing_type through the verified cascade (fixes - code 1 "DG pre-2002" read as single), then call the shared core.""" + code 1 "DG pre-2002" read as single), then call the shared core. The "ND" + string (no cascade mapping) falls back to the DG-modal default, mirroring + `_synthesise_18_0_sap_windows` — without this the cascade raised + UnmappedApiCode on every ND-glazed 20.0.0 cert.""" + mgt = schema.multiple_glazing_type + glazing_type = ( + _api_cascade_glazing_type(mgt) + if isinstance(mgt, int) + else _RDSAP20_ND_GLAZING_TYPE + ) return _synthesise_reduced_field_windows( schema.glazed_area, schema.total_floor_area, - _api_cascade_glazing_type(schema.multiple_glazing_type), + glazing_type, ) diff --git a/datatypes/epc/schema/rdsap_schema_20_0_0.py b/datatypes/epc/schema/rdsap_schema_20_0_0.py index dbd5feef..9a4b78f2 100644 --- a/datatypes/epc/schema/rdsap_schema_20_0_0.py +++ b/datatypes/epc/schema/rdsap_schema_20_0_0.py @@ -267,7 +267,9 @@ class RdSapSchema20_0_0: energy_rating_current: int lighting_cost_current: float main_heating_controls: List[EnergyElement] - multiple_glazing_type: int + # "ND" (Not Defined) appears in the corpus alongside the 1-8 integer + # codes — type as Union to match the data (mirrors RdSAP-Schema-18.0). + multiple_glazing_type: Union[int, str] open_fireplaces_count: int heating_cost_potential: float hot_water_cost_current: float From c11eb46b8ab5409ad340552783ad6d3e6d06edce Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 15 Jun 2026 06:53:14 +0000 Subject: [PATCH 72/87] fix(modelling): HHR overlay sets off-peak immersion type so HW Table 13 applies MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The HHR-storage HeatingOverlay (ADR-0024) added an off-peak electric immersion cylinder but never set `immersion_heating_type`, so the overlaid cert left it None. The calculator then could not resolve `immersion_single` for the SAP 10.2 Table 13 HW high-rate split and billed hot water 100% at the off-peak low rate — £127.41 vs the relodged after-cert's £169.39, overstating the overlay's SAP by +1.26 (CO2/PE matched, isolating it to the HW cost path). Add `immersion_heating_type` to HeatingOverlay, route it through `_fold_heating` (it lives on `sap_heating`), and set it to 1 (single off-peak immersion) on the HHR overlay to match the relodged reference. Closes both `test_hhr_storage_overlay_reproduces_the_relodged_after_*` cascade pins (electric-storage and no-system befores share the after). Pre-existing failure (present before this branch's recent commits), outside the handover regression gate. Full modelling suite 220 pass, pyright net- zero. Co-Authored-By: Claude Opus 4.8 --- domain/modelling/generators/heating_recommendation.py | 4 ++++ domain/modelling/scoring/overlay_applicator.py | 1 + domain/modelling/simulation.py | 5 +++++ tests/domain/modelling/test_heating_recommendation.py | 1 + 4 files changed, 11 insertions(+) diff --git a/domain/modelling/generators/heating_recommendation.py b/domain/modelling/generators/heating_recommendation.py index 410eb5e6..b10e1b76 100644 --- a/domain/modelling/generators/heating_recommendation.py +++ b/domain/modelling/generators/heating_recommendation.py @@ -64,6 +64,10 @@ _HHR_STORAGE_OVERLAY = HeatingOverlay( cylinder_insulation_type=1, cylinder_insulation_thickness_mm=120, cylinder_thermostat="Y", + # Single off-peak electric immersion — drives the SAP 10.2 Table 13 HW + # high-rate split (matches the relodged after-cert; without it the HW + # bills 100% at the low rate, +1.26 SAP over the reference). + immersion_heating_type=1, has_hot_water_cylinder=True, meter_type="Dual", ) diff --git a/domain/modelling/scoring/overlay_applicator.py b/domain/modelling/scoring/overlay_applicator.py index c11285ca..d47d84a7 100644 --- a/domain/modelling/scoring/overlay_applicator.py +++ b/domain/modelling/scoring/overlay_applicator.py @@ -98,6 +98,7 @@ _SAP_HEATING_FIELDS: tuple[str, ...] = ( "cylinder_insulation_type", "cylinder_insulation_thickness_mm", "cylinder_thermostat", + "immersion_heating_type", ) _ENERGY_SOURCE_FIELDS: tuple[str, ...] = ("meter_type", "mains_gas") diff --git a/domain/modelling/simulation.py b/domain/modelling/simulation.py index 083f8898..7d951ac5 100644 --- a/domain/modelling/simulation.py +++ b/domain/modelling/simulation.py @@ -133,6 +133,11 @@ class HeatingOverlay: cylinder_insulation_type: Optional[Union[int, str]] = None cylinder_insulation_thickness_mm: Optional[int] = None cylinder_thermostat: Optional[str] = None + # The cylinder's immersion-heater arrangement (e.g. single off-peak + # immersion = 1). Drives the SAP 10.2 Table 13 HW high-rate fraction on + # off-peak tariffs — without it the calculator cannot resolve + # `immersion_single` and bills HW 100% at the low rate. + immersion_heating_type: Optional[Union[int, str]] = None # EpcPropertyData (top-level) has_hot_water_cylinder: Optional[bool] = None # sap_energy_source diff --git a/tests/domain/modelling/test_heating_recommendation.py b/tests/domain/modelling/test_heating_recommendation.py index 5e8e1576..81b9f692 100644 --- a/tests/domain/modelling/test_heating_recommendation.py +++ b/tests/domain/modelling/test_heating_recommendation.py @@ -67,6 +67,7 @@ def test_electric_storage_dwelling_yields_an_hhr_storage_bundle() -> None: cylinder_insulation_type=1, cylinder_insulation_thickness_mm=120, cylinder_thermostat="Y", + immersion_heating_type=1, has_hot_water_cylinder=True, meter_type="Dual", ) From 383b8b0c375c3b1d6e0971af3dfa64f196e7b0a3 Mon Sep 17 00:00:00 2001 From: Daniel Roth Date: Mon, 15 Jun 2026 10:48:17 +0000 Subject: [PATCH 73/87] =?UTF-8?q?SharePoint=20renamer=20build=5Fcanonical?= =?UTF-8?q?=5Ffilename=20behaviour=20verified=20by=20tests=20=F0=9F=9F=A9?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- pytest.ini | 2 + scripts/tests/__init__.py | 0 .../tests/test_build_canonical_filename.py | 106 ++++++++++++++++++ 3 files changed, 108 insertions(+) create mode 100644 scripts/tests/__init__.py create mode 100644 scripts/tests/test_build_canonical_filename.py diff --git a/pytest.ini b/pytest.ini index 2bcd6178..a6eba3be 100644 --- a/pytest.ini +++ b/pytest.ini @@ -25,5 +25,7 @@ testpaths = etl/epc_clean/tests etl/hubspot/tests etl/spatial/tests + scripts/tests + ; tests/ markers = integration: mark a test as an integration test diff --git a/scripts/tests/__init__.py b/scripts/tests/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/scripts/tests/test_build_canonical_filename.py b/scripts/tests/test_build_canonical_filename.py new file mode 100644 index 00000000..3890477c --- /dev/null +++ b/scripts/tests/test_build_canonical_filename.py @@ -0,0 +1,106 @@ +# scripts/tests/test_build_canonical_filename.py +from scripts.rename_sharepoint_files import build_canonical_filename + +UPRN = "10093456789" +ADDRESS = "1 High Street, Anytown" +POSTCODE = "SW1A 1AA" +STREET = "1 High Street" + + +def test_already_renamed_returns_none() -> None: + # Arrange + original = f"{UPRN}_High Street SW1A 1AA_EPC Report.pdf" + + # Act + result = build_canonical_filename(UPRN, ADDRESS, POSTCODE, original) + + # Assert + assert result is None + + +def test_address_postcode_prefix_stripped() -> None: + # Arrange + original = f"{ADDRESS} {POSTCODE} - EPC Report.pdf" + + # Act + result = build_canonical_filename(UPRN, ADDRESS, POSTCODE, original) + + # Assert + assert result == f"{UPRN}_{STREET} {POSTCODE}_EPC Report.pdf" + + +def test_address_only_prefix_stripped() -> None: + # Arrange + original = f"{ADDRESS} - EPC Report.pdf" + + # Act + result = build_canonical_filename(UPRN, ADDRESS, POSTCODE, original) + + # Assert + assert result == f"{UPRN}_{STREET} {POSTCODE}_EPC Report.pdf" + + +def test_street_postcode_prefix_stripped() -> None: + # Arrange + original = f"{STREET} {POSTCODE} - EPC Report.pdf" + + # Act + result = build_canonical_filename(UPRN, ADDRESS, POSTCODE, original) + + # Assert + assert result == f"{UPRN}_{STREET} {POSTCODE}_EPC Report.pdf" + + +def test_street_only_prefix_stripped() -> None: + # Arrange + original = f"{STREET} - EPC Report.pdf" + + # Act + result = build_canonical_filename(UPRN, ADDRESS, POSTCODE, original) + + # Assert + assert result == f"{UPRN}_{STREET} {POSTCODE}_EPC Report.pdf" + + +def test_dash_separator_removed_after_prefix_strip() -> None: + # Arrange – " - " separator between prefix and doc name + original = f"{STREET} {POSTCODE} - Floor Plan.pdf" + + # Act + result = build_canonical_filename(UPRN, ADDRESS, POSTCODE, original) + + # Assert + assert result == f"{UPRN}_{STREET} {POSTCODE}_Floor Plan.pdf" + + +def test_underscore_separator_removed_after_prefix_strip() -> None: + # Arrange – " _ " separator between prefix and doc name + original = f"{STREET} {POSTCODE} _ Floor Plan.pdf" + + # Act + result = build_canonical_filename(UPRN, ADDRESS, POSTCODE, original) + + # Assert + assert result == f"{UPRN}_{STREET} {POSTCODE}_Floor Plan.pdf" + + +def test_no_recognised_prefix_preserves_stem() -> None: + # Arrange + original = "Completely Different Name.pdf" + + # Act + result = build_canonical_filename(UPRN, ADDRESS, POSTCODE, original) + + # Assert + assert result == f"{UPRN}_{STREET} {POSTCODE}_Completely Different Name.pdf" + + +def test_no_doc_name_after_strip_omits_trailing_separator() -> None: + # Arrange – stem is exactly the address prefix with no trailing doc name + original = f"{STREET} {POSTCODE}.pdf" + + # Act + result = build_canonical_filename(UPRN, ADDRESS, POSTCODE, original) + + # Assert + assert result == f"{UPRN}_{STREET} {POSTCODE}.pdf" From b3e9d858d9dbd2c20b83390f93366a4b013cf6b6 Mon Sep 17 00:00:00 2001 From: Daniel Roth Date: Mon, 15 Jun 2026 10:49:01 +0000 Subject: [PATCH 74/87] =?UTF-8?q?SharePoint=20renamer=20Lambda=20handler?= =?UTF-8?q?=20stub=20created=20=F0=9F=9F=A5?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- applications/sharepoint_renamer/__init__.py | 0 applications/sharepoint_renamer/handler/__init__.py | 0 2 files changed, 0 insertions(+), 0 deletions(-) create mode 100644 applications/sharepoint_renamer/__init__.py create mode 100644 applications/sharepoint_renamer/handler/__init__.py diff --git a/applications/sharepoint_renamer/__init__.py b/applications/sharepoint_renamer/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/applications/sharepoint_renamer/handler/__init__.py b/applications/sharepoint_renamer/handler/__init__.py new file mode 100644 index 00000000..e69de29b From 8cb0e986e65a669d1b384f93848bc8db6d0baa7e Mon Sep 17 00:00:00 2001 From: Daniel Roth Date: Mon, 15 Jun 2026 10:52:52 +0000 Subject: [PATCH 75/87] =?UTF-8?q?Deploy=20SharePoint=20renamer=20as=20Lamb?= =?UTF-8?q?da=20with=20SQS=20trigger=20=F0=9F=9F=A9?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .github/workflows/deploy_terraform.yml | 39 +++++++++++++ .../sharepoint_renamer/handler/Dockerfile | 16 ++++++ .../sharepoint_renamer/handler/handler.py | 7 +++ .../handler/requirements.txt | 2 + .../lambda/sharepoint_renamer/main.tf | 22 ++++++++ .../lambda/sharepoint_renamer/outputs.tf | 9 +++ .../lambda/sharepoint_renamer/provider.tf | 20 +++++++ .../lambda/sharepoint_renamer/variables.tf | 55 +++++++++++++++++++ deployment/terraform/shared/main.tf | 14 +++++ scripts/__init__.py | 0 scripts/rename_sharepoint_files.py | 2 +- 11 files changed, 185 insertions(+), 1 deletion(-) create mode 100644 applications/sharepoint_renamer/handler/Dockerfile create mode 100644 applications/sharepoint_renamer/handler/handler.py create mode 100644 applications/sharepoint_renamer/handler/requirements.txt create mode 100644 deployment/terraform/lambda/sharepoint_renamer/main.tf create mode 100644 deployment/terraform/lambda/sharepoint_renamer/outputs.tf create mode 100644 deployment/terraform/lambda/sharepoint_renamer/provider.tf create mode 100644 deployment/terraform/lambda/sharepoint_renamer/variables.tf create mode 100644 scripts/__init__.py diff --git a/.github/workflows/deploy_terraform.yml b/.github/workflows/deploy_terraform.yml index 338ef11d..0780c580 100644 --- a/.github/workflows/deploy_terraform.yml +++ b/.github/workflows/deploy_terraform.yml @@ -495,6 +495,45 @@ jobs: TF_VAR_pashub_coordination_password: ${{ secrets.PASHUB_COORDINATION_PASSWORD }} + # ============================================================ + # Build SharePoint Renamer image and Push + # ============================================================ + sharepoint_renamer_image: + needs: [determine_stage, shared_terraform] + uses: ./.github/workflows/_build_image.yml + with: + ecr_repo: sharepoint-renamer-${{ needs.determine_stage.outputs.stage }} + dockerfile_path: applications/sharepoint_renamer/handler/Dockerfile + build_context: . + secrets: + AWS_ACCESS_KEY_ID: ${{ secrets.DEV_AWS_ACCESS_KEY_ID }} + AWS_SECRET_ACCESS_KEY: ${{ secrets.DEV_AWS_SECRET_ACCESS_KEY }} + AWS_REGION: ${{ secrets.DEV_AWS_REGION }} + + + # ============================================================ + # Deploy SharePoint Renamer Lambda + # ============================================================ + sharepoint_renamer_lambda: + needs: [sharepoint_renamer_image, determine_stage] + uses: ./.github/workflows/_deploy_lambda.yml + with: + lambda_name: sharepoint_renamer + lambda_path: deployment/terraform/lambda/sharepoint_renamer + stage: ${{ needs.determine_stage.outputs.stage }} + ecr_repo: sharepoint-renamer-${{ needs.determine_stage.outputs.stage }} + image_digest: ${{ needs.sharepoint_renamer_image.outputs.image_digest }} + terraform_apply: ${{ needs.determine_stage.outputs.terraform_apply }} + secrets: + AWS_ACCESS_KEY_ID: ${{ secrets.DEV_AWS_ACCESS_KEY_ID }} + AWS_SECRET_ACCESS_KEY: ${{ secrets.DEV_AWS_SECRET_ACCESS_KEY }} + AWS_REGION: ${{ secrets.DEV_AWS_REGION }} + TF_VAR_sharepoint_client_id: ${{ secrets.SHAREPOINT_CLIENT_ID }} + TF_VAR_sharepoint_client_secret: ${{ secrets.SHAREPOINT_CLIENT_SECRET }} + TF_VAR_sharepoint_tenant_id: ${{ secrets.SHAREPOINT_TENANT_ID }} + TF_VAR_social_housing_wave_3_sharepoint_id: ${{ secrets.SOCIAL_HOUSING_WAVE_3_SHAREPOINT_ID }} + + # ============================================================ # Deploy FastAPI Lambda # ============================================================ diff --git a/applications/sharepoint_renamer/handler/Dockerfile b/applications/sharepoint_renamer/handler/Dockerfile new file mode 100644 index 00000000..10c40e89 --- /dev/null +++ b/applications/sharepoint_renamer/handler/Dockerfile @@ -0,0 +1,16 @@ +FROM public.ecr.aws/lambda/python:3.11 + +WORKDIR /var/task + +COPY applications/sharepoint_renamer/handler/requirements.txt . +RUN pip install --no-cache-dir -r requirements.txt + +COPY utils/ utils/ +COPY backend/__init__.py backend/__init__.py +COPY backend/pashub_fetcher/ backend/pashub_fetcher/ +COPY applications/sharepoint_renamer/ applications/sharepoint_renamer/ +COPY scripts/__init__.py scripts/__init__.py +COPY scripts/rename_sharepoint_files.py scripts/rename_sharepoint_files.py +COPY scripts/sero_address_list.csv scripts/sero_address_list.csv + +CMD ["applications.sharepoint_renamer.handler.handler.handler"] diff --git a/applications/sharepoint_renamer/handler/handler.py b/applications/sharepoint_renamer/handler/handler.py new file mode 100644 index 00000000..850d1ae6 --- /dev/null +++ b/applications/sharepoint_renamer/handler/handler.py @@ -0,0 +1,7 @@ +from typing import Any + +from scripts.rename_sharepoint_files import main + + +def handler(event: dict[str, Any], context: Any) -> None: + main() diff --git a/applications/sharepoint_renamer/handler/requirements.txt b/applications/sharepoint_renamer/handler/requirements.txt new file mode 100644 index 00000000..94317b81 --- /dev/null +++ b/applications/sharepoint_renamer/handler/requirements.txt @@ -0,0 +1,2 @@ +msal +requests diff --git a/deployment/terraform/lambda/sharepoint_renamer/main.tf b/deployment/terraform/lambda/sharepoint_renamer/main.tf new file mode 100644 index 00000000..0c245061 --- /dev/null +++ b/deployment/terraform/lambda/sharepoint_renamer/main.tf @@ -0,0 +1,22 @@ +module "lambda" { + source = "../../modules/lambda_with_sqs" + + name = "sharepoint_renamer" + stage = var.stage + + image_uri = local.image_uri + timeout = var.timeout + + reserved_concurrent_executions = var.reserved_concurrent_executions + + batch_size = var.batch_size + + environment = { + STAGE = var.stage + + SHAREPOINT_CLIENT_ID = var.sharepoint_client_id + SHAREPOINT_CLIENT_SECRET = var.sharepoint_client_secret + SHAREPOINT_TENANT_ID = var.sharepoint_tenant_id + SOCIAL_HOUSING_WAVE_3_SHAREPOINT_ID = var.social_housing_wave_3_sharepoint_id + } +} diff --git a/deployment/terraform/lambda/sharepoint_renamer/outputs.tf b/deployment/terraform/lambda/sharepoint_renamer/outputs.tf new file mode 100644 index 00000000..e71fac8b --- /dev/null +++ b/deployment/terraform/lambda/sharepoint_renamer/outputs.tf @@ -0,0 +1,9 @@ +output "sharepoint_renamer_queue_url" { + value = module.lambda.queue_url + description = "URL of the SharePoint Renamer SQS queue" +} + +output "sharepoint_renamer_queue_arn" { + value = module.lambda.queue_arn + description = "ARN of the SharePoint Renamer SQS queue" +} diff --git a/deployment/terraform/lambda/sharepoint_renamer/provider.tf b/deployment/terraform/lambda/sharepoint_renamer/provider.tf new file mode 100644 index 00000000..e6f8e32c --- /dev/null +++ b/deployment/terraform/lambda/sharepoint_renamer/provider.tf @@ -0,0 +1,20 @@ +terraform { + required_providers { + aws = { + source = "hashicorp/aws" + version = ">= 5.0" + } + } + + backend "s3" { + bucket = "sharepoint-renamer-terraform-state" + key = "terraform.tfstate" + region = "eu-west-2" + } + + required_version = ">= 1.2.0" +} + +provider "aws" { + region = "eu-west-2" +} diff --git a/deployment/terraform/lambda/sharepoint_renamer/variables.tf b/deployment/terraform/lambda/sharepoint_renamer/variables.tf new file mode 100644 index 00000000..97cca538 --- /dev/null +++ b/deployment/terraform/lambda/sharepoint_renamer/variables.tf @@ -0,0 +1,55 @@ +variable "stage" { + description = "Deployment stage (e.g. dev, prod)" + type = string +} + +variable "ecr_repo_url" { + type = string + description = "ECR repository URL (no tag, no digest)" +} + +variable "image_digest" { + type = string + description = "Image digest (sha256:...)" +} + +variable "timeout" { + type = number + default = 900 + description = "Lambda timeout in seconds." +} + +variable "reserved_concurrent_executions" { + type = number + default = 1 + description = "Prevent parallel renames causing race conditions on SharePoint." +} + +variable "batch_size" { + type = number + default = 1 +} + +variable "sharepoint_client_id" { + type = string + sensitive = true +} + +variable "sharepoint_client_secret" { + type = string + sensitive = true +} + +variable "sharepoint_tenant_id" { + type = string + sensitive = true +} + +variable "social_housing_wave_3_sharepoint_id" { + type = string + sensitive = true +} + +locals { + image_uri = "${var.ecr_repo_url}@${var.image_digest}" +} diff --git a/deployment/terraform/shared/main.tf b/deployment/terraform/shared/main.tf index 7ca116e7..3d6bbd39 100644 --- a/deployment/terraform/shared/main.tf +++ b/deployment/terraform/shared/main.tf @@ -844,3 +844,17 @@ module "audit_generator_registry" { stage = var.stage } +################################################ +# SharePoint Renamer – Lambda +################################################ +module "sharepoint_renamer_state_bucket" { + source = "../modules/tf_state_bucket" + bucket_name = "sharepoint-renamer-terraform-state" +} + +module "sharepoint_renamer_registry" { + source = "../modules/container_registry" + name = "sharepoint-renamer" + stage = var.stage +} + diff --git a/scripts/__init__.py b/scripts/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/scripts/rename_sharepoint_files.py b/scripts/rename_sharepoint_files.py index a7306d88..7ed126e3 100644 --- a/scripts/rename_sharepoint_files.py +++ b/scripts/rename_sharepoint_files.py @@ -17,7 +17,7 @@ from utils.sharepoint.domna_sharepoint_client import DomnaSharepointClient from utils.sharepoint.domna_sites import DomnaSites DRY_RUN: bool = False -CSV_PATH: str = "scripts/sero_address_list_test.csv" +CSV_PATH: str = "scripts/sero_address_list.csv" BASE_PATH = ( "Osmosis-ACD Projects/Sero-Clarion Housing/" From beb4e5d0d919744df6fb9b2263d8e579b3c53e93 Mon Sep 17 00:00:00 2001 From: Daniel Roth Date: Mon, 15 Jun 2026 11:01:51 +0000 Subject: [PATCH 76/87] Move SharePoint renamer logic from scripts/ into orchestrator and app-root handler --- applications/sharepoint_renamer/handler.py | 13 ++ .../sharepoint_renamer/handler/Dockerfile | 5 +- .../sharepoint_renamer/handler/handler.py | 7 - .../sharepoint_renamer_orchestrator.py | 113 +++++++++++++++ scripts/__init__.py | 0 scripts/rename_sharepoint_files.py | 137 ------------------ .../tests/test_build_canonical_filename.py | 2 +- 7 files changed, 129 insertions(+), 148 deletions(-) create mode 100644 applications/sharepoint_renamer/handler.py delete mode 100644 applications/sharepoint_renamer/handler/handler.py create mode 100644 orchestration/sharepoint_renamer_orchestrator.py delete mode 100644 scripts/__init__.py delete mode 100644 scripts/rename_sharepoint_files.py diff --git a/applications/sharepoint_renamer/handler.py b/applications/sharepoint_renamer/handler.py new file mode 100644 index 00000000..5a290878 --- /dev/null +++ b/applications/sharepoint_renamer/handler.py @@ -0,0 +1,13 @@ +from typing import Any + +from orchestration.sharepoint_renamer_orchestrator import SharepointRenamerOrchestrator +from utils.sharepoint.domna_sharepoint_client import DomnaSharepointClient +from utils.sharepoint.domna_sites import DomnaSites + +CSV_PATH = "scripts/sero_address_list.csv" + + +def handler(event: dict[str, Any], context: Any) -> None: + sp_client = DomnaSharepointClient(DomnaSites.SOCIAL_HOUSING_WAVE_3) + orchestrator = SharepointRenamerOrchestrator(sp_client, CSV_PATH) + orchestrator.run() diff --git a/applications/sharepoint_renamer/handler/Dockerfile b/applications/sharepoint_renamer/handler/Dockerfile index 10c40e89..bb946cc2 100644 --- a/applications/sharepoint_renamer/handler/Dockerfile +++ b/applications/sharepoint_renamer/handler/Dockerfile @@ -8,9 +8,8 @@ RUN pip install --no-cache-dir -r requirements.txt COPY utils/ utils/ COPY backend/__init__.py backend/__init__.py COPY backend/pashub_fetcher/ backend/pashub_fetcher/ +COPY orchestration/ orchestration/ COPY applications/sharepoint_renamer/ applications/sharepoint_renamer/ -COPY scripts/__init__.py scripts/__init__.py -COPY scripts/rename_sharepoint_files.py scripts/rename_sharepoint_files.py COPY scripts/sero_address_list.csv scripts/sero_address_list.csv -CMD ["applications.sharepoint_renamer.handler.handler.handler"] +CMD ["applications.sharepoint_renamer.handler.handler"] diff --git a/applications/sharepoint_renamer/handler/handler.py b/applications/sharepoint_renamer/handler/handler.py deleted file mode 100644 index 850d1ae6..00000000 --- a/applications/sharepoint_renamer/handler/handler.py +++ /dev/null @@ -1,7 +0,0 @@ -from typing import Any - -from scripts.rename_sharepoint_files import main - - -def handler(event: dict[str, Any], context: Any) -> None: - main() diff --git a/orchestration/sharepoint_renamer_orchestrator.py b/orchestration/sharepoint_renamer_orchestrator.py new file mode 100644 index 00000000..764776ae --- /dev/null +++ b/orchestration/sharepoint_renamer_orchestrator.py @@ -0,0 +1,113 @@ +import csv +import logging +import os +from typing import Optional + +from backend.pashub_fetcher.sharepoint_subfolders import SharepointSubfolders +from utils.sharepoint.domna_sharepoint_client import DomnaSharepointClient + +BASE_PATH = ( + "Osmosis-ACD Projects/Sero-Clarion Housing/" + "Sero Project Documents/Property Folders" +) +ASSESSMENT_SUBFOLDER = "A. Assessment" + +logger = logging.getLogger(__name__) + + +def build_canonical_filename( + uprn: str, address: str, postcode: str, original_name: str +) -> Optional[str]: + """ + Returns the canonical filename, or None if the file is already renamed. + + Already-renamed: name starts with "{uprn}_". + Strips any existing address prefix (address+postcode first, then address alone) + before inserting the canonical prefix. + """ + if original_name.startswith(f"{uprn}_"): + return None + + stem, ext = os.path.splitext(original_name) + stem_lower = stem.lower() + + street = address.split(",")[0].strip() + prefixes = [ + f"{address} {postcode}", + address, + f"{street} {postcode}", + street, + ] + + doc_name = stem + for prefix in prefixes: + if stem_lower.startswith(prefix.lower()): + doc_name = stem[len(prefix) :] + break + + if doc_name.startswith(" - "): + doc_name = doc_name[3:] + elif doc_name.startswith(" _ "): + doc_name = doc_name[3:] + doc_name = doc_name.strip() + + street_post = f"{street} {postcode}" + if doc_name: + return f"{uprn}_{street_post}_{doc_name}{ext}" + return f"{uprn}_{street_post}{ext}" + + +class SharepointRenamerOrchestrator: + def __init__(self, sp_client: DomnaSharepointClient, csv_path: str) -> None: + self._sp_client = sp_client + self._csv_path = csv_path + + def run(self) -> None: + with open(self._csv_path, newline="", encoding="utf-8-sig") as f: + reader = csv.DictReader(f) + required = {"UPRN", "Address", "Postcode"} + if not reader.fieldnames or not required.issubset(set(reader.fieldnames)): + raise ValueError( + f"CSV missing required columns. Expected {required}, got {reader.fieldnames}" + ) + + for row in reader: + uprn = row["UPRN"].strip() + address = row["Address"].strip() + postcode = row["Postcode"].strip() + folder_path = ( + f"{BASE_PATH}/{address}, {postcode}" + f"/{SharepointSubfolders.ASSESSMENT.value}/{ASSESSMENT_SUBFOLDER}" + ) + self._process_folder(folder_path, uprn, address, postcode) + + def _process_folder( + self, folder_path: str, uprn: str, address: str, postcode: str + ) -> None: + try: + contents = self._sp_client.get_folders_in_path(folder_path) + except ValueError: + logger.warning(f"Missing folder for UPRN {uprn}: {folder_path}") + return + + for item in contents.get("value", []): + if "folder" in item: + self._process_folder( + f"{folder_path}/{item['name']}", uprn, address, postcode + ) + elif "file" in item: + original_name: str = item["name"] + new_name = build_canonical_filename(uprn, address, postcode, original_name) + + if new_name is None: + continue + + try: + self._sp_client.rename_file(item["id"], new_name) + logger.info( + f'Renamed: "{original_name}" → "{new_name}" (UPRN: {uprn})' + ) + except Exception as e: + logger.error( + f'Failed to rename "{original_name}" → "{new_name}" (UPRN: {uprn}): {e}' + ) diff --git a/scripts/__init__.py b/scripts/__init__.py deleted file mode 100644 index e69de29b..00000000 diff --git a/scripts/rename_sharepoint_files.py b/scripts/rename_sharepoint_files.py deleted file mode 100644 index 7ed126e3..00000000 --- a/scripts/rename_sharepoint_files.py +++ /dev/null @@ -1,137 +0,0 @@ -""" -Rename files in SharePoint property folders to the canonical format: - {UPRN}_{Street} {Postcode}_{Document Name}.ext - -Set DRY_RUN = False when ready to commit. Run from repo root. -Required env vars: SHAREPOINT_CLIENT_ID, SHAREPOINT_CLIENT_SECRET, - SHAREPOINT_TENANT_ID, SOCIAL_HOUSING_WAVE_3_SHAREPOINT_ID -""" - -import csv -import os -from typing import Optional - -from backend.pashub_fetcher.sharepoint_subfolders import SharepointSubfolders -from utils.logger import setup_logger -from utils.sharepoint.domna_sharepoint_client import DomnaSharepointClient -from utils.sharepoint.domna_sites import DomnaSites - -DRY_RUN: bool = False -CSV_PATH: str = "scripts/sero_address_list.csv" - -BASE_PATH = ( - "Osmosis-ACD Projects/Sero-Clarion Housing/" - "Sero Project Documents/Property Folders" -) -ASSESSMENT_SUBFOLDER = "A. Assessment" - -logger = setup_logger() - - -def build_canonical_filename( - uprn: str, address: str, postcode: str, original_name: str -) -> Optional[str]: - """ - Returns the canonical filename, or None if the file is already renamed. - - Already-renamed: name starts with "{uprn}_". - Strips any existing address prefix (address+postcode first, then address alone) - before inserting the canonical prefix. - """ - if original_name.startswith(f"{uprn}_"): - return None - - stem, ext = os.path.splitext(original_name) - stem_lower = stem.lower() - - street = address.split(",")[0].strip() - prefixes = [ - f"{address} {postcode}", - address, - f"{street} {postcode}", - street, - ] - - doc_name = stem - for prefix in prefixes: - if stem_lower.startswith(prefix.lower()): - doc_name = stem[len(prefix) :] - break - - if doc_name.startswith(" - "): - doc_name = doc_name[3:] - elif doc_name.startswith(" _ "): - doc_name = doc_name[3:] - doc_name = doc_name.strip() - - street_post = f"{street} {postcode}" - if doc_name: - return f"{uprn}_{street_post}_{doc_name}{ext}" - return f"{uprn}_{street_post}{ext}" - - -def process_folder( - sp_client: DomnaSharepointClient, - folder_path: str, - uprn: str, - address: str, - postcode: str, -) -> None: - try: - contents = sp_client.get_folders_in_path(folder_path) - except ValueError: - logger.warning(f"Missing folder for UPRN {uprn}: {folder_path}") - return - - for item in contents.get("value", []): - if "folder" in item: - process_folder( - sp_client, f"{folder_path}/{item['name']}", uprn, address, postcode - ) - elif "file" in item: - original_name: str = item["name"] - new_name = build_canonical_filename(uprn, address, postcode, original_name) - - if new_name is None: - continue - - if DRY_RUN: - logger.info( - f'[DRY RUN] Renaming: "{original_name}" → "{new_name}" (UPRN: {uprn})' - ) - else: - try: - sp_client.rename_file(item["id"], new_name) - logger.info( - f'Renamed: "{original_name}" → "{new_name}" (UPRN: {uprn})' - ) - except Exception as e: - logger.error( - f'Failed to rename "{original_name}" → "{new_name}" (UPRN: {uprn}): {e}' - ) - - -def main() -> None: - sp_client = DomnaSharepointClient(DomnaSites.SOCIAL_HOUSING_WAVE_3) - - with open(CSV_PATH, newline="", encoding="utf-8-sig") as f: - reader = csv.DictReader(f) - required = {"UPRN", "Address", "Postcode"} - if not reader.fieldnames or not required.issubset(set(reader.fieldnames)): - raise ValueError( - f"CSV missing required columns. Expected {required}, got {reader.fieldnames}" - ) - - for row in reader: - uprn = row["UPRN"].strip() - address = row["Address"].strip() - postcode = row["Postcode"].strip() - folder_path = ( - f"{BASE_PATH}/{address}, {postcode}" - f"/{SharepointSubfolders.ASSESSMENT.value}/{ASSESSMENT_SUBFOLDER}" - ) - process_folder(sp_client, folder_path, uprn, address, postcode) - - -if __name__ == "__main__": - main() diff --git a/scripts/tests/test_build_canonical_filename.py b/scripts/tests/test_build_canonical_filename.py index 3890477c..67d4fcae 100644 --- a/scripts/tests/test_build_canonical_filename.py +++ b/scripts/tests/test_build_canonical_filename.py @@ -1,5 +1,5 @@ # scripts/tests/test_build_canonical_filename.py -from scripts.rename_sharepoint_files import build_canonical_filename +from orchestration.sharepoint_renamer_orchestrator import build_canonical_filename UPRN = "10093456789" ADDRESS = "1 High Street, Anytown" From 38b9e6384446100d82420689861ceb90de34dbd1 Mon Sep 17 00:00:00 2001 From: Daniel Roth Date: Mon, 15 Jun 2026 11:02:48 +0000 Subject: [PATCH 77/87] revert pytest.ini --- pytest.ini | 2 -- 1 file changed, 2 deletions(-) diff --git a/pytest.ini b/pytest.ini index a6eba3be..2bcd6178 100644 --- a/pytest.ini +++ b/pytest.ini @@ -25,7 +25,5 @@ testpaths = etl/epc_clean/tests etl/hubspot/tests etl/spatial/tests - scripts/tests - ; tests/ markers = integration: mark a test as an integration test From 5c314e2914ae4fdf2e28d07a5605783348bf1e24 Mon Sep 17 00:00:00 2001 From: Daniel Roth Date: Mon, 15 Jun 2026 11:11:08 +0000 Subject: [PATCH 78/87] move tests out of scripts/ --- pytest.ini | 1 + scripts/tests/__init__.py | 0 .../orchestration}/test_build_canonical_filename.py | 0 3 files changed, 1 insertion(+) delete mode 100644 scripts/tests/__init__.py rename {scripts/tests => tests/orchestration}/test_build_canonical_filename.py (100%) diff --git a/pytest.ini b/pytest.ini index 2bcd6178..cb6af047 100644 --- a/pytest.ini +++ b/pytest.ini @@ -25,5 +25,6 @@ testpaths = etl/epc_clean/tests etl/hubspot/tests etl/spatial/tests + tests/ markers = integration: mark a test as an integration test diff --git a/scripts/tests/__init__.py b/scripts/tests/__init__.py deleted file mode 100644 index e69de29b..00000000 diff --git a/scripts/tests/test_build_canonical_filename.py b/tests/orchestration/test_build_canonical_filename.py similarity index 100% rename from scripts/tests/test_build_canonical_filename.py rename to tests/orchestration/test_build_canonical_filename.py From 0fc81da4cf3108c6a0f267c47f9e5987055d1b08 Mon Sep 17 00:00:00 2001 From: Daniel Roth Date: Mon, 15 Jun 2026 11:14:09 +0000 Subject: [PATCH 79/87] move input files out of scripts/ --- applications/sharepoint_renamer/handler.py | 2 +- applications/sharepoint_renamer/handler/Dockerfile | 2 -- .../sharepoint_renamer}/sero_address_list.csv | 0 applications/sharepoint_renamer/sero_address_list_test.csv | 2 ++ 4 files changed, 3 insertions(+), 3 deletions(-) rename {scripts => applications/sharepoint_renamer}/sero_address_list.csv (100%) create mode 100644 applications/sharepoint_renamer/sero_address_list_test.csv diff --git a/applications/sharepoint_renamer/handler.py b/applications/sharepoint_renamer/handler.py index 5a290878..998458bc 100644 --- a/applications/sharepoint_renamer/handler.py +++ b/applications/sharepoint_renamer/handler.py @@ -4,7 +4,7 @@ from orchestration.sharepoint_renamer_orchestrator import SharepointRenamerOrche from utils.sharepoint.domna_sharepoint_client import DomnaSharepointClient from utils.sharepoint.domna_sites import DomnaSites -CSV_PATH = "scripts/sero_address_list.csv" +CSV_PATH = "applications/sharepoint_renamer/sero_address_list.csv" def handler(event: dict[str, Any], context: Any) -> None: diff --git a/applications/sharepoint_renamer/handler/Dockerfile b/applications/sharepoint_renamer/handler/Dockerfile index bb946cc2..a81294f9 100644 --- a/applications/sharepoint_renamer/handler/Dockerfile +++ b/applications/sharepoint_renamer/handler/Dockerfile @@ -10,6 +10,4 @@ COPY backend/__init__.py backend/__init__.py COPY backend/pashub_fetcher/ backend/pashub_fetcher/ COPY orchestration/ orchestration/ COPY applications/sharepoint_renamer/ applications/sharepoint_renamer/ -COPY scripts/sero_address_list.csv scripts/sero_address_list.csv - CMD ["applications.sharepoint_renamer.handler.handler"] diff --git a/scripts/sero_address_list.csv b/applications/sharepoint_renamer/sero_address_list.csv similarity index 100% rename from scripts/sero_address_list.csv rename to applications/sharepoint_renamer/sero_address_list.csv diff --git a/applications/sharepoint_renamer/sero_address_list_test.csv b/applications/sharepoint_renamer/sero_address_list_test.csv new file mode 100644 index 00000000..72b28047 --- /dev/null +++ b/applications/sharepoint_renamer/sero_address_list_test.csv @@ -0,0 +1,2 @@ +UPRN,Address,Postcode +U1014630,"118 Faringdon Avenue, Bromley",BR2 8BU \ No newline at end of file From a6050fc1c7b60d58be541a5355c5daa8d7d65257 Mon Sep 17 00:00:00 2001 From: Daniel Roth Date: Mon, 15 Jun 2026 12:04:33 +0000 Subject: [PATCH 80/87] remove tests/ from pytest.ini --- pytest.ini | 1 - 1 file changed, 1 deletion(-) diff --git a/pytest.ini b/pytest.ini index cb6af047..2bcd6178 100644 --- a/pytest.ini +++ b/pytest.ini @@ -25,6 +25,5 @@ testpaths = etl/epc_clean/tests etl/hubspot/tests etl/spatial/tests - tests/ markers = integration: mark a test as an integration test From b9cbea367db3057365e2c654f89645e1a8ff3c22 Mon Sep 17 00:00:00 2001 From: Daniel Roth Date: Mon, 15 Jun 2026 12:21:32 +0000 Subject: [PATCH 81/87] correct import in test file --- tests/scripts/test_rename_sharepoint_files.py | 59 +++++++------------ 1 file changed, 22 insertions(+), 37 deletions(-) diff --git a/tests/scripts/test_rename_sharepoint_files.py b/tests/scripts/test_rename_sharepoint_files.py index 4525fe84..7b3e6587 100644 --- a/tests/scripts/test_rename_sharepoint_files.py +++ b/tests/scripts/test_rename_sharepoint_files.py @@ -1,10 +1,12 @@ from typing import Any -from unittest.mock import MagicMock, call, patch +from unittest.mock import MagicMock import pytest -import scripts.rename_sharepoint_files as module -from scripts.rename_sharepoint_files import build_canonical_filename, process_folder +from orchestration.sharepoint_renamer_orchestrator import ( + SharepointRenamerOrchestrator, + build_canonical_filename, +) def _make_file(name: str, item_id: str = "id-1") -> dict[str, Any]: @@ -19,6 +21,12 @@ def _make_package(name: str) -> dict[str, Any]: return {"name": name, "package": {}} +def _make_orchestrator(sp: MagicMock) -> SharepointRenamerOrchestrator: + orchestrator = SharepointRenamerOrchestrator.__new__(SharepointRenamerOrchestrator) + orchestrator._sp_client = sp + return orchestrator + + # --------------------------------------------------------------------------- # build_canonical_filename # --------------------------------------------------------------------------- @@ -39,7 +47,7 @@ def test_no_prefix_still_canonical() -> None: # --------------------------------------------------------------------------- -# process_folder — files only at root level +# _process_folder — files only at root level # --------------------------------------------------------------------------- @@ -52,8 +60,7 @@ def test_renames_top_level_files(caplog: pytest.LogCaptureFixture) -> None: ] } - with patch.object(module, "DRY_RUN", False): - process_folder(sp, "some/path", "100", "1 High St", "AB1 2CD") + _make_orchestrator(sp)._process_folder("some/path", "100", "1 High St", "AB1 2CD") assert sp.rename_file.call_count == 2 sp.rename_file.assert_any_call("id-1", "100_1 High St AB1 2CD_Survey.pdf") @@ -61,7 +68,7 @@ def test_renames_top_level_files(caplog: pytest.LogCaptureFixture) -> None: # --------------------------------------------------------------------------- -# process_folder — recursive two-level hierarchy +# _process_folder — recursive two-level hierarchy # --------------------------------------------------------------------------- @@ -84,8 +91,7 @@ def test_recurses_into_subfolders_and_renames_all_files() -> None: root_contents if path == "base/path" else suba_contents ) - with patch.object(module, "DRY_RUN", False): - process_folder(sp, "base/path", "200", "2 Main Rd", "XY9 8ZW") + _make_orchestrator(sp)._process_folder("base/path", "200", "2 Main Rd", "XY9 8ZW") assert sp.rename_file.call_count == 2 sp.rename_file.assert_any_call("root-file", "200_2 Main Rd XY9 8ZW_Root.pdf") @@ -95,25 +101,22 @@ def test_recurses_into_subfolders_and_renames_all_files() -> None: # --------------------------------------------------------------------------- -# process_folder — non-file, non-folder items are skipped +# _process_folder — non-file, non-folder items are skipped # --------------------------------------------------------------------------- def test_ignores_package_items() -> None: sp = MagicMock() - sp.get_folders_in_path.return_value = { - "value": [_make_package("Notebook")] - } + sp.get_folders_in_path.return_value = {"value": [_make_package("Notebook")]} - with patch.object(module, "DRY_RUN", False): - process_folder(sp, "some/path", "300", "3 Oak Ave", "ZZ1 1ZZ") + _make_orchestrator(sp)._process_folder("some/path", "300", "3 Oak Ave", "ZZ1 1ZZ") sp.rename_file.assert_not_called() assert sp.get_folders_in_path.call_count == 1 # --------------------------------------------------------------------------- -# process_folder — missing folder +# _process_folder — missing folder # --------------------------------------------------------------------------- @@ -121,31 +124,14 @@ def test_missing_folder_logs_warning_and_returns(caplog: pytest.LogCaptureFixtur sp = MagicMock() sp.get_folders_in_path.side_effect = ValueError("not found") - with patch.object(module, "DRY_RUN", False): - process_folder(sp, "missing/path", "400", "4 Elm St", "AA2 2BB") + _make_orchestrator(sp)._process_folder("missing/path", "400", "4 Elm St", "AA2 2BB") sp.rename_file.assert_not_called() assert any("Missing folder" in r.message and "400" in r.message for r in caplog.records) # --------------------------------------------------------------------------- -# process_folder — dry run -# --------------------------------------------------------------------------- - - -def test_dry_run_logs_without_renaming(caplog: pytest.LogCaptureFixture) -> None: - sp = MagicMock() - sp.get_folders_in_path.return_value = {"value": [_make_file("Doc.pdf", "id-x")]} - - with patch.object(module, "DRY_RUN", True): - process_folder(sp, "some/path", "500", "5 Pine Ln", "BB3 3CC") - - sp.rename_file.assert_not_called() - assert any("[DRY RUN]" in r.message for r in caplog.records) - - -# --------------------------------------------------------------------------- -# process_folder — already-canonical files are skipped +# _process_folder — already-canonical files are skipped # --------------------------------------------------------------------------- @@ -155,7 +141,6 @@ def test_skips_already_canonical_files() -> None: "value": [_make_file("500_Pine Ln BB3 3CC_Doc.pdf", "id-y")] } - with patch.object(module, "DRY_RUN", False): - process_folder(sp, "some/path", "500", "5 Pine Ln", "BB3 3CC") + _make_orchestrator(sp)._process_folder("some/path", "500", "5 Pine Ln", "BB3 3CC") sp.rename_file.assert_not_called() From 4fdc23f83d6587b51b76ca91d3b02927659408a4 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 15 Jun 2026 13:31:36 +0000 Subject: [PATCH 82/87] =?UTF-8?q?test(worksheet):=20pin=20simulated=20case?= =?UTF-8?q?=2038=20=E2=80=94=20mains-gas=20secondary=20reproduces=20worksh?= =?UTF-8?q?eet=20exactly?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The realistic re-generation of case 37 (code-117 gas boiler, control 2102, + a MAINS-GAS condensing gas-fire secondary code 611, vs case 37's biogas 605). The full extractor -> mapper -> calculator pipeline reproduces the worksheet's SAP-rating block EXACTLY: continuous SAP 60.9152 (Δ 2e-5) and (272) CO2 5801.0770 (Δ ~0). This confirms the boiler-efficiency / control-2102 −5pp interlock / secondary-fuel handling are all correct, and that case 37's +7 gap was purely the biogas sub-fuel the Summary export cannot carry. Summary mirrored into backend/documents_parser/tests/fixtures so the pin runs without the unstaged workspace. PE not pinned — it is a separate DPER block (different scope) already guarded by the corpus PE gauge. Worksheet harness 47/47 unchanged; pyright net-zero. Co-Authored-By: Claude Opus 4.8 --- .../tests/fixtures/Summary_001431_case38.pdf | Bin 0 -> 84371 bytes .../_elmhurst_worksheet_001431_case38.py | 116 ++++++++++++++++++ 2 files changed, 116 insertions(+) create mode 100644 backend/documents_parser/tests/fixtures/Summary_001431_case38.pdf create mode 100644 tests/domain/sap10_calculator/worksheet/_elmhurst_worksheet_001431_case38.py diff --git a/backend/documents_parser/tests/fixtures/Summary_001431_case38.pdf b/backend/documents_parser/tests/fixtures/Summary_001431_case38.pdf new file mode 100644 index 0000000000000000000000000000000000000000..7b4cbcae212a62d2ab0aed4325f980fdeb81e0a7 GIT binary patch literal 84371 zcmeFa1yo#Jwx}CHf(9oc1Pe}Z5AH6(Dp;X~yF0;yy9G_~AO$421$PMqcXxMx#ecf{ zoYUuye)sSDZjU$4*_E+bd+#-u>|Jx&T={BJD2j^HGc&LuGm|ipSnHed@-nKp+CUgZ z^c?gotc@8J^^76*BrLEqj1cGP!xsECv$tOkrAQ$q(65*8*V zMoEaNv55l-GbuB3?>HMH8C@fr#Zupu|-{FxkUJyCT>&&OGHySmb zd>I=xD;RCa0Bfu&sg7;Cc|Kw}6$^MOFj`ph?OavO=yh^;w%>pFv@UkY^WM3xo*FB& zTGx@GHxY&fB?_-Iys!0uvPVnS?-FnmT74A8-~aJUDHsPFT)+E`n^3K<2^?MHaFWt1 zk$q3^{d=!1L4@QZQ_k}#LnZCnY2P4h8OGJ1rbZixtLL&MXl&7DVRdQ>(e;wWJ8->q z2|9gvkwI&|+BCPd)9pfL2$)>tt`a+^d*v-Q@6hkvjgIdx@3=GF5+B}Q6qhVXDRYM3 zsA!?q<>I4l+f;RCtZ-%&%+*4 z{D=Gh!dtW=JCs-{WfNO#Mux1lWw@dHdyTsU4cIPE>yahYUb$WKbap7`rUw_?ZPpwf z47W%dnj2Gvo-*s%IH5OPTbKnZU2nYbnq6x@{jk$~9NnZci)y-vC7-A^=Q7M%b7IEn z7ez?&ZP@gJ+gk@pnJJSICijWsDz9;K5`Qr#(cJuH2e#|e`gh|tpnZ=Nhuk7$&zN=% z8o|!ns}|fQ%MwH(qheVVv~0p-)AXrFZ6xq&?Lxc zSdF7WlAD{f9sE^mqZiFlf~MIC&3e5W&146RTW-eAA32O7L8THqVic%o^+&F&j`whK zs%>932r^`e!WtcRrs))&@2}lYi?QJ;lG4<{v)ivwp`yBOpV-){hWMCOrzeZy>Z7p5 zeLNF$_w3EVUt}h)JRHiN)s75rX|Po`ybN6=W|}-XBjR@M28+3Ju`S(u3N)%1S{Ieo zUmbXOO>ca-KY6<&8eL08$f*SO3dq-n2M)!l{Ja%JEcbk+CHdIySbN(RzPX z-*wcZgNx`m>GZnpLsIZz$%zV>z?P2C#R zjY7=C=;CD)SnEdj<%;~aX&L+Pg z(xG<4qGDP3D;Dqp@;9+eyOFhcERD}fw%Z2xRdGy8xuSN0Rh|xDA%St7=LGwG2(`KI zuQJ2$vXuvvjEq|77A{r`ul9|cD}prA#wQ3*+JCsVmk`W)3>)-B^52wqIIW{^URMs> z)82wz^N)K(Rb+5`u}Z&3n6_YJUFYRe#76z-(YekwR>r&DJ4W`!r?eqp%~+GrBEI(ecGe4yExL!eqW9%Q0aMQBL?mdg z4#RfSGZZpEH6!fJLuul4QfoS&e%_jso*-JqQ+LW-1f%tL;+^+ZC8TIz%g$vQq{)jP z0+w$u?W7nLC!fQM@HeVtuSnHAN=Jg;?K{PS>GDN3E10*R)y_r8b95t)G*=3!seHdY z9JxY3W^>X89j>bm^Oz*YXh`aevI~{M?SB0gPzsI%+iRC~{@&NIo2uW+NQb=oL`wL> zk^4z+VNWG7v!`zLdo$vfeoUq>UmQ(lbL_ecdm9~J?+oh4@TuC-V?bl(%#xmyt~!+G zYvs>yP*>jz(mWoiO{cGKu{gQTIFX42ougiJnRq2Q+p-8&4zL!9XIWep897#-ReBeX zRaN>Ph1fu?hss-+?7vPI7>f_7dOLeWbEEgm2i#HJqFxWc59S(m;On3d41}`rHZd`= zBEC`=5TR4rH++Zq#ooW@z>K-P2XCO{{hLWyaZYYr1`245`*z9U(pYB#J8`OTi#G{f$$BpHj}-DXFlY>=df3_8~*hv#nCLN{U-|EN$)ed_Jb4=e{2VYQzVH^rBI zFDORXGfwH(a54sy(Qals6Mt9qT&448a2N%Ee9%K1UmCU)qR5@!q7Z-99GWi|K!p+~ z!$xm~CJm0sC#2Sz;b~JFS~)Q=HQ_ut8+UNnEI2g1w?0-yzfL`I*F2}3d-MVPWnP!r z2RBaFCHbk=tH$meP=(@g9Fa?ULVMS zBS8F@D4#Va`-=GDj~IA;8q`wh!DV@shJ_)s68!HKkJQCZH;1Kf{Nyf1l8@c{>&dFRLIljE~FAbEtm9=bK?5Iu#WYi^?34k zSWG`G;pPBip>RFZhv>Ub8g-7(E;&0Wuy5#^P%Lz}O&wE%GA&PkR8poQsXm-(9R;B}2g#jhiOf}6Tz9=#dAgQ+c%Y=}o<~E~ zTe#|Y`ZYg2!pr`pjf5V=e3# znHfm$UE0&DW!`j)Bf8hJhubIZsN|=f*w+u28F1b9rgP~(=eWrGK5mWAR;ZD{|D3@l zZ}Uk`#4p<#!7+TDA*rZGJi?69je5H-?O!cd;IDJOPEWh7RZx4YM9L|4$IMi+TR-31 z%iUFxoPGv%HRDG>!`e{U1UfRAc~yUB5gAi zy4Jax70ht_6(26A#-B8t*N%I|LSUGf&br(CY)6GU1JCF-^p3>^HkU!;L0?WA9zS&+ zh*O=+SoaBh(rB&w~kbKL(<2&@~LpwHn%NlM5M+aL_nKTvnDZqa#N?`qwdesYcczaz=J(eeF7WMORw8U7b9BbbTlJ|mXP>FBc*$HmpKRrSG$DfyU zuTpfzW{`1|JBVO&+R3R}wj1wvgib@QM@2#66j9Uug72s2n98>aW`#)d>NQSWM){cI zLt9u8XFvDp|71b*IsXhwhX!Y2uD`ZeaDfmbO$m_@Oj{AZq_gt$X!!1Km3;WM`v}>r z-i8qI*1*MUaqD>`tV?-57=-I+cz24Nj$g^!2HnVuzy-}y zDozh0(CGxZ4VubuvGF8Y3d5FY9M}mgU%n7V+doCyT6?TEDBb6;EncDPuM#xL5|;Ei+NgsrAl zhUs!dPYgRZxV{C=;d%PbVKA16Roy^&7obI1j%?EB$!9{~MmvLmRosABNjq?Gg!bE+ zLqgk3$J*l$6ee#EL+T@i!{4(Ib2}sTCmv@M&(?VwTs7wNYU5#F{HPW#$U8Xox%blc z?sQ;8@*d5i@N#J+$svfW-Q0^4(J1|h%jke(YUQa#f;4tJ@XVR2N~;beJWd)rM}>Fp z0okWs$X4nKzQID3Ix-S;rJX(_L`jl$yt1if#J8m(YL#~B!@x{kg?1lp$-0h8B!+ms zAVhW*D$1Py+WOw!j>pe7=sxEwo*Eg_J}Z)dSNZR+TSDT$bb5&@vILe7xk7wzP)(Z! zyYZu*Lx;kTVwEYz`JqJ=r`IN6e)0TI?}VCBtavwt)~V{RO&lEGSL5M0-`u|+f95m8 zaYUAb?S)x^=#LqX6S20ks8COknwB->95hgSVt};g#b1&J7b`p%`53xcfka|+j4w;%K*vBG|deaKKXK5=d%)}rW5}) zCHuap=}mfRJz0$p9L={loAP|Zd3--@=g4{fcY&aT@HXrv+|S}B7^sV|+A5TUmoj^9?^DWXhzeve5yq$AYc(domW(9v$oDb4|i(h9W zlg*0s&rYgV2S;;SaAfD&{3?*u>-PNK@a5c?mSL~%*6{T-f68a1*A4+VaiEn2$lzvw z^CjKhX&-m1_EZL)tXrLf<|HBx-Fg{mI=+~R#ysd=?M#HaJRS6F`iq}cS{DuBv`sCP z9t96`Q9chYpu@=6{%Yv{{Xv?Guoa!d`FNDS3x8(7?uE>0e-yXIV>u~(jIw$%TwgSZ zFu&@z8r`L~WAm6S-KF?Van^p4a`W;Nm-eZYY?+I?734O$m{(UOWMPa9-K{|`6qK*w zS4iF$r9=r6UC5^td!cFfnF-IlXmLbs`cN?(PSaf<{hbI5NX&d5TpV#n;)#mouaMS_e% z@%p2Og7sg^I?iZ4U)O+wLF1?@(G^!KUIL2SvJ_dkOOD96*^zE$5mH}cq>CI#UMe4{ z+UC>sWWNz)FWB7)9Ei;AMj|_LI#=H4YwFqpKNvV_cVE$mt&)#mt?}HosWrxL&f&|*q=ZN*WkC%u3N^Grs-u| z^Ni0fqXr-FbE3pqpdMo{O{zV~3ev!Q(B!Rx8w}TN-yhMAF@oD)+w^+7y4nb1#WD{< zu7nPYSDuz5I_4R&Wqj5@-*J0=k>dB1wFYV(i>shBgmallSTE|)%7}~sH7Z)gJq%H8 zkgaDauG+4@%Bxyx6AvOC{1FnZ-%aOXU_|5htBvjibt>l^z2@v{5)^NY zveRFWS5og&-(eK{#BWn|3=KB__D4WvP(u{t?3s#t4sCO*E{7u>nHp0>i5);=4u>Z3D9$Wl}%tWvz~!=#j14ersAN@XaoB`cl4%Y4+`zqNIJZ5G~= z>nVaMC=@h^w-#pqxO>j?Bmwg(e2hZ3NXT{=gd3B-MIrm1n9&$6 zOkHCbgyo-J&dJ>agbywfQn{$l(lX^%7-E~n5;@+GAl41mhEd@qcU~8~L zbYP|heSuyozUF5lS$#Zq4@(v{KauXBLo#10k?g^j!k_dPB7+iLV*6t!%Bhroyf&Ep zA#8p{$IpRfQ=du-~1gvS)W zkT6IbqMrWv?d&Az*F4ERLPqO0xDG|3i@uuUk3)?mt{AR!8>}JZ?A>lECkWr$-Q5#t z=5>_)j);WAtT=CHXLEp9%v$PhFSQ95>L)_7QCIea&HNfs)7fS4v4M`jyztp)4_YaX zknWqZVC&U`Ip4VBNCrGywkqK&9A?i0Ew$0i#+J$eM-jXdFSq7qA%woiNxG#90n8VW ze)?HtY@WQKmC0G~-%W6WW)|-sOQP33LWY^R|6&IIi?N&SpEFOxEY*L`Jk9(M%+s7~ z>`ed3JdNn^YCL{4-EuNcuH3zzJJ@-=fk!^oFfw*RX@|=yW`?3SfnGhJ;T&15)cHM5 z9ddzi8E2_a12*}Gp9olRJVK?PvM1_F6JH=bevJBT7%!CV{KiiN{jTqx)&Ac7{1zel z(-U}P9BRc(L7)G6(&87KoTC|9$jYOQg8ZC->vmF|PYs1`nMWHF`8m2;qUh77B4w6l zQnOTbR5DJr?{J7PX432V*vV=4gKL}b+0wmz*>TG2YfDGszm~2uv#_x6ku9Cz&1_uM zN-o6x6i6O5=V% zM^#o{9vRMeaq&xz?R$?F zMdO3V*A$>7axgkmUq72%Vl7!t`TBUg9rqiGZwqBIRs!^8$80NOC&NSXpoEt6=97~j6G$a`CN3t9!fhtU&&+SlCo>V7>Bw!~=E(3B zYp#!Zbjpt>w#@yUm`G{^&)NqBtaDj^(*6?8Rxw=b5c6&jy`N*>A)aQ(sR;H-F8D%< z1QnO)pp?LG;g&GC=nO#^drwz2AMvX=qf3B3sbF?9M0sK~DI+71L(QKKXBID~vi5?b zQGbZc3cKP!wMD^48FVXhpi8F3cW8`pDmR2A1Mwhz$4 zr#drO6=r;?B$XdOl@tHVfa^P3{m8kHp0iNxx-V4_zSxI}$&MS(pTz`H1UhmdqTyB!%N5l~yNq*Gtd}T$wgJ+|u zImym`dwZKXwy}|qT9IpAe1h!CfqQSCb{=27uRUhCdZG?`*06lJMVTWoLNhhal2CD5 z^$65hT57XC;K`Xd^!51oSVl(Xye_jLVs&*jVQo!=qpqw zs0wS{+%lLz4V;V+!pv4|qPEP;H8nLdD|D;H#l^F=^+*L;N_R3?-TMr$=AvnXJX%wp zCnw(ff^W$ta@4E$kS;#6Q+a?0#)Hd%~JKa?H$jN(YHavhHvOoNCT&{#tB zhTfA0Wp{Hg;mmIsi666|t`J4q%W6r!Dau;)E?=kB9MV$3oYq6yw7sjN8F$}OaK&w^ z6g(38qo-510>OxD+6Dy&8fPHk67T|}Ia3%+imBfi-W$M*BNxiR=~bw66{)z--nu?g zhh|B8KVGsW6^saLFyJwoDg4>9UFU*Va8$HWsc|?ji7@R*FRn(-1K`B3&P!}FZcb`rtD%WD6%$D*@wo9UN_e1w!QRM5EL==HujQ#zEh7ekI9c_h`mL?4 z9$iDN@Q_9vSLU^=a4kyZyQ00ly|5Eh589f^v&yr+FxkaSMZ0XjaUZ(p$OysSR8&+l zGSV0!M^mmbyw3{l?32d7Lcb2=b9aqQ4bxsad03$S0_h3)ypaoJX0Op88410+#Y13mPe9daHjruShrNoOLTY8bury#Uy@UM}v@UTxw8BIKA{_bt90`6B~Px}Yr(_3=Z(#`pvu;;i| z)YqeuZJL;U!TDT)ChxEQGhTa>;U>cI*mPuMSm2!lmv%wBVBzISR%CB)4~2x_odW+& zetnTe@k|1qF(Os?cQaz5cW*aW1&vlXSTO_-XNR`{(bdvapS#B5w; zT#_aBkd~J%Y;T2^heuQ8!^Q46)mH|c&D~A0w(vJ{;c}0??rJrrx~00i)Zc?`x7wrx zi03)1ru$A%yT(RlpQ1dHc6WDM#OY>70_WsrFIZbESXYb~A13ITby$VVg()1k@ZyA+ zxx2c2#pzaOO37#CkDsw~)A-yNVWn-Ew(_hGzbo<#KDe;>i>ak)Koa^`L7~0n*|WMp z%gn-{I)p>r9z|o1uau>!U8qE4&);eaSrJ`qt}M8$4zGrkA1$?f-RXN<`Fo^}_=7r_ zk}E3}bnc;VVBBhBxQo=<<6U0tp~YPgmHL6PmhDa{=up-js;gY#8_dMY;iJ8^qr!BX zo0Zkt(KFRNK;#w9CEP)W@w9VsU`TQi#jxZl`p2)CkAqqtdoPF@vdGHGBfzJ3*660K zkhnnf%+S%bYCI;pewNCV1S%^l;}0M%7M2gQbT`u<_w=>lNHD;`fAQ&h_Hk$d!{BKv z0e9ffn)9AB0>L9t%vI07)9CY}Z|OD(b59)`$xI9kHj{`ngc@BCQ$?1vh*Ladc)~ArF{Xy`@*|NwcW* zEyrjU0sv`G9bGzT8nrbUMQz-5bx=6i)pvc`13> z;gK=sp^+dF339UHg6y*ETM{MEh$IcwXyB<#=T=^~%c|Oak$WpEvAfq%!s?;l$(trt zZW|6sg?8HZ&UPQBs1Gh`&-7NXi!i{G6&ogS*CYCf5*>70%0HYxeJC%t8~2am5GF52 zeFp6dgSZB`9CSEBLTt_JrRAhX$A85x;+n)}Btlo-UNkRcqEw^Eu{(aK^u6(r@=k-B z9;9O;>~~eJH?y=@W_Ls^ov$#wMJy=Wa*^e^3+I%T_YxF1@;Q#?w0z#Me9*m6Egg%7 zg^hK{L`O%E_7iKl%zN51X*h@9m zniwRuyWFu%feO-QpQrKPS1e|r5J>4{uwlTNnwv3yw^wNj<@;H9^8_c`AM|P8_sO8< zXLaf&TvM&Leusy)r6>`v923s&J{9Q=u~bHb1nuqZ8(gw;YYL>q%ud@N8l8~#s52HPtO7qhMuILO+mn?xmh*vrmYU1JGqFPJ6O zSoFLF9=)TWpj*@?M@$EVjhyhMby1C zx6~_m2Zc>WTM8W=9euuf%{9h+Q80Y7HeyFW5T|l|4d0aYhJyR0EgUNq6%QSXw;(lH z&!ajGO^q&>(P;;ss=E;QZDa2Y@>aLz@0+IAM~U+l_e-y*m`#;P8;lOmIPVAwoz+x} zaN9m61X5}})u;^BRlzqa);4AuR?nj);W@=p!$fdn;j-QDDV|`eBd2GYP z(4}oNc7DlpxvaO-&#>N?Hc&K(JU00av+Fg-PE2K>(bknjbo%GqyyETE#o%!X2d?>j zc1os{O;giLS69&<2upKG{me|rk=;;AnD%bCc6d129(z+2__pYCN~rlxe+u^`_scNUgU;c;o1KD@$BkYlNilu- z-;&FaES}7zwfP_*z%A)HIXlMUHcJ|K*za&nh$TFJJIs|u^b(I$J8G(cHIXddRE(FI zo;`dBg>)EkduvOOLsG)}`JjLG#Y*-{=@8RXZBI)ydS7lDDH^IsZ)NTFnADimq?^;S z<(zh8C!V0;wwMPrrDmyzY(AJ{m4kGG6~7+QwDmPoXO!2N>C*P?KEu$2mE^WZcL)c^_`V44VnUu0Xuu!Sdg+y zVAP0>YpIJQy3-kK_KAM-9Ah322WMhrCnaa8x}`<w)VCHmG?NIAA3SVP!U8vgG7Bt$tF}al!V`oHpkZBbCgIG z^#x+EjzuCFtWKX@a4zoeaq^K`7e$>-d)Q2@hL)L5B3Xaawbc{iqTbou^O2}4HW3wh zGI3T@!#(=6d$)6|XG-k_)0YOm2IVE8{QZdJ%}t*>FD-0(6I;FI{xi&%&5YKq8ZtjH zCEvP&r6%B2pZlpPY4w#9_+(SPqj*PfJ(jYrBNGkChJGFn&QF}67=7^cV>@%C*t#S>7IoP z8w8vf7dl6}q^0U)b0D~WD*f@pp9zD(TEvVLQ_nz8=yBmUyG+-JUn}O*H_yd4p$(>o zr`PflxG`q*L~P#IE8P7&WM-OCK`3ZwXa!4i2&(xMdLUIVWiZ57Vw%w*=KcUb6xuBr zaPqDDaayUR#+jJjxvrrhIUhZmlC&6jHXh6kA2117-=1fqyT|Q1E@`l+iYIT#zL&Uu zQI*yxgUs{s=*TWmMIs@jfHsf5T)4k-QztT==O$s*HXxfUS692IsnV8$n%8Dye;!W9Zr8_yLhX(d*$=ghzjRui!bkCTT`6d*?w6ItFN!`#KGD5;9a19%c!cV`%YqF z0^HN5X&zqQI`h8Bzp5l>CO_RsF596z>i($vTCdKdkDr}=bYym9bdsHy*F)kLb?ebl zc}Vo#w->Zy#?D`he&G0g*`h@$AWv-XGq|YaHAuQ$(kX6IqMkj6DMgf*gN55j3cjPW zb5Ni^);7+K=k134fMgCud2EB&Qg=e~%a?8Q_TRq;BL&N!MKH(@fQi6d?PF7ve$ijD zNu1=up;eLJv#|N233ql5X&P=~e-1A#wRd%mjn$zD(Dn52&GMZ%Is{XA!NENdlC*i& z-!T@xx3#&pXtKYsBuf<-LP<$U_J&Nn-Cj#wRkiwe;BPZ?^Ew-Tlh1};ci_h=Y8ukg z46(3&8um>OY@QYBwnB2-#p8rbK1L@F-40B)s$`+*rm3BJHx31FZ|(SZ3lF_> z+E%mGi~f@C(a_QH@NrWJQrfhdNMB%lv~F9dR;2ztGFAYc*PZfK zT)4Eqdz|9wP=}Ek7?|DG)}K_Qr6%vLl3bwO+eC!&?jkqGLgkqZyscx7uDwiyRY%*# z_6Chs8yvoYk$%WHx#xkI3fmj{*NnY_pr#MMxLCSOGYhGj+}h*!e1FtuXNEQwlwtlR z`yl!#&d+Db*&*|-`{&oZnGSax3?OAWZLi%hNnXbB-qw-9PEIy1m2xE_UPMcr`KA|{ zg*CyZ?wWNmyzcNet0bej^?LB%X>ty}^6al~F++Q0ikc?9Z$7tATRtu47em@I-yTZH z35tgd4G-&QTTe_MpR2Rkmf4P%fW<~iqBMdS{EpY3L`z(Cg>Q2%kj3{&3uD`teviFD z0mIBeABU@PHt1tt-tAqn4^O}eqwI~2Cd(I{V$7zTkyIORuV516Kj({2c>*nQqYn9J@#uhw5Wae9%jK^5?uZn@CaW$;q(gA`SVr z0#bHvc6ws@^Hz~oqmEp(m8@E_!$FD(4Rw2hJq_*``JYN{?6FxdH_u^9SgsNek(8Be zL)bC~3HpebasQ&4D%5|M9kUuUQYPo=>})@F7g0m(H-o+P2jp-*mZDDKw`*MAmR zejtX8bPkC>hjtGjnYXFG=TP95tU8wHRulEdON;5>=2#ck7ZHY_;KU4SsVRl5lPsGN zKl)j-?D|c-c;;K`*#wqz=NPfq%bUjR<=f2TXaV&``%hjD$|D9t!kZBTpN8t(Yk4Fe z389bpAIKd|=*^Qf#p}H6k@yVy_5K$rzAtHam$c8BxJ?koBko{jMZ&fZ7Gb{h4475I z*tF0`Et>v8!Ugcf)g%#dG$A$(ybVjlL$kJra3nt#usO5wgj>RVx%ST_24^moy+-P= zwo7~Q70hlqqA+Uob*VWW>qzdi*eIkoDMc>~(7x37;sXLxL!P0pL>5IJ$5 zWv^L38TRAUBIoEM`OVC<3Jn}|!lLp9q(@5ns)K-!k53*w;z(`drsjA! zcCi^ET7dS-(UR}Q#~I&6GPONwLlEwF_Ae8IWtUW)zH{EaM=9fP=e))bN4h8F6oQK& z<5L=w)H9pHNR>Ye{i|I}^5o*b`hyP3Y+rE%!8$>68M@1;Y?$G1OPj@$EwS6bYF4HF zjE%%-s(NOpsA#24bi=jgaT=`=QESF^Uvf&*eduK{6`M{l4XHLm9vtfks~TQHPhx7X z0ZP4@nFSjM8v)5ae@qY-nB9&mw}aA8`g~VbL8gjLb9|8Gu`sUGJin+m<)s?=bJ|P~ zn6(TU1z9OJR#{m|w{Z_`V%r7u>eZvH;iFS#=tu>-2nYrpB*Q zT?dqyn1bSzLr!JPG~COo2~{I_sdA&+v|Cs4+ETOeVr%PLTDaxMjEorGr?__3vn|5}j<@7E5iDeHJrsXkkzgSC7<=n@Ygz2% zbcOaQd{!?xExiQpoUwP z-m$-Sswr|tOglJcj#zWXZtfr25d$mfu?CMK9`o`}M}D z*Kie8)pH7RZJAl@M5RSfPfwfinweVGuP!fQig`lAYmk=UFRm)81fD(hoqwZ-CpNN- zf zUP4Cb*rM?Q5%{~ioA9fCpg~FFmGxC)CKg?A5&hfuKTamjZ*J-fqJvKYe;jRWZX)nm z6c^d{!wleaMCk|SqWN4-+X&4&+J!uIa9mZ>W*Zz(Y< zJSxP)Kw5>YK1(fBFY0z*L&sHYDXfqONf@tYdsdfyjX$Kj-wC2tQp0@x`t<^?IzDpY zx1R21TPr7QO*CE$$$h&_h)3f{#e`#5PoM7&)ivH6E9qS`A;wK3le+|lh8{CBMWH{j z(OY`@$;qK%DGsNFO1pl!{>nvX?(=tCeiXjJ>1ieHmHM!qoo?$G;f^9Cds}!! zxmyXpp0g&X;U$8IN{GjahzOj@o$Cp3XPcMh=`I!R*0%^0J6wDBjt{K)QX@^aZR=m?-tIOWuK0+fi zOKU!z`S)UJ2RPJ{22M_uvGbEt!^0hxu{T_Cyp$bOT!iwG!^9NX|Emqi|FR_+c#lYy zoZq<;ll#PSa|ZvnOlV2l2*;zj^l1h7Q_TLiF009ypGMF3j_utfk{1h7Q_ zTLiF009ypGMF3j_utfk{1h7Q_TlD`&wus}OGf)3%wuto~n5O}@2w;lQUL55A556>JgPKiIql*dl-}0@xzJZxP_P$R6-p1o$li{1yRzivYhxfZrm( zZxP_P=^WV26w4X#ApW4c<8SO@J)|*dl-}0@xyeEdtmgfGzrO%@#2; z{d4x|KMfag{sa3o02cvp5daqfa1j6(0dNrj7Xfe)02cvp5daqfa1j6(0dNrj7Xfe) z02cvp5daqfa1j6(0dNrj7Xfe)02cvp5daqfaMAyGxQLngpWD6t({vHnKiItm=pukF z0_Y-uE&}KxfGz^)A}~M~0dx^S7Xfq;KobPyOB)ZMc zxF(ZpW_#G>yS79mmvS-K&sw#Cm6omJwTK2U`FI}h3d7Cg6}RvD@(Db$ zaqnD8AtCkl)=(L@>i0n{;M$q2>zm7)o9m&a{`ZNbe8#!*gDdU2SrW2wJlbh;6C1y$ zwnn_Fz|Iwl1;Y_1R|j|Z_)EKsKIP7ei99-)lJyg18X3Zhi2@NdpYCo^u5YiKrpr@0 zeGkrdV9)bwdCh1%ZgVqiB~U6rg^H?r`!RSxkxytN`e$j9`?P#9=mI{%{)&_Emh!s60dYLw_2*8 zQMU5L`Y235J40IeGrvGEyK16r*#NX`P`E@lLrF3I;khNFxTIpZb+SZ*Dm~Ay&UTJ> zoQkY;v&AH$xdlQv6k;V6qG2|frDY#nVw`=FjB?i!y}Rg)CVvW-6f{5-Zg2Fv8)9vNa~ z_!rfMomuOfAv3f6Q>}{q&$cS&zqTrI3q1$Oe{NCi9Nd3vQ6B=tVg?nkg>X9WDXz@x zEKhYwn9ca2(ogdb4S6L`hM&4&Xqx)`h^zb&phoXc%r5?hLiIPsZxZS7csO=22G-NB z!)eK!$3~vh9bCyfR^=JzYdrJ&BO24Q0ZRLvy9)3PZm%=IV}|1r(-5?}iTsqjhTro(GLMVS-ch(JTN|;{&P@Be>(Pyg)TM2d zNs9D{RQy)t=Ze(XHrpgKL}SNqz8$<^=o|@j?DZ*D=^{9F(1oUMwKn9A?CNUvXJE#X zj%q~7fF|=3(P|8DbEjMfdog{#IAAsLIlqniSo}`y8m>U2hSVJYA}JAn&~!UWONrQ= zcfL>~G$QtEF<7pkF3*O893|I-4CfK~;pF#hMPFu4AyR`sV>80>#-Rlo`Z zSYiB&Rv3T6o%Vk)N&RWn{>R?5fyFbhc>e!<@yzlkJn{UWsP;c{!~@uc0K3rt1-lT- z-~CLPxV4poD8$~t&eX=i+K%zzQeMvz!YE{Cs%P<5*xJI75jLeTwkKhQS$bfR!ot=r zn)Ga(+$8jzoNOe_Y-~&<%uGxi+Wh=~F{-ls9ccEDic#57-@(-e!l+{B2vK=B`J)hV zCPraBd&nOW|DixJE7+EUsg*G!$ka;6%HH&E*W#vj_6{N@dUg-Bk=6UlIWr3jOw!Q7 z#9otylZ)ixU}a|~VQ1&~lkDZI{-(+F;{+jNuY5w2B zzsbY$c*y6E16Ce06EpiCg*_C=340R{$HO@X?C0S)VHJD0en`yD#PrAghl03Z75GCI zcK?rbf06rJy1(SX&Bpe(ydU!WYb72+HV)VW!SZ@|z`s=oh?|0we>FNEcx4zS~) z4iDi&`G1xBYxqY#5AhFmeMkp8f2iX__?JWb4}JJ|-68oywUqu{5B_e;>fu}dcj>>= zv_Gw6e`mS>M@q*0cS^?iH+}lw)GrQhK)?R=`UTTvSOeu?`XhX3U=M@x|JlC>ZT?R! zxsYh^HHGQV;7oG)RuZXU|1kW{#JJRWzj?%Y#BL+ZXj_~E zuCPk{awEO(J$(qmQCuM@_s`o2nirc+bZfzm*S#O3^B$vp<%;%4Kq~SPaFM?3c~Xrv zl7=UbK=p=+g@u0pkV-mlD~soNksCT4sC@{kW?P=hoCpBs>!obl?;%g6EY|$g4@Hu06@_ zRoFEHb{#GPqqdkRx1?BKE?9YIewa~88$%$hSlL@fV^)-D?_*TyYI_;%_Ow0VgEjVJ zYiX3<$vw~Y@GXLJuw(r1jBy?h;;XCrd{kMP=5eq8-Q(dtzb#zd8-p6dMppS2S^JVu zji8~c`Epf=Mi6g6iuzXh<40}eo9;SP5|JpMIgugAqsIcHB-m8y>BJaLaQBH6!7jy; z`7Y|@Db`S2q^xaJF`8jdWqvb*#1&c1_|?(vMyHn%SmdkN&Mn?1%LAW5PCA({PZz^@ zFiqLJK>nZ~U4qj)5xngf3z(iMs3p~y7dcP%a^g)x2@_(I!PR!Pqx0K+8^OffEVkNL zHX(2UZfcKb`*>=exr6)cRAoJ0a1R%!D6^=v7pbuhF6c8U8ja2b#W=>Ft*)I-MI!rn zaG88s$uD2FTdJXeyCH}^S&2o|Op;FJm=DvNFhL*D61!7$ElK2$lcFNWhiD54T}Al4 z=)ty2P{oT=YCX8d6>q(h!g$+u!|l2$5&n_)P%mWY0unwp7KzwASKbu$b(q-CY$Au4 z@yXS}CwXG=)!6j{%nykn!Q)~SD`{M*366_(#Bj%Y&Gr+yb>R;0Uk%0vA7o@8TSoL8 zR(tjRh>Quy_lXRnrDb=j;$qR@cPoGM6x73OQb?7pRC7j>XiD`eQS3DT)gCM4wwgIT zsFUs8THLvOMg&(h*NTC*s^L<9$WMz_g3Hp5H2>6r(+i)U#mPSdVm7=pXVjhwZz`Lz zY{3cP!o7>iSpGC$Sk{AqB#b0H9mT$f1xefIH!=jxM?w&e1-U5sbHA?-}$KTb&k6E-x zr)h4=U5a)a+p0JpVRpQtu6?9p5X(2!;P(_cJ&c3$gP@do#}(~smLZA6t77o0bqDiH zcy&2MenK+dPg$B?VqK~n+ytfuzbCdgY-^hBK7s8cC-c;}Y+jEUE6;B6Xc9Ec*tfT) zh@fAUGX3h%oHbUPUH?3ju-Xz^>~~JvWRO)48R>EC;*54u^O!;VgbyvxKfp&Hwiuc-}U@9zz8lgDjaSvCOo!0@fY4}vJLMtJ(bhu zBfZ>s{CwKX#D`D66yduIil_PqOXT=#8)?@MDm>zf`_zBhUf-khB1;TaU*JUQKJ*0_YyTa(K}J1h3LJH5;YjY=!A$MI?nm!Ix{{PFSYov%=PX&6^s%qPWt8&8LZnt zcYU%PG}J;q&m{lwQPPfW#tHLrbyW;8N8WF6SWj^Teo5{&T<(>(s||RG8(K6a^#j@$ zY$m}l$lGV1xal<#;OjyEK8{EB%b5xlWbWGNPW!OwTOiI~KB+Rp9z>>JzoniCuXEZu z1uZ*d7gWHJw&V(X{*W{qW|1htwhl8~wmil}NyaFt{eaCr90)M`JDp?@ke-?)=mAg) z&%-F}wzNDuUD^A7w@JT6U+STJxzwP>>)KCVbR>~xHKd%9>k>ty-nD1BMs_oY8ly*; zUww0$dJVO9?wJd!nnev=%x6q`%#Cd7xgvmwhUT+LxaNs}bzzwCyT=luOvHwHu|AdtPcWMWN1VH~r z?FAWgPIEBQkliEl#+edD&8{U0k}vZjRV#APg#GAMC0pb?2yE{BR~$`z#MHUWc`~7F z;)G~)5RQJ6tmNTl@ACDSKi0L#2bi;;T9#}f3=;hKptXO-IIc$MIatPgp%7ZOF#294RGxo@ZU3x%lJKiG8{|iX&}6y*FXL-Sj#hJ0%0J zo-5=U)Ri%l(L@B^Jvn;{Us24rY0s2nBjtK!nW(LzHB6O&IV1DmN{tLa+Ir;@y|X%;n;uiG@< zdcGA_P5Fz?yK}}1nLB_Xyncuqv{HDpnTCr?x)~R7-uFalVRD;+iwz*|ny8__9e^+& zuRQZ|tH4u$jEDzQd9Qy(o>c2R+1j1BEtxvm z`MPO-Uf2}bpauQJQ@>GaWa*1o8rKW-R-P>_4sW&a=X(zpanQ|9i5X;C zGa)rViebgog5pVHLzbM{VFB#-M zG|k|;5$~|r!`gYzAZxnOpn=|(UBlq@>X_z}mk~-djBzNCh%!K~d~R$6 z$x`lQ3&!x+4dpN)pHUas$vW%l7b3}XFwrL(vi?wfux85D^djBqWGDm7ds|&w`x5@V zxtxOrFzi{2{&B0a{>2uhS?9x@c!T4G<~DPijakKk63xnOlJ4@U?uL@Kx^}uvWJ*M$ zU{c&p;Fm&LKW42J4Ti7ZTXmB+E#`Tz^IOr@!nrwV*kXh!gJNiTD87ado**u`*FvrN z8k5dMXw zSf#L@0!}I!;`pnLvD4bcg#u^)pjmRR9(26rz|Uxz!C%nM zfVlk*xczuP`x*?bBm3R5hifoSEJDaXO343~kpJ9Q3y-MBFD8$quEF}(^xTl6)(g!T4;iLgDM55?CKQeoF<>UB($9k0F<Q_Z)?n0UG?t2w6V4!T!6}l2qT%!L6?k%0`R#}bY;&p}uPJ(v@`s413IZ5A30-7mT zXh{G=LeE$v^V7_NmKG% zs*|jsDct>4i$cO+;1=!lU}U_OwF81h5fc+D4F~a;zZHPmh&F0WF~K~1q(_5@Fy0Ey zgh67)+W5V_k|iz57&xOE<2iAFbR3Y`ICV>brtbR;cCwum0rCYw2hQRm=lMG7DStHM z@`q%1KGULwG~yU@=sP#QWu2#}vm$UaT(oZw;}{@2EH?AYNZoD;<&l@9n49XLD7b{U zOK(*8pKIbcWP~#{_0_f=ij(c|6)#@RM%(sZPfOP{8=iFc^UapPz0Ia#y0y!2j;o#` zcdn4TNCVsK@5kA2-jt&^Cc?I&v*<4d${!u)w%9Kq&vF;_@4>mxEBI&nh8Y5I8#=45 zi-9W`w@zFQt753MhBx@_>)Woy=h;+S|E&81|CW{eFB}-iP4MDBb6^AkH-3~Fznl=@ zrx)~#1M_oTRQc^qkmG-kO$fmt|K`BF)-iCOl_Bw2dorZpJ?s2pZ|7S=AN2~1%WYmF zef*IQFh}>}z4*$NtGufZBZ1g=u_l)iLZ_b}FQ&atYpLmu$>Q(0a+Wp>xC@~6xCGG$ zkyx%}>o*FviF=EC8DBCxV8Saf5ca;Kb1xhYi%rIy>G3X%4&4!+3%PsB&BA_{6H_sw z_z2Ollb55|W5oNA(`;J2IvELd3`dgXH|_$`u~t>9_u-@fgVYi4!8o4+3_*UpAJL?Z zuyfn$!Rogj0m^6dcH-eu$P&KE(M-JSpL$Sxc2uv z+w(%-mb~#Owp5`BQ&<12e3(B9tidDt0KBu4fvnGcFZ%|c=XF<`oSH!;khz~;;o_mg z0~R7@#kvV>psU6{Ft;D5AnErh{(Ry+alfy;yc#v7?b7x*yGU~?n`Uql_76) z&b58A2bA#LokMXM^Rx$69+JL{9GymvUL(2R6)nfF|x7PoE4Z-`f5n z5aC*6ttyY<_#Bow@;Hw#@>!j>76^KB!ViR2pXo6!6wM}gDOQlNR@CJ>y64E8f2+f* z$y$RilFrrD3VZ4;`_QNar#R%Tm?v^LCaP7fHS)RDIk3q3;FN?{i|!|X=(1|juW)VN zBBD)64`2>Zw2cPQHw8TpQ!DiM@?x zKh@lJBHHST;OIu$&!GE$ECA@>l=LHp8RM?KP5Fnx&uwo*kJ8yV9; zuZs{>ynp7+sLj@3Xuo?qhPUs0*j8t*W`tMRhQhq1~{?+JQprY35>B@asK z7%_5yC0J`^4A5H;2)3p=z3r}i=|w79{T4r#*PogYEs^}&t^2oYINt!rduYDk5SW51 zE3$?XHQ!r3%mh4>F@FYrp_Bym(W#HSZ1q2t>T@w>qfJ;SsTXCWWP2DZ*}?w^Yi@sz zXwj`qy^`TxorA5if@FVF4n5e)qrpg>WN|;97lhF!l^x{I>b_`i7qNFevWdWrK40Y9 zsT-f)jM-@udSo~;_?%$?Bu%eA@FEAZ~(eOL-Co z-SuS|@Zy#uwQ8A6la0ScRS>!xy<0C@x*4L%9j}N>=RGqkeWx7$$q_c-QaRum2v&^^ zSb9*!nnnb*s@Rgv;2|OJO@1i%Er&wCf;VceZfEHpCapgHRG}Bq-X5FMp2^Y!tb7)e zX$+$^=V__WO7g5}^ma5H1Mt?D>>2jofjZ{&GGMxk#1x)$eG{em^K;xf%Yz<-Su8^OY#BYzG;XA?QY>M5*6$b zBehP;n)mlwwT-jQ;NZr{h)@W1#nuX=UWj!#{Mo!I`XF1q3W#w4ZGJQI)$TQBcdWek zpR5S^E$jK;u;SlA`Tyit4+Y$iGl~^a$;hAF`YSEoEd2kZMG)xU&-Har=ImCWBtBD5 zhD@oZVXUr4;~0yu4DyHRy-%s7$8dEEd0pAY6x1?K6Mm#H=p;+`#T8KUngA$!L?D(P z79m$VSXd?EF;UcwSY1`(5pNu^9DRM5I$pW>y0+g}t-)^vQCE!;KNb!->h(&Qg_RQ; zlT7aVoEUZPB{@un8Xyem3HWc*U>koyr;%>Ny{-~4_Qtx!#=6U*m^ncI@awnr-5<$l zM@H>#sS1yxf~lWUN9)TtI3A6c4mp>*IT)4}3^vJ`?;BqgL0-d&rMPnj;Rx$%yoWxD z2+OoJ^SE?|c{xl%yP#%&GEGow)o9+M)FOm%LuH~BJNbjiI+ZtJ(i}Kt0?%9sByZzz zc$R*4&GS;;Ieq9XKdk$8$ZNW~(wr^|Z^*2dqE!RPd?&Te1QNtM3k3;vi77iwy>wU_ zoaTC;x5c8c;9NbdE3RY0U>jyerB7!Z98pB}feAwzGPj)Z0`1Ufo^h-#r04UFCypLl zE$6L#>{O|xJk(bgZrLGUI&m(Po;6CMKkp{Iz|*l#-K}4kt?I{7VnCX01lV|2YirYN2}zcg9f^mQ@swcEy`-QD*%%m@)}xTI5}=)nYGk`92Q;k8=L*xmG&LW zgnOQjX0hBwF7>q&9jft;8$a-=*!Gd+ihg-%Vl{UGoU5=N@b;EnMLefF7Sm7^c%CDotG*pKQaDEBEmKC;YJC2vZ}srm{U(|+38yUcCpD>g zQ9EutY@udRwST#Ity>|JeC`tD?%l2o`~>HX-+aCFE#~1s&7KO#z5u= zx^HRO>382~F06A|{RjB)EBv5<3+!>AvU~xNV%%h*l^-qR#|LGyVv$M_JOuEo>Ze6j z{uQSmu%GSSvB2?7mdiC(vrdekoTiy*h}l!274+zwRoZ))qlBH4pn>m^`jMoyMQ-@Y zUga-;*Z&Ki{gzw*Z}9AA^6)A`DtHGuD+?Pw4H_?x-voRMBVU$F zizS^#bebig#+%ZTIK``8lJ0w7!y`TqsFWNiVudxN_7oPq9Ixl6htO)(!SW@Kp6A4K zr(!?q@=DUy&caotMxJ0+%q)NAwI0sUJn_@MT?+Ijm#HE?=Ij+Z?NhRu0vK_}wNWk4 z-^*s{*QnhW^R4dOH@?@PkDzI?3)+w#>vy(og`I_cB3=Bx zGo~5WwN7u7I=8ACJ=%|{iF9>hEL?4|;IMF2R)0!fo0%45mM}Iq{J4$gIaAY>#UW8P zk5NIXQw<%Xc;;$g{EV)l1}kD?Yrm10)nZY?SCW$?6XEV?--J=E{}srlEV7Mj?anH~ z|5(R!mhNPR;8FAZX}Gm?NVw@}?FfAfYj#B#y07w?~%85baRTDg(t&w^IEeQo~%IEzwU6cN8jW!zVI1CgO zxJedmTFZs8+<9-7s1+{#hqEf9KLsCo2f6A-HWt3$Q}isxLa^nWbw+T@ayC_WbNkac_0BHn9ft$oG6)}|D=!HtpAakgCv=T;zR6g+}CoC z$8|hC-an*YFMX`6yj-f^{cjcd-COUX5VEbd?bNt~#9U39PiLVR8RFH>@nWer551 zF}yk49WWM;7l7%yt$&Jh;PE0T>6EuJ^7DS9pT%6j5A#A>_zdWZPk_D_kKPheU z3VUxF!0sQJ5DMIXlL_2JaekEviQH6x{UQ^92;HFduQCw`7{vv@$iPq$2nzmxkwL)1 zH^Ho5WP%`w;O~18gbJX_c77cv1Q8ZQG0QJ=QDr!g-`9f*3kdx_7YYM`exD143IDM_ zsK`yx-LKaW20_5TuO}=ZApHA!!U7_o-|tr#4EzC`oKq7*_?*)d+ zTcVtUU&nz_Z^u6}7bjyY8*?Xu2M-AN)vdhDZ{BBqHG6y1uU*_Y7zAQs1kNtTPA)&+ Qb{GWZ>=UrDNh{0z589=g!~g&Q literal 0 HcmV?d00001 diff --git a/tests/domain/sap10_calculator/worksheet/_elmhurst_worksheet_001431_case38.py b/tests/domain/sap10_calculator/worksheet/_elmhurst_worksheet_001431_case38.py new file mode 100644 index 00000000..f96b17ce --- /dev/null +++ b/tests/domain/sap10_calculator/worksheet/_elmhurst_worksheet_001431_case38.py @@ -0,0 +1,116 @@ +"""Mapper-driven cascade pin against the Elmhurst P960-0001-001431 +"simulated case 38" worksheet — a mains-gas dwelling with a code-117 +regular boiler (1979-1997, winter 66%), control 2102 (programmer, no room +thermostat → −5pp interlock → (206)=61%), and a **mains-gas condensing gas +fire secondary** (SAP code 611). + +This is the realistic re-generation of "simulated case 37": case 37 lodged +the same dwelling's code-605 gas fire on BIOGAS (7.60 p/kWh), which the +Elmhurst Summary export cannot carry (it lodges only the secondary SAP +code, not its sub-fuel — see `_elmhurst_secondary_fuel_from_sap_code`), so +the mains-gas modal default left a +7 SAP gap that was purely the biogas +sub-fuel. With a mains-gas secondary the whole cascade reproduces the +worksheet EXACTLY, confirming the boiler-efficiency / control-2102 / +secondary handling is all correct. + +Like 000565 / the _rr cases / case 20 / 21, this fixture does NOT hand- +build the EpcPropertyData: it routes the Summary PDF through +ElmhurstSiteNotesExtractor + from_elmhurst_site_notes so the pin exercises +the WHOLE extractor + mapper + calculator pipeline. + +Source: user-simulated PDFs at `sap worksheets/golden fixture debugging/ +simulated case 38/`. The Summary is mirrored into the tracked +`backend/documents_parser/tests/fixtures/Summary_001431_case38.pdf` so the +test runs without depending on the unstaged workspace. + +Worksheet pin targets (P960-0001-001431, "11a. SAP rating" block): +- SAP value (un-rounded, before (258) integer rounding) = 60.9152 +- (272) Total CO2, kg/year = 5801.0770 + +Per [[feedback-zero-error-strict]] + [[feedback-continuous-sap-tolerance]]: +pins are abs <= 1e-3 against the worksheet PDF (the worksheet prints the +SAP value to 4 dp). +""" + +from __future__ import annotations + +import re +import subprocess +from pathlib import Path +from typing import Final + +from backend.documents_parser.elmhurst_extractor import ElmhurstSiteNotesExtractor +from datatypes.epc.domain.epc_property_data import EpcPropertyData +from datatypes.epc.domain.mapper import EpcPropertyDataMapper +from domain.sap10_calculator.calculator import calculate_sap_from_inputs +from domain.sap10_calculator.rdsap.cert_to_inputs import cert_to_inputs + +# parents[0]=worksheet/, [1]=sap10_calculator/, [2]=domain/, [3]=tests/, +# [4]=repo root. +_SUMMARY_PDF: Final[Path] = ( + Path(__file__).resolve().parents[4] + / "backend" / "documents_parser" / "tests" / "fixtures" + / "Summary_001431_case38.pdf" +) + +LINE_258_SAP_VALUE_CONTINUOUS: Final[float] = 60.9152 +LINE_272_TOTAL_CO2_KG_PER_YR: Final[float] = 5801.0770 +_PIN_ABS: Final[float] = 1e-3 + + +def _summary_pdf_to_textract_style_pages(pdf_path: Path) -> list[str]: + """Convert a Summary PDF into the per-page text format the + ElmhurstSiteNotesExtractor expects (label/value token sequences). + Mirror of the helper in the other `_elmhurst_worksheet_*` fixtures. + """ + info = subprocess.run( + ["pdfinfo", str(pdf_path)], capture_output=True, text=True, check=True, + ).stdout + m = re.search(r"Pages:\s+(\d+)", info) + if m is None: + raise RuntimeError(f"Could not parse page count from {pdf_path}") + page_count = int(m.group(1)) + pages: list[str] = [] + for i in range(1, page_count + 1): + layout = subprocess.run( + [ + "pdftotext", "-layout", "-f", str(i), "-l", str(i), + str(pdf_path), "-", + ], + capture_output=True, text=True, check=True, + ).stdout + tokens: list[str] = [] + for line in layout.splitlines(): + if not line.strip(): + tokens.append("") + continue + parts = [p for p in re.split(r"\s{2,}", line.strip()) if p] + tokens.extend(parts) + pages.append("\n".join(tokens)) + return pages + + +def build_epc() -> EpcPropertyData: + """Route the simulated case-38 Summary through extractor + mapper. + No hand-built EpcPropertyData — the extractor and mapper are part of + the test target.""" + pages = _summary_pdf_to_textract_style_pages(_SUMMARY_PDF) + site_notes = ElmhurstSiteNotesExtractor(pages).extract() + return EpcPropertyDataMapper.from_elmhurst_site_notes(site_notes) + + +def test_case38_mains_gas_secondary_reproduces_the_worksheet_sap_and_co2() -> None: + # Arrange — the full extractor -> mapper -> calculator pipeline on the + # simulated case-38 Summary (mains-gas boiler 117 + mains-gas + # condensing gas-fire secondary 611). + epc = build_epc() + + # Act + result = calculate_sap_from_inputs(cert_to_inputs(epc)) + + # Assert — the SAP-rating block reproduces the worksheet exactly. + assert ( + abs(result.sap_score_continuous - LINE_258_SAP_VALUE_CONTINUOUS) + <= _PIN_ABS + ) + assert abs(result.co2_kg_per_yr - LINE_272_TOTAL_CO2_KG_PER_YR) <= _PIN_ABS From 077e3a39473996b527158ae2bac21c0775a7fe7b Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 15 Jun 2026 13:46:22 +0000 Subject: [PATCH 83/87] test(orchestration): re-pin multi-measure plan to the gain-maximising package MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The optimiser-package expectation was stale: it predated the optimiser folding a triggered measure's forced dependency into its candidate gain (ADR-0016). The run considers ALL measures (considered_measures defaults to None — no restriction), so once the ASHP bundle became SAP-beneficial (ADR-0025) the gain-maximising package shifted. Verified the new package is CORRECT, not a regression: on the test EPC, cavity-wall insulation earns +2.9 SAP alone but its forced fabric→ ventilation dependency (ADR-0016) drags the wall+ventilation pair to a NET −1.8 SAP (−0.9 on top of the ASHP package), so the gain-maximising Optimiser correctly excludes the wall and its forced ventilation. Update the expected set to {air_source_heat_pump, suspended_floor_insulation, low_energy_lighting, secondary_heating_removal} and drop the wall/vent- specific assertions — the forced wall→ventilation edge is covered by test_measure_dependency / test_optimiser; this integration test keeps its end-to-end optimise→persist→telescope coverage on the chosen package. Pre-existing failure (present before this branch's recent commits), outside the handover regression gate. Co-Authored-By: Claude Opus 4.8 --- ...test_ara_first_run_pipeline_integration.py | 43 ++++++------------- 1 file changed, 14 insertions(+), 29 deletions(-) diff --git a/tests/orchestration/test_ara_first_run_pipeline_integration.py b/tests/orchestration/test_ara_first_run_pipeline_integration.py index 7caffe6f..192f07a5 100644 --- a/tests/orchestration/test_ara_first_run_pipeline_integration.py +++ b/tests/orchestration/test_ara_first_run_pipeline_integration.py @@ -384,29 +384,29 @@ def test_modelling_optimises_and_persists_a_multi_measure_plan( assert plan.energy_consumption_savings > 0.0 by_type = {rec.type: rec for rec in rec_rows} + # The gain-maximising package: the efficient representative heat pump + # (Vaillant aroTHERM plus 5 kW, ADR-0025) now raises SAP even on this gas + # dwelling, plus the cheap positive-SAP fabric/lighting/secondary measures. + # CAVITY-WALL INSULATION is NOT selected: it earns +2.9 SAP alone, but the + # fabric→ventilation forced dependency (ADR-0016) drags the wall+ventilation + # pair to a NET −1.8 SAP (−0.9 on top of the ASHP package), so the Optimiser + # correctly leaves the wall — and therefore its forced ventilation — out. + # (The forced wall→ventilation edge itself is covered by + # test_measure_dependency / test_optimiser; here we prove the end-to-end + # optimise→persist→telescope pipeline on the package the Optimiser keeps.) + # The sample EPC lodges 8 low-energy-unknown bulbs (LED upgrade, ADR-0023) + # and an electric secondary heater (SAP 691, removal offered per ADR-0028). assert set(by_type) == { - "cavity_wall_insulation", "suspended_floor_insulation", - "mechanical_ventilation", - # The sample EPC lodges 8 low-energy-unknown bulbs, so the LED upgrade is - # a cheap positive-SAP candidate the Optimiser also keeps (ADR-0023). "low_energy_lighting", - # The efficient representative heat pump (Vaillant aroTHERM plus 5 kW, - # ADR-0025) now raises SAP even on this gas dwelling, so the Optimiser - # also keeps the ASHP bundle in the least-cost-to-band package (ADR-0024). "air_source_heat_pump", - # The sample lodges an electric secondary (SAP 691), so removal is offered - # (ADR-0028); the Optimiser keeps it in its all-beneficial-measures package - # — its SAP gain is 0 once the ASHP (category 4) ignores the secondary, but - # the heater is still physically removed at its own cost. "secondary_heating_removal", } # Each persisted measure carries the catalogue id of the Product it installs # (the MaterialRow ids seeded above), replacing the retired # recommendation_materials BOM with a single material_id on the row. - assert by_type["cavity_wall_insulation"].material_id == 1 + assert by_type["air_source_heat_pump"].material_id == 5 assert by_type["suspended_floor_insulation"].material_id == 2 - assert by_type["mechanical_ventilation"].material_id == 3 assert by_type["low_energy_lighting"].material_id == 4 assert by_type["secondary_heating_removal"].material_id == 9 for rec in rec_rows: @@ -414,27 +414,12 @@ def test_modelling_optimises_and_persists_a_multi_measure_plan( assert rec.already_installed is False assert rec.sap_points is not None assert rec.estimated_cost is not None - # The forced ventilation costs two £450 units and is priced even though it - # was never a free choice in the pool. - vent_cost: float | None = by_type["mechanical_ventilation"].estimated_cost - assert vent_cost is not None - assert abs(vent_cost - 900.0) <= 1e-6 - # The insulation measures earn positive SAP; ventilation's contribution is - # not positive (it only ever costs SAP — ADR-0016). - wall_sap: float | None = by_type["cavity_wall_insulation"].sap_points - vent_sap: float | None = by_type["mechanical_ventilation"].sap_points - assert wall_sap is not None and vent_sap is not None - assert wall_sap > 0.0 - assert vent_sap <= 0.0 # Per-measure bill savings (telescoping cascade, ADR-0014 amendment): each # measure carries its delivered-kWh and £ saving, and they telescope exactly - # to the Plan's headline savings. Ventilation increases energy, so its - # savings are negative — and the telescoping still holds. + # to the Plan's headline savings. for rec in rec_rows: assert rec.kwh_savings is not None assert rec.energy_cost_savings is not None - vent_kwh: float | None = by_type["mechanical_ventilation"].kwh_savings - assert vent_kwh is not None and vent_kwh < 0.0 kwh_total: float = sum(rec.kwh_savings or 0.0 for rec in rec_rows) cost_total: float = sum(rec.energy_cost_savings or 0.0 for rec in rec_rows) assert plan.energy_consumption_savings is not None From fffb07d04bd8d04155fff714d406f16c692690c2 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 15 Jun 2026 13:57:27 +0000 Subject: [PATCH 84/87] test(harness): re-pin golden-cert plans to the gain-maximising packages MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three more pre-existing failures (present at 9ee38211, before this branch's recent commits; same family as the orchestration multi-measure re-pin) — golden-cert plan expectations that predate the ASHP generator (ADR-0025) and the optimiser folding forced dependencies into candidate gain (ADR-0016): - test_console: a multi-measure plan now leads with air_source_heat_pump, not cavity_wall_insulation (which is dropped — its forced ventilation makes the pair net-negative). Assert a measure actually in the package. - test_report 0330: package is now {solid_floor_insulation, air_source_heat_ pump}; cavity_wall + forced mechanical_ventilation correctly excluded. - test_report 0036: gain-maximising package is now {solid_floor_insulation, low_energy_lighting}. Same verified-correct optimiser evolution as 077e3a39 (cavity_wall +2.9 SAP alone but its forced fabric→ventilation dep drags the pair net-negative). Re-pin to the actual packages + their trigger fields; the forced wall→vent edge stays covered by test_measure_dependency / test_optimiser. Co-Authored-By: Claude Opus 4.8 --- tests/harness/test_console.py | 2 +- tests/harness/test_report.py | 49 ++++++++++++++++------------------- 2 files changed, 23 insertions(+), 28 deletions(-) diff --git a/tests/harness/test_console.py b/tests/harness/test_console.py index 5712a62c..fb7c1c07 100644 --- a/tests/harness/test_console.py +++ b/tests/harness/test_console.py @@ -70,7 +70,7 @@ def test_run_one_returns_a_plan_and_prints_the_table( assert len(plan.measures) >= 1 printed: str = capsys.readouterr().out assert "Plan SAP" in printed - assert "cavity_wall_insulation" in printed + assert "air_source_heat_pump" in printed def test_run_modelling_inspects_a_plan_without_baseline_or_lodged_performance() -> None: diff --git a/tests/harness/test_report.py b/tests/harness/test_report.py index af020aee..6b8bc9a4 100644 --- a/tests/harness/test_report.py +++ b/tests/harness/test_report.py @@ -80,35 +80,28 @@ def test_each_fired_measure_carries_the_attributes_that_triggered_it() -> None: # Assert — the Plan ran and every fired measure names its trigger fields. assert report.plan is not None assert report.plan_error is None - # This gas dwelling lodges an electric secondary heater (SAP 691) on a - # category-2 main, so secondary-heating removal (ADR-0028) is a very cheap - # SAP lever (\£250); the Optimiser reaches the target band via the fabric - # stack + that removal, leaving the \£12k ASHP unselected (it owns the - # economics — ADR-0024). + # The gain-maximising package: the efficient representative ASHP (ADR-0025) + # plus solid-floor insulation. The cavity wall + its forced mechanical + # ventilation (ADR-0016) are NOT selected — the wall earns +SAP alone but + # the forced-ventilation penalty makes the pair net-negative, so the + # Optimiser correctly leaves them out (see test_measure_dependency / + # test_optimiser for the forced-edge unit coverage; cavity_wall + + # mechanical_ventilation trigger fields are exercised in + # test_cavity_wall_recommendation / the ventilation generator tests). triggers: dict[str, MeasureTrigger] = _triggers_by_measure(report) assert set(triggers) == { - "cavity_wall_insulation", - "mechanical_ventilation", "solid_floor_insulation", - "secondary_heating_removal", - } - # Cavity-fill fired off an uninsulated cavity wall; its dependent MEV fired - # because no mechanical ventilation is lodged. - assert triggers["cavity_wall_insulation"].triggers == { - "wall_construction": 4, - "wall_insulation_type": 4, - } - assert triggers["mechanical_ventilation"].triggers == { - "mechanical_ventilation_kind": None, + "air_source_heat_pump", } # Solid-floor insulation fired off an uninsulated solid ground floor. assert triggers["solid_floor_insulation"].triggers == { "floor_insulation_thickness": None, "floor_construction_type": "Solid", } - # Secondary-heating removal fired off the lodged secondary (SAP code 691). - assert triggers["secondary_heating_removal"].triggers == { - "secondary_heating_type": 691, + # The ASHP bundle fired off the gas-dwelling main it replaces. + assert triggers["air_source_heat_pump"].triggers == { + "property_type": "0", + "main_heating_category": 2, } @@ -174,18 +167,20 @@ def test_few_measure_cert_surfaces_only_its_fired_measures_triggers() -> None: # Act report: PropertyReport = build_property_report(path) - # Assert — 0036 reaches the target band with solid-floor insulation plus - # secondary-heating removal (it lodges an electric secondary, SAP 691, on a - # gas main — a cheap SAP lever, ADR-0028), and nothing else. The cheaper-to- - # target pair displaces the LED upgrade the Optimiser used to add. + # Assert — 0036's gain-maximising package is solid-floor insulation plus the + # low-energy-lighting upgrade (it lodges 7 low-energy + 0 incandescent fixed + # bulbs, so the LED top-up is a cheap positive-SAP lever, ADR-0023), and + # nothing else. triggers: dict[str, MeasureTrigger] = _triggers_by_measure(report) - assert set(triggers) == {"solid_floor_insulation", "secondary_heating_removal"} + assert set(triggers) == {"solid_floor_insulation", "low_energy_lighting"} assert triggers["solid_floor_insulation"].triggers == { "floor_insulation_thickness": None, "floor_construction_type": "Solid", } - assert triggers["secondary_heating_removal"].triggers == { - "secondary_heating_type": 691, + assert triggers["low_energy_lighting"].triggers == { + "incandescent_fixed_lighting_bulbs_count": 0, + "cfl_fixed_lighting_bulbs_count": None, + "low_energy_fixed_lighting_bulbs_count": 7, } From 963b7d70fe304337711240f6e9e1198ae78bcf5e Mon Sep 17 00:00:00 2001 From: Daniel Roth Date: Mon, 15 Jun 2026 14:06:54 +0000 Subject: [PATCH 85/87] fix terraform error and pass handler bool for dry runs --- applications/sharepoint_renamer/handler.py | 4 ++- .../sharepoint_renamer_request.py | 5 +++ .../lambda/sharepoint_renamer/variables.tf | 5 +++ .../sharepoint_renamer_orchestrator.py | 32 ++++++++++++------- tests/scripts/test_rename_sharepoint_files.py | 24 +++++++++++++- 5 files changed, 57 insertions(+), 13 deletions(-) create mode 100644 applications/sharepoint_renamer/sharepoint_renamer_request.py diff --git a/applications/sharepoint_renamer/handler.py b/applications/sharepoint_renamer/handler.py index 998458bc..67e64cf7 100644 --- a/applications/sharepoint_renamer/handler.py +++ b/applications/sharepoint_renamer/handler.py @@ -1,5 +1,6 @@ from typing import Any +from applications.sharepoint_renamer.sharepoint_renamer_request import SharepointRenamerRequest from orchestration.sharepoint_renamer_orchestrator import SharepointRenamerOrchestrator from utils.sharepoint.domna_sharepoint_client import DomnaSharepointClient from utils.sharepoint.domna_sites import DomnaSites @@ -8,6 +9,7 @@ CSV_PATH = "applications/sharepoint_renamer/sero_address_list.csv" def handler(event: dict[str, Any], context: Any) -> None: + request = SharepointRenamerRequest.model_validate(event) sp_client = DomnaSharepointClient(DomnaSites.SOCIAL_HOUSING_WAVE_3) - orchestrator = SharepointRenamerOrchestrator(sp_client, CSV_PATH) + orchestrator = SharepointRenamerOrchestrator(sp_client, CSV_PATH, dry_run=request.dry_run) orchestrator.run() diff --git a/applications/sharepoint_renamer/sharepoint_renamer_request.py b/applications/sharepoint_renamer/sharepoint_renamer_request.py new file mode 100644 index 00000000..6a10447b --- /dev/null +++ b/applications/sharepoint_renamer/sharepoint_renamer_request.py @@ -0,0 +1,5 @@ +from pydantic import BaseModel + + +class SharepointRenamerRequest(BaseModel): + dry_run: bool = False diff --git a/deployment/terraform/lambda/sharepoint_renamer/variables.tf b/deployment/terraform/lambda/sharepoint_renamer/variables.tf index 97cca538..192dbafa 100644 --- a/deployment/terraform/lambda/sharepoint_renamer/variables.tf +++ b/deployment/terraform/lambda/sharepoint_renamer/variables.tf @@ -1,3 +1,8 @@ +variable "lambda_name" { + type = string + description = "Logical name of the lambda (e.g. sharepoint_renamer)" +} + variable "stage" { description = "Deployment stage (e.g. dev, prod)" type = string diff --git a/orchestration/sharepoint_renamer_orchestrator.py b/orchestration/sharepoint_renamer_orchestrator.py index 764776ae..b73c41b5 100644 --- a/orchestration/sharepoint_renamer_orchestrator.py +++ b/orchestration/sharepoint_renamer_orchestrator.py @@ -1,9 +1,9 @@ import csv -import logging import os from typing import Optional from backend.pashub_fetcher.sharepoint_subfolders import SharepointSubfolders +from utilities.logger import setup_logger from utils.sharepoint.domna_sharepoint_client import DomnaSharepointClient BASE_PATH = ( @@ -12,7 +12,7 @@ BASE_PATH = ( ) ASSESSMENT_SUBFOLDER = "A. Assessment" -logger = logging.getLogger(__name__) +logger = setup_logger() def build_canonical_filename( @@ -58,9 +58,12 @@ def build_canonical_filename( class SharepointRenamerOrchestrator: - def __init__(self, sp_client: DomnaSharepointClient, csv_path: str) -> None: + def __init__( + self, sp_client: DomnaSharepointClient, csv_path: str, dry_run: bool = False + ) -> None: self._sp_client = sp_client self._csv_path = csv_path + self._dry_run = dry_run def run(self) -> None: with open(self._csv_path, newline="", encoding="utf-8-sig") as f: @@ -97,17 +100,24 @@ class SharepointRenamerOrchestrator: ) elif "file" in item: original_name: str = item["name"] - new_name = build_canonical_filename(uprn, address, postcode, original_name) + new_name = build_canonical_filename( + uprn, address, postcode, original_name + ) if new_name is None: continue - try: - self._sp_client.rename_file(item["id"], new_name) + if self._dry_run: logger.info( - f'Renamed: "{original_name}" → "{new_name}" (UPRN: {uprn})' - ) - except Exception as e: - logger.error( - f'Failed to rename "{original_name}" → "{new_name}" (UPRN: {uprn}): {e}' + f'Would rename: "{original_name}" → "{new_name}" (UPRN: {uprn})' ) + else: + try: + self._sp_client.rename_file(item["id"], new_name) + logger.info( + f'Renamed: "{original_name}" → "{new_name}" (UPRN: {uprn})' + ) + except Exception as e: + logger.error( + f'Failed to rename "{original_name}" → "{new_name}" (UPRN: {uprn}): {e}' + ) diff --git a/tests/scripts/test_rename_sharepoint_files.py b/tests/scripts/test_rename_sharepoint_files.py index 7b3e6587..5affea7e 100644 --- a/tests/scripts/test_rename_sharepoint_files.py +++ b/tests/scripts/test_rename_sharepoint_files.py @@ -21,9 +21,10 @@ def _make_package(name: str) -> dict[str, Any]: return {"name": name, "package": {}} -def _make_orchestrator(sp: MagicMock) -> SharepointRenamerOrchestrator: +def _make_orchestrator(sp: MagicMock, dry_run: bool = False) -> SharepointRenamerOrchestrator: orchestrator = SharepointRenamerOrchestrator.__new__(SharepointRenamerOrchestrator) orchestrator._sp_client = sp + orchestrator._dry_run = dry_run return orchestrator @@ -144,3 +145,24 @@ def test_skips_already_canonical_files() -> None: _make_orchestrator(sp)._process_folder("some/path", "500", "5 Pine Ln", "BB3 3CC") sp.rename_file.assert_not_called() + + +# --------------------------------------------------------------------------- +# _process_folder — dry_run=True logs intent but never calls rename_file +# --------------------------------------------------------------------------- + + +def test_dry_run_logs_would_rename_without_calling_api( + caplog: pytest.LogCaptureFixture, +) -> None: + sp = MagicMock() + sp.get_folders_in_path.return_value = { + "value": [_make_file("Survey.pdf", "id-1")] + } + + _make_orchestrator(sp, dry_run=True)._process_folder( + "some/path", "100", "1 High St", "AB1 2CD" + ) + + sp.rename_file.assert_not_called() + assert any("Would rename" in r.message for r in caplog.records) From 8b27a5fda22cf3b3977e9bc212eaa1655477628d Mon Sep 17 00:00:00 2001 From: Daniel Roth Date: Mon, 15 Jun 2026 14:08:40 +0000 Subject: [PATCH 86/87] correct lambda name --- deployment/terraform/lambda/sharepoint_renamer/variables.tf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/deployment/terraform/lambda/sharepoint_renamer/variables.tf b/deployment/terraform/lambda/sharepoint_renamer/variables.tf index 192dbafa..79b1a8d4 100644 --- a/deployment/terraform/lambda/sharepoint_renamer/variables.tf +++ b/deployment/terraform/lambda/sharepoint_renamer/variables.tf @@ -1,6 +1,6 @@ variable "lambda_name" { type = string - description = "Logical name of the lambda (e.g. sharepoint_renamer)" + description = "sharepoint_renamer" } variable "stage" { From b31db4b58b9f7ef7b4614b0b62f4e718e163c840 Mon Sep 17 00:00:00 2001 From: Daniel Roth Date: Mon, 15 Jun 2026 14:29:04 +0000 Subject: [PATCH 87/87] correct Dockerfile imports --- applications/sharepoint_renamer/handler/Dockerfile | 1 + 1 file changed, 1 insertion(+) diff --git a/applications/sharepoint_renamer/handler/Dockerfile b/applications/sharepoint_renamer/handler/Dockerfile index a81294f9..6a0a28b4 100644 --- a/applications/sharepoint_renamer/handler/Dockerfile +++ b/applications/sharepoint_renamer/handler/Dockerfile @@ -6,6 +6,7 @@ COPY applications/sharepoint_renamer/handler/requirements.txt . RUN pip install --no-cache-dir -r requirements.txt COPY utils/ utils/ +COPY utilities/ utilities/ COPY backend/__init__.py backend/__init__.py COPY backend/pashub_fetcher/ backend/pashub_fetcher/ COPY orchestration/ orchestration/