Commit graph

534 commits

Author SHA1 Message Date
Jun-te Kim
8f7a344707 Resolve a landlord from-main-gas water-heating override to its codes 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:02:48 +00:00
Jun-te Kim
46d1f8bbb2 Guarantee every classifier age-band value maps to an overlay 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:45:31 +00:00
Jun-te Kim
fc591c6550 Classify the landlord Age column into a construction-age-band category 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:43:47 +00:00
Jun-te Kim
e71df5df99 Re-date a building part's EPC age band from a landlord override 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:43:28 +00:00
Jun-te Kim
55cdc1c777 Route a construction_age_band override row through its overlay mapper 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:42:19 +00:00
Jun-te Kim
cd14751fdb Route age-band overrides to the right part, normalise, reject unknown 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:42:02 +00:00
Jun-te Kim
0135f0f27b Resolve a landlord age-band override onto the main building part 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:41:20 +00:00
Jun-te Kim
5a6d9a4e5d Guarantee every classifier glazing value maps to an overlay code 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:38:07 +00:00
Jun-te Kim
0b782bd1a6 Classify the landlord Glazing column into a glazing category 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:35:39 +00:00
Jun-te Kim
8ce22a5ccb Remap every window's glazing from a landlord glazing override 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:21:06 +00:00
Jun-te Kim
1810f09240 Route a glazing override row through the glazing overlay mapper 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:20:09 +00:00
Jun-te Kim
71bdcabb73 Produce no overlay for an unresolvable landlord glazing value 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:19:56 +00:00
Jun-te Kim
917627c833 Decode single/double/triple glazing overrides to SAP codes 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:19:33 +00:00
Jun-te Kim
f1c6825cae Resolve a landlord double-glazing override to its glazing code 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:18:45 +00:00
Jun-te Kim
fd922a26c2 Satisfy strict type-checking for the main_fuel classifier wiring 🟪
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:01:07 +00:00
Jun-te Kim
cab69bbca9 Guarantee every classifier fuel value maps to an overlay code 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:45:22 +00:00
Jun-te Kim
04dd2dd222 Classify the landlord Main Fuel column into a fuel category 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:43:54 +00:00
Jun-te Kim
3dab6fc425 Remap the EPC primary fuel from a landlord fuel override 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:24:31 +00:00
Jun-te Kim
991eb74132 Route a main_fuel override row through the fuel overlay mapper 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:23:37 +00:00
Jun-te Kim
35b48cc8fc Produce no overlay for an unresolvable landlord fuel value 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:23:25 +00:00
Jun-te Kim
39950d410c Map community mains gas to its distinct fuel code 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:22:56 +00:00
Jun-te Kim
7f8e2762d0 Decode landlord electricity/LPG/oil/coal overrides to fuel codes 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:22:32 +00:00
Jun-te Kim
240d7b1025 Resolve a landlord mains-gas override to the primary fuel code 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:07:32 +00:00
Jun-te Kim
37b0a38425 add more test cases 2026-06-19 10:59:51 +00:00
Jun-te Kim
e6a829aaea more examples 2026-06-19 09:51:49 +00:00
Jun-te Kim
269a7fdaa7 Merge branch 'main' of https://github.com/Hestia-Homes/Model into feature/hyde_make_it_more_accurate_with_tests 2026-06-18 16:24:26 +00:00
Khalim Conn-Kowlessar
a7de8c5c35 Merge remote-tracking branch 'origin/main' into feature/per-cert-mapper-validation 2026-06-18 15:00:32 +00:00
Khalim Conn-Kowlessar
8942d45772 fix(fuel): price secondary dual-fuel/anthracite at their own rate, not the colliding LPG code (RdSAP 10 Table 32)
The gov-API lodges secondary fuel as an enum whose value can COLLIDE with a
different same-valued RdSAP 10 Table 32 / SAP 10.2 Table 12 fuel code:
  - enum 9  = "dual fuel (mineral and wood)"  vs Table code 9  = LPG SC11F
  - enum 5  = "anthracite"                    vs Table code 5  = LPG (bulk)
The main-fuel boundary already canonicalises these (`_GOV_API_COLLISION_
FUELS`), but the SECONDARY-heating cost + CO2/PE paths never did — they took
the bare same-value lookup, so a dual-fuel room heater was priced as LPG
(3.48 vs dual-fuel 3.99 p/kWh) and emitted as LPG (CO2 0.241 vs 0.087),
and an anthracite secondary as bulk LPG (12.19 vs 3.64 p/kWh). The price
under-count over-rates SAP; the CO2 over-count inflates emissions.

Fix: add enum 9 to `_GOV_API_COLLISION_FUELS` (5 and 33 were already there)
and canonicalise the secondary fuel code on both the cost
(`_secondary_fuel_cost_gbp_per_kwh`) and factor (`_secondary_fuel_code`)
paths, mirroring the main-fuel boundary. canonical_fuel_code only touches
{5,9,33}, so genuinely Table-coded secondaries (House coal 11, wood logs 20,
community fuels 30-32) are left unchanged — confirmed by a full-map audit.

Corpus: within-0.5 69.7% -> 70.2% (MAE 0.854 -> 0.845; dual-fuel-secondary
cohort 42.9% -> 49.0%, signed +0.55 -> +0.41) and CO2 MAE 0.12 -> 0.08 t/yr
(bias +0.04 -> 0.00). Ratcheted the corpus floors (within 0.70, MAE 0.85,
CO2 0.09, PE 4.0). A prior session deferred enum 9 ("direction not
understood") while the EPC PE/CO2 lens was confounded by the climate-cascade
bug (fc7c4d2d); on the corrected lens the over-rate direction is clear.

pyright not installed in this codespace (strict gate not run locally).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 14:52:58 +00:00
Khalim Conn-Kowlessar
fc7c4d2d3b fix(climate): compute EPC CO2/PE on the postcode demand cascade (SAP 10.2 Appendix U p.124)
The SAP/EI rating is computed on UK-average weather (Appendix U Tables
U1-U3 region 0) so ratings are nationally comparable, but Appendix U
paragraph 1 (PDF p.124) requires that "other calculations (such as for
energy use and costs on EPCs) are done using local weather. Weather data
for each postcode district are taken from the PCDB". `Sap10Calculator.
calculate` ran ONE cascade (UK-average) and fed it to SAP, CO2 AND primary
energy, so every cert's EPC-displayed CO2/PE were computed on the wrong
climate. Because most of England is warmer than the UK-average, this
systematically OVER-counted heating demand on the emissions/PE outputs.

The two cascades (`cert_to_inputs` rating, `cert_to_demand_inputs`
postcode) already existed; this wires the demand cascade into the
production entry point and grafts its CO2/PE onto the rating result (SAP
unchanged). The corpus gauge's longstanding +5% CO2/PE over-estimate was
mostly this climate bug, NOT (as previously diagnosed) per-cert mapper
fidelity:
  CO2 MAE 0.26 -> 0.12 t/yr  (bias +0.18 -> +0.04)
  PE  MAE 13.6 -> 3.8 kWh/m2 (bias +9.0  -> +0.24)
  SAP within-0.5 = 69.7% (rating cascade, unchanged)

Worksheet-validated to 1e-4 on simulated case 45 (heat-pump ground-floor
flat, postcode W6): the P960 prints the current dwelling twice — Block 1
on UK-average weather (SAP 60.5318, CO2 692.13) and Block 2 on postcode
weather (CO2 626.78, PE 6581.59). Both reproduce exactly. Added a tracked
case-45 Summary fixture + two-cascade cascade pin as a permanent guard,
and ratcheted the corpus CO2/PE ceilings to 0.13 / 4.2. The e2e Elmhurst
suite (Block-1 line refs) now pins the rating cascade directly; the two
Vaillant overlay snapshots refreshed to demand-cascade CO2/PE.

pyright not installed in this codespace (strict gate not run locally);
change is type-trivial (dataclasses.replace over SapResult).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 14:15:34 +00:00
Khalim Conn-Kowlessar
72ef0f0e7b fix(water): don't apply heat-pump water SCOP to a separate immersion (SAP N3.7a)
When a heat-pump cert lodges a PCDB Table 362 record, the APM override
set BOTH the space efficiency (N3.6) and the water efficiency (N3.7a)
from the heat pump unconditionally. But the PCDB η_water applies only
when the DHW is heated BY the heat pump (water-heating code "from main":
901/902/914). A separate electric immersion (WHC 903) heats the water at
100% regardless of the space system, so applying the HP's water SCOP
(187.5% × 0.6 in-use = 112.5%) under-counted the immersion's hot-water
fuel.

Gate the η_water override on the DHW-from-main codes; a separate immersion
keeps its own 100% efficiency. Space η_space still always uses the APM
value (the heat pump is the space main).

Worksheet-validated to 1e-4 on simulated case 45 (HP space + WHC-903
immersion): water fuel (62) 1893.57 -> 2130.2639, total cost (255)
619.7433, CO2 692.13 — all matching the P960 exactly; SAP 60.53 -> rounds
to the worksheet's 61. RdSAP-21.0.1 corpus unchanged (no HP+WHC903 certs
in it). Pinned in test_cert_to_inputs (immersion fuel is main-independent).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 13:44:51 +00:00
Khalim Conn-Kowlessar
9b0c590bf8 fix(heat-transmission): bill a ground-floor flat's ground floor (RdSAP 10 §3.12)
The flat floor-exposure heuristic keys on dwelling_type: a flat defaults
to has_exposed_floor=False (assuming a heated dwelling below). The
Elmhurst Summary path lodges a ground-floor flat's vertical position as a
"Ground floor" floor_type rather than the API floor_heat_loss=1 exposed
code, and the mapper can label such a flat "Top-floor flat" — so the
cascade dropped the ground floor entirely (a ground floor is in contact
with the ground and carries heat loss).

Treat a "ground floor" floor_type as a heat-loss floor, overriding the
dwelling-level suppression upward — mirroring the existing "another
dwelling below" party override downward.

Worksheet-validated to 1e-4 on simulated case 45 (a ground-floor flat
the mapper labelled "Top-floor flat"): floor (28a) 0 -> 25.38 W/K,
fabric (33) 75.63 -> 101.0104, HTC (39) 112.93 -> 145.3579, all matching
the P960 exactly; SAP 67.81 -> 62.52. RdSAP-21.0.1 corpus within-0.5
69.5% -> 69.7% (MAE 0.859 -> 0.854). Floors ratcheted. Pinned in
test_heat_transmission (ground-floor billed + party-floor suppressed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 13:16:44 +00:00
Jun-te Kim
f0411b2cf1 Merge https://github.com/Hestia-Homes/Model into feature/hyde_make_it_more_accurate_with_tests 2026-06-18 10:33:27 +00:00
Jun-te Kim
b07472cf38 sap calculator variaince changes 2026-06-18 10:22:21 +00:00
Jun-te Kim
7f8bfa5d06 move overlas to its own thing 2026-06-18 10:04:14 +00:00
Jun-te Kim
4219ef9d8b Overlay landlord property-type and built-form corrections onto the Effective EPC 🟩
Adds whole-dwelling property_type/built_form to EpcSimulation (folded by
apply_simulations) and maps those override components. property_type drives
party-wall heat loss + ASHP/solar/wall eligibility, so a landlord correction now
moves both the SAP calc and the measure menu; built_form has no calculator
consumer today (feeds the ML transform). Written as the landlord text value
(park-home check is text-only). Refines ADR-0032 dec-4.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 18:06:29 +00:00
Jun-te Kim
0305241ad3 Overlay more wall materials and roof loft-insulation depth from landlord overrides 🟩
Extends WallType coverage to timber/stone/system-built/cob/park-home/curtain and
adds RoofType "Pitched, N mm loft insulation" -> roof_insulation_thickness. The
"(assumed) insulated"/"partial" wall states stay deferred (ambiguous code, needs
Elmhurst validation per ADR-0032); property_type/built_form carry no SAP weight.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 17:15:50 +00:00
Jun-te Kim
8abd5e8b54 better smoke tests to check what is failign 2026-06-17 11:53:33 +00:00
Jun-te Kim
a955a09e9c Pin uprn_10093116330 (full-SAP gas-combi 2-storey semi): engine 82 vs Elmhurst 78
7th sibling full-SAP cert; documented full-SAP→RdSAP +4 residual. Build clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 10:26:11 +00:00
Jun-te Kim
d08c35ee03 Merge branch 'feature/hyde_make_it_more_accurate_with_tests' into feature/landlord-overrides 2026-06-17 09:43:28 +00:00
Jun-te Kim
e519de26a4 Pin uprn_10093116336 (full-SAP gas-combi 2-storey semi): engine 83 vs Elmhurst 79
6th sibling full-SAP cert; same documented full-SAP→RdSAP +4 residual. Build clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 09:41:29 +00:00
Jun-te Kim
317220beba Pin uprn_10093116334 (full-SAP gas-combi bungalow): engine 81 vs Elmhurst 77
5th sibling full-SAP cert validated against Elmhurst (semi-detached bungalow,
same street/boiler PCDB 17505 as 10093116324). Engine 81 (lodged 82); Elmhurst
worksheet 77. The +4 is the documented full-SAP→RdSAP residual. Build verified
clean (storeys=1, no phantom conservatory). Worklist strategy note added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 09:32:57 +00:00
Jun-te Kim
3eb0022034 Pin uprn_10093116324 (full-SAP gas-combi bungalow): engine 79 vs Elmhurst 74
4th sibling full-SAP cert validated against Elmhurst. Engine 79 (lodged 80);
Elmhurst worksheet 74. The +5 is the documented full-SAP→RdSAP residual — engine
uses the cert's measured U-values (wall 0.19/floor 0.12/roof 0.12) + PCDB combi
17505 (88.5%); Elmhurst uses RdSAP band-L defaults + generic 84% BGW combi. Build
verified clean (single-storey bungalow, no phantom conservatory, TFA 52/51.9).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 09:10:36 +00:00
Jun-te Kim
d87718f316 Merge remote-tracking branch 'origin/main' into feature/hyde_make_it_more_accurate_with_tests
# Conflicts:
#	datatypes/epc/domain/mapper.py
2026-06-17 09:05:37 +00:00
Jun-te Kim
70ca56415e Merge branch 'main' of https://github.com/Hestia-Homes/Model into feature/landlord-overrides 2026-06-17 08:50:52 +00:00
Jun-te Kim
74c1aff530
Merge pull request #1245 from Hestia-Homes/feature/per-cert-mapper-validation
Feature/per cert mapper validation
2026-06-17 09:49:36 +01:00
Khalim Conn-Kowlessar
e136e937d6 fix(heat-transmission): match roof description per part by kind (RdSAP 10 §5.11)
The deduplicated `epc.roofs[]` list cannot be indexed 1:1 against the
building parts (190/329 multi-part certs have len(roofs) != len(parts)),
so every part's `u_roof` consumed a SINGLE join of all roof descriptions.
That leaked one part's insulation state onto another: a "Flat, no
insulation" extension dragged a "Pitched, insulated (assumed)" main roof
to the uninsulated 2.30, ~3x over-stating its heat loss. 3-part certs
systematically under-rated (56% within-0.5, mean -0.79 SAP).

Partition the non-RR roof descriptions into flat vs pitched/sloping and
match each part to its own kind (`_main_roof_descriptions_by_kind`),
falling back to the global join when a part's kind has no matching entry.

Corpus cert 100010129331: roof 110.5 -> 31.3 W/K, +13.10 -> -0.05 SAP.
RdSAP-21.0.1 within-0.5 68.8% -> 69.5% (MAE 0.888 -> 0.859; PE 13.9 ->
13.6); 3-part cohort 56% -> 61%. Floors/ceilings ratcheted. Pinned in
test_heat_transmission (by_kind split + mixed-roof no-contamination).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 00:48:50 +00:00
Khalim Conn-Kowlessar
c5aa5620ca fix(uvalues): apply §5.8 insulation R to stone walls (RdSAP 10 p.41-42)
The §5.8 Table-14 added-insulation R-value adjustment was gated to
WALL_SOLID_BRICK, so a stone (granite/sandstone) wall lodging
wall_insulation_type 1/3 ("External"/"Internal") + a thickness fell
through the §5.6 thin-wall branch and was billed at its UNINSULATED U
(e.g. sandstone 520 mm + 100 mm internal: 1.64 instead of 0.30 → ~5×
the wall heat loss). Mirror the brick insulation branch into the stone
block, feeding the RAW §5.6 U₀ into the §5.8 chain per the same rule the
brick branch and the dry-lined granite pin 000565 already follow (the
Table-6 footnote (a) 1.7 cap does not apply on the insulated path).

Corpus cert 100052159386 (sandstone 520 mm + 100 mm internal): -26.20 ->
-4.08 SAP, walls 300 -> 55 W/K. RdSAP-21.0.1 corpus within-0.5 68.6% ->
68.8% (SAP MAE 0.942 -> 0.888; PE MAE 14.3 -> 13.9; CO2 0.27 -> 0.26);
floors/ceilings ratcheted. Unit-pinned in test_rdsap_uvalues.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 00:26:25 +00:00
Khalim Conn-Kowlessar
0d24f5b13a Merge branch 'main' of https://github.com/Hestia-Homes/Model into feature/per-cert-mapper-validation 2026-06-16 23:43:16 +00:00
Khalim Conn-Kowlessar
d501535cbc fix(mapper): map dropped §6.1 non-separated conservatory (API)
The gov API lodges a NON-SEPARATED conservatory (conservatory_type=4) as a
glazed "building part" carrying only {floor_area, room_height,
double_glazed, glazed_perimeter} — no fabric, no floor dimensions. The
four fields were undeclared on the 21.0.1 SapBuildingPart, so `from_dict`
dropped them and the conservatory was silently lost: it billed no §6.1
window/rooflight/floor and added nothing to TFA (5 corpus certs over-rated
— too little heat loss → SAP too high).

Fix (21.0.1 schema + mapper):
- declare the four glazed fields on `SapBuildingPart`;
- `_api_sap_conservatory` builds `EpcPropertyData.sap_conservatory` from
  the glazed BP (identified by a lodged `glazed_perimeter`; only type-4
  conservatories lodge it — separated ones, §6.2, lodge nothing);
- exclude the glazed BP from the fabric building-part loop (it is billed
  by the §6.1 cascade, not as a dwelling part);
- `_total_floor_area_from_building_parts` adds the conservatory floor area
  to TFA (drives occupancy → §4/§5 demand).

Validation is cross-mapper parity, NOT a corpus back-solve: the API mapper
feeds the SAME worksheet-validated §6.1 cascade (`conservatory_geometry`,
pinned to 1e-4 against the case-44 Summary) as the Elmhurst path — so the
API conservatory fabric is correct by construction. `from_api_response`
on an injected type-4 cert reproduces the glazed wall (perimeter × ground-
floor room height = 22.05), glazed roof (floor/cos20 = 12.77) and Table 25
double U_eff (2.758 wall / 2.993 roof); a separated (type 2/3) cert lodges
no glazed BP → disregarded per §6.2.

Gauges: corpus within-0.5 67.9% → 68.6% (MAE 0.959 → 0.942; floor 0.67→0.68,
ceiling 0.97→0.95); /tmp eval mean|err| 0.822 → 0.817. Harness 47/47
0-raised; regression = the 3 pre-existing fails; pyright net-zero (65=65).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 23:37:25 +00:00
Khalim Conn-Kowlessar
694cdd9c23 feat(modelling): mark a Property as run via has_recommendations + updated_at
The new pipeline left no per-Property record of a run (the old engine set
property.has_recommendations and populated property_details_epc). Restore the
marker: PropertyRepository.mark_modelled sets has_recommendations (true when the
Plan carries measures, mirroring the old engine) and bumps updated_at, so a
first-run under the new process is identifiable as updated_at >= 2026-06-01.

ModellingOrchestrator marks each Property after its Scenarios (true if any
Scenario yielded a measure); run_modelling_e2e's --persist path marks it too
(its compute runs on in-memory fakes, so the DB UoW sets it directly). Adds the
has_recommendations/updated_at columns to the PropertyRow mirror.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 23:34:33 +00:00