Commit graph

348 commits

Author SHA1 Message Date
Khalim Conn-Kowlessar
80d617aa5b modelling: glazing overlay models draught-proofing + frame-factor re-lodge
Fitting sealed glazing units changes two things beyond the pane's U/g
that the cascade reads, which the overlay didn't model — leaving the
double/secondary before→after pins ~0.7 SAP short (xfail):

1. Draught-proofing (RdSAP 10 §8.1). Sealed units draught-proof the panes
   they replace, re-lodging the dwelling-level `percent_draughtproofed`
   (cert 001431: 84 → 100). The §2 cascade reads that dwelling-level
   value, so the overlay now carries it. `_recompute_percent_draughtproofed`
   anchors on the lodged before-% — `after = round((round(before%/100 × N)
   + flips) / N × 100)`, N = openable windows (vertical + roof) + doors,
   flips = upgraded panes that were not draught-proofed — so it's robust
   to incomplete window extraction (unchanged openings are already in the
   aggregate). ~0.3 SAP.

2. Frame factor (§6 solar gains). A replacement unit re-lodges its own
   FF=0.70, overriding the pane it replaced — the two "single glazing,
   known data" panes lodge FF 1.00 / 0.50 (one is 6.6 m²), so leaving them
   unchanged understated solar gains by ~+150 kWh space heating. `WindowOverlay`
   now carries `frame_factor`, written flat onto the window. ~0.4 SAP.

Wiring: `EpcSimulation.percent_draughtproofed` + `WindowOverlay.frame_factor`
new fields; `apply_simulations` / `_fold_window` write them; the glazing
generator computes both from the upgraded set and cert 001431's after.

Un-xfails `test_{double,secondary}_glazing_overlay_reproduces_the_relodged_after`
— both now pin SAP/CO2/PE to the relodged after within tolerance. Updates
the two `test_glazing_recommendation` overlay expectations for the new
`frame_factor`. 96 modelling tests pass; zero new pyright errors.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 08:25:08 +00:00
Jun-te Kim
9956df07ff Land an override on the EPC's part at its position when the label is absent 🟩
A Landlord Override's building_part is a positional index (0=main, 1=extension
1…, ADR-0004), but the gov-API EPC can label that slot differently (e.g. lodge
the 2nd part as 'other', not 'extension_1'). The previous fix skipped such
orphaned overrides, silently discarding the landlord's correction. Now the
override falls back onto the EPC's part at that position (via _resolve_part), so
the correction lands; only a position the EPC models no part at is skipped
(no geometry to model a wholly-absent part). Replaces the skip-only behaviour.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 17:09:11 +00:00
Jun-te Kim
8f0432721c test(accuracy): pin RdSAP-20.0.0 PV semi uprn_22086693 (PV-list fix corpus)
Corpus validation of the modelling_e2e photovoltaic_supply-as-list fix. Cert
6102-6227-8000-0083-2292 (RdSAP-20.0.0 semi, gas combi + 2× 1.14 kW PV arrays)
crashed from_rdsap_schema_20_0_0 on the measured-array list; the fix routes it
through the dict-tolerant _map_schema_21_pv. PV correctly credited: engine 61
(no PV) → 66 (+5). Built in Elmhurst (evidence: epc.json + summary + worksheet,
fabric+heating; the PV "New Technologies" Panel-details grid deferred): worksheet
55 = engine-on-Elmhurst-inputs 55 exactly → calculator faithful. The +6 engine-vs-
Elmhurst base-dwelling residual is the documented RdSAP-default gap (band-C cavity-
uninsulated suspended-floor semi). Pinned engine 66.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 16:13:34 +00:00
Jun-te Kim
539f661430 Skip a landlord override that targets a building part the EPC lacks 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 15:35:19 +00:00
Jun-te Kim
17b1d63f0e test(accuracy): pin SAP-16.0 storage-flat uprn_10070004512 (built_form fix corpus)
Corpus validation of the modelling_e2e built_form fix. Cert 8742-6624-9300-2780-4926
(SAP-Schema-16.0, ground-floor electric-storage-heater flat) omits built_form; the
mapper now derives it from dwelling_type. built_form is ML-only so the fix is
SAP-neutral: engine 66 = lodged 66 exactly. Built in Elmhurst (evidence: epc.json +
summary + worksheet): worksheet 54, engine-on-Elmhurst-inputs 53 ≈ 54 → calculator
faithful. The +12 engine-vs-Elmhurst is a build/input gap (cert size-1 small cylinder
unrepresentable in Elmhurst's Normal/110L-minimum entry → higher HW + reduced-field
16.0 defaults). Pinned engine 66.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 15:35:06 +00:00
Jun-te Kim
666cf30437 gated propertly 2026-06-23 13:16:34 +00:00
Jun-te Kim
210ca6397f updated sap scaema to take in inputs 2026-06-23 10:22:40 +00:00
Jun-te Kim
9c89a0e680 neighbouring properties added 2026-06-22 14:38:00 +00:00
Jun-te Kim
2afa7acea4
Merge pull request #1251 from Hestia-Homes/feature/hyde_make_it_more_accurate_with_tests
Feature/hyde make it more accurate with tests
2026-06-22 10:04:28 +01:00
Jun-te Kim
1b53c57a07 re run ddd tests 2026-06-22 08:52:45 +00:00
Khalim Conn-Kowlessar
a9632937d5 fix(ventilation): use lodged extract-fan count when known, not max(lodged, age default) (RdSAP 10 §4.1 Table 5, PDF p.28)
Table 5 reads "Number of extract fans if known; if number is unknown:
[age-band default]" — the default is an UNKNOWN-fallback, NOT a floor. The
cascade applied `max(lodged, table_5_default)`, flooring a genuinely-lodged
count up to the age-band minimum: e.g. an age H-M dwelling lodging 2 extract
fans was billed at the 6-8-room default of 3, over-counting ventilation line
(8) and the heat-loss coefficient. Fixed to `lodged if lodged > 0 else
default` (a lodged 0 is the Elmhurst/RdSAP "unknown" form → default; any
positive count is taken literally).

Surfaced by Khalim's Elmhurst stress worksheet (simulated case 46): this was
its last ventilation residual — our Jan effective ACH 9.14 -> 9.0748 (exact
match to the accredited worksheet), SAP 29 -> 30 = Elmhurst, cost £1496 vs
£1493. Corpus IMPROVED: within-0.5 71.6% -> 72.5%, MAE 0.819 -> 0.815 (the
max-flooring over-counted ventilation on every cert lodging fans below its
age default). Floor ratcheted 0.71 -> 0.72. pyright not installed locally.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 06:22:26 +00:00
Khalim Conn-Kowlessar
34e52a893c fix(heating): assume portable-electric secondary for unheated habitable rooms (SAP 10.2 Appendix A.2.2)
When the main heating system does not heat every habitable room (heated rooms
< habitable rooms), SAP 10.2 Appendix A.2.2 assumes the unheated rooms are
served by a portable-electric secondary heater, so the Table 11 secondary
fraction (0.10 for a boiler main) must be costed at the electricity tariff —
even when the cert lodges no explicit secondary system.

`_secondary_fraction` previously returned 0 unless a secondary was lodged or
the main was a forced-secondary electric-storage code, dropping the assumed
secondary and billing 100% of space heat to the (cheaper) main fuel — an
over-rate. Added an `unheated_habitable_rooms` trigger plus
`_has_unheated_habitable_rooms(epc)`, which prefers the lodged
`any_unheated_rooms` flag and guards the gov-API `heated_rooms_count == 0`
"not provided" sentinel. The secondary fuel/efficiency cascade already
defaults to portable electric (code 693) when no secondary code is lodged.

Worksheet-validated on simulated case 46 (heated 4 < habitable 7, no lodged
secondary): the assumed 10% electric secondary (2289 kWh, ~£260) lifted our
SAP 39 -> 29.35 vs accredited Elmhurst 30 (cost £1502 vs £1493, within 0.6%).

Corpus UNCHANGED (71.6% / MAE 0.819): all 17 corpus certs with heated <
habitable already lodge an explicit secondary description, so the gov-API
path was already costing it; this only adds the assumed secondary where none
is lodged (Elmhurst / reduced-field path). pyright not installed locally.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 06:15:37 +00:00
Jun-te Kim
a3e2566378 landlord override data added 2026-06-20 12:57:54 +00:00
Jun-te Kim
abd4bbc2d0 Merge branch 'feautre/hyde_upload_and_extend_landlord_overrides' into feature/hyde_make_it_more_accurate_with_tests 2026-06-20 07:26:35 +00:00
Jun-te Kim
3044c70202 sap score and elmhirst mapper optimsaiation 2026-06-20 07:25:42 +00:00
Jun-te Kim
3a0a122b7f Group landlord property-override enums under domain/epc/property_overrides 🟪
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:49:58 +00:00
Jun-te Kim
4fbd8d5999 Decode old/slimline/convector storage-heater heating overrides 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:20:41 +00:00
Jun-te Kim
49f783de45 Decode from-main oil/LPG/coal and gas boiler-circulator water heating 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:20:02 +00:00
Jun-te Kim
b7d2cff220 Decode bottled/special LPG, community electric/biomass, dual-fuel, smokeless coal 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:18:33 +00:00
Jun-te Kim
9d0d12c278 Lock heating-override value coverage and reader/overlay registry parity 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:10:02 +00:00
Jun-te Kim
a6f2ae99df Remap the primary heating system and compose the heating override trio 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:06:52 +00:00
Jun-te Kim
2ede859158 Decode regular/CPSU/storage/direct-electric heating overrides 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:05:39 +00:00
Jun-te Kim
21afbefa9d Resolve a landlord gas-combi heating override to its SAP code 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:05:00 +00:00
Jun-te Kim
b5ef41ea62 Remap the EPC hot-water arrangement from a landlord override 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:04:33 +00:00
Jun-te Kim
a117e61b31 Decode more water-heating system/fuel overrides to their codes 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:03:25 +00:00
Khalim Conn-Kowlessar
fc5f10ea92 fix(rating): floor the continuous SAP score at 1 (SAP 10.2 §13 / RdSAP 10 §13)
The SAP rating is spec-floored at 1 ("if the result of the calculation is
less than 1, the rating is 1"). `sap_rating_integer` already clamps, but the
continuous `sap_score_continuous` did not — so a degenerate dwelling could
emit a physically impossible negative SAP. Apply the same max(1, …) floor to
the continuous value (the un-rounded part is for sensitivity near real
ratings, not for negative ratings).

Removes a -12.3 accuracy outlier on the committed corpus (cert 422000111926,
lodged at the floor of 1, was computing -11.3): within-0.5 70.2% -> 70.3%,
MAE 0.845 -> 0.833. Ratcheted the corpus MAE ceiling to 0.84. Unit-pinned in
test_calculator.

pyright not installed in this codespace (strict gate not run locally).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:03:13 +00:00
Jun-te Kim
8f7a344707 Resolve a landlord from-main-gas water-heating override to its codes 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:02:48 +00:00
Jun-te Kim
46d1f8bbb2 Guarantee every classifier age-band value maps to an overlay 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:45:31 +00:00
Jun-te Kim
e71df5df99 Re-date a building part's EPC age band from a landlord override 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:43:28 +00:00
Jun-te Kim
cd14751fdb Route age-band overrides to the right part, normalise, reject unknown 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:42:02 +00:00
Jun-te Kim
0135f0f27b Resolve a landlord age-band override onto the main building part 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:41:20 +00:00
Jun-te Kim
5a6d9a4e5d Guarantee every classifier glazing value maps to an overlay code 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:38:07 +00:00
Jun-te Kim
8ce22a5ccb Remap every window's glazing from a landlord glazing override 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:21:06 +00:00
Jun-te Kim
71bdcabb73 Produce no overlay for an unresolvable landlord glazing value 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:19:56 +00:00
Jun-te Kim
917627c833 Decode single/double/triple glazing overrides to SAP codes 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:19:33 +00:00
Jun-te Kim
f1c6825cae Resolve a landlord double-glazing override to its glazing code 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:18:45 +00:00
Jun-te Kim
cab69bbca9 Guarantee every classifier fuel value maps to an overlay code 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:45:22 +00:00
Jun-te Kim
3dab6fc425 Remap the EPC primary fuel from a landlord fuel override 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:24:31 +00:00
Jun-te Kim
35b48cc8fc Produce no overlay for an unresolvable landlord fuel value 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:23:25 +00:00
Jun-te Kim
39950d410c Map community mains gas to its distinct fuel code 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:22:56 +00:00
Jun-te Kim
7f8e2762d0 Decode landlord electricity/LPG/oil/coal overrides to fuel codes 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:22:32 +00:00
Jun-te Kim
240d7b1025 Resolve a landlord mains-gas override to the primary fuel code 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:07:32 +00:00
Jun-te Kim
37b0a38425 add more test cases 2026-06-19 10:59:51 +00:00
Jun-te Kim
e6a829aaea more examples 2026-06-19 09:51:49 +00:00
Jun-te Kim
269a7fdaa7 Merge branch 'main' of https://github.com/Hestia-Homes/Model into feature/hyde_make_it_more_accurate_with_tests 2026-06-18 16:24:26 +00:00
Khalim Conn-Kowlessar
a7de8c5c35 Merge remote-tracking branch 'origin/main' into feature/per-cert-mapper-validation 2026-06-18 15:00:32 +00:00
Khalim Conn-Kowlessar
8942d45772 fix(fuel): price secondary dual-fuel/anthracite at their own rate, not the colliding LPG code (RdSAP 10 Table 32)
The gov-API lodges secondary fuel as an enum whose value can COLLIDE with a
different same-valued RdSAP 10 Table 32 / SAP 10.2 Table 12 fuel code:
  - enum 9  = "dual fuel (mineral and wood)"  vs Table code 9  = LPG SC11F
  - enum 5  = "anthracite"                    vs Table code 5  = LPG (bulk)
The main-fuel boundary already canonicalises these (`_GOV_API_COLLISION_
FUELS`), but the SECONDARY-heating cost + CO2/PE paths never did — they took
the bare same-value lookup, so a dual-fuel room heater was priced as LPG
(3.48 vs dual-fuel 3.99 p/kWh) and emitted as LPG (CO2 0.241 vs 0.087),
and an anthracite secondary as bulk LPG (12.19 vs 3.64 p/kWh). The price
under-count over-rates SAP; the CO2 over-count inflates emissions.

Fix: add enum 9 to `_GOV_API_COLLISION_FUELS` (5 and 33 were already there)
and canonicalise the secondary fuel code on both the cost
(`_secondary_fuel_cost_gbp_per_kwh`) and factor (`_secondary_fuel_code`)
paths, mirroring the main-fuel boundary. canonical_fuel_code only touches
{5,9,33}, so genuinely Table-coded secondaries (House coal 11, wood logs 20,
community fuels 30-32) are left unchanged — confirmed by a full-map audit.

Corpus: within-0.5 69.7% -> 70.2% (MAE 0.854 -> 0.845; dual-fuel-secondary
cohort 42.9% -> 49.0%, signed +0.55 -> +0.41) and CO2 MAE 0.12 -> 0.08 t/yr
(bias +0.04 -> 0.00). Ratcheted the corpus floors (within 0.70, MAE 0.85,
CO2 0.09, PE 4.0). A prior session deferred enum 9 ("direction not
understood") while the EPC PE/CO2 lens was confounded by the climate-cascade
bug (fc7c4d2d); on the corrected lens the over-rate direction is clear.

pyright not installed in this codespace (strict gate not run locally).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 14:52:58 +00:00
Khalim Conn-Kowlessar
fc7c4d2d3b fix(climate): compute EPC CO2/PE on the postcode demand cascade (SAP 10.2 Appendix U p.124)
The SAP/EI rating is computed on UK-average weather (Appendix U Tables
U1-U3 region 0) so ratings are nationally comparable, but Appendix U
paragraph 1 (PDF p.124) requires that "other calculations (such as for
energy use and costs on EPCs) are done using local weather. Weather data
for each postcode district are taken from the PCDB". `Sap10Calculator.
calculate` ran ONE cascade (UK-average) and fed it to SAP, CO2 AND primary
energy, so every cert's EPC-displayed CO2/PE were computed on the wrong
climate. Because most of England is warmer than the UK-average, this
systematically OVER-counted heating demand on the emissions/PE outputs.

The two cascades (`cert_to_inputs` rating, `cert_to_demand_inputs`
postcode) already existed; this wires the demand cascade into the
production entry point and grafts its CO2/PE onto the rating result (SAP
unchanged). The corpus gauge's longstanding +5% CO2/PE over-estimate was
mostly this climate bug, NOT (as previously diagnosed) per-cert mapper
fidelity:
  CO2 MAE 0.26 -> 0.12 t/yr  (bias +0.18 -> +0.04)
  PE  MAE 13.6 -> 3.8 kWh/m2 (bias +9.0  -> +0.24)
  SAP within-0.5 = 69.7% (rating cascade, unchanged)

Worksheet-validated to 1e-4 on simulated case 45 (heat-pump ground-floor
flat, postcode W6): the P960 prints the current dwelling twice — Block 1
on UK-average weather (SAP 60.5318, CO2 692.13) and Block 2 on postcode
weather (CO2 626.78, PE 6581.59). Both reproduce exactly. Added a tracked
case-45 Summary fixture + two-cascade cascade pin as a permanent guard,
and ratcheted the corpus CO2/PE ceilings to 0.13 / 4.2. The e2e Elmhurst
suite (Block-1 line refs) now pins the rating cascade directly; the two
Vaillant overlay snapshots refreshed to demand-cascade CO2/PE.

pyright not installed in this codespace (strict gate not run locally);
change is type-trivial (dataclasses.replace over SapResult).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 14:15:34 +00:00
Khalim Conn-Kowlessar
72ef0f0e7b fix(water): don't apply heat-pump water SCOP to a separate immersion (SAP N3.7a)
When a heat-pump cert lodges a PCDB Table 362 record, the APM override
set BOTH the space efficiency (N3.6) and the water efficiency (N3.7a)
from the heat pump unconditionally. But the PCDB η_water applies only
when the DHW is heated BY the heat pump (water-heating code "from main":
901/902/914). A separate electric immersion (WHC 903) heats the water at
100% regardless of the space system, so applying the HP's water SCOP
(187.5% × 0.6 in-use = 112.5%) under-counted the immersion's hot-water
fuel.

Gate the η_water override on the DHW-from-main codes; a separate immersion
keeps its own 100% efficiency. Space η_space still always uses the APM
value (the heat pump is the space main).

Worksheet-validated to 1e-4 on simulated case 45 (HP space + WHC-903
immersion): water fuel (62) 1893.57 -> 2130.2639, total cost (255)
619.7433, CO2 692.13 — all matching the P960 exactly; SAP 60.53 -> rounds
to the worksheet's 61. RdSAP-21.0.1 corpus unchanged (no HP+WHC903 certs
in it). Pinned in test_cert_to_inputs (immersion fuel is main-independent).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 13:44:51 +00:00
Khalim Conn-Kowlessar
9b0c590bf8 fix(heat-transmission): bill a ground-floor flat's ground floor (RdSAP 10 §3.12)
The flat floor-exposure heuristic keys on dwelling_type: a flat defaults
to has_exposed_floor=False (assuming a heated dwelling below). The
Elmhurst Summary path lodges a ground-floor flat's vertical position as a
"Ground floor" floor_type rather than the API floor_heat_loss=1 exposed
code, and the mapper can label such a flat "Top-floor flat" — so the
cascade dropped the ground floor entirely (a ground floor is in contact
with the ground and carries heat loss).

Treat a "ground floor" floor_type as a heat-loss floor, overriding the
dwelling-level suppression upward — mirroring the existing "another
dwelling below" party override downward.

Worksheet-validated to 1e-4 on simulated case 45 (a ground-floor flat
the mapper labelled "Top-floor flat"): floor (28a) 0 -> 25.38 W/K,
fabric (33) 75.63 -> 101.0104, HTC (39) 112.93 -> 145.3579, all matching
the P960 exactly; SAP 67.81 -> 62.52. RdSAP-21.0.1 corpus within-0.5
69.5% -> 69.7% (MAE 0.859 -> 0.854). Floors ratcheted. Pinned in
test_heat_transmission (ground-floor billed + party-floor suppressed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 13:16:44 +00:00