Commit graph

565 commits

Author SHA1 Message Date
Daniel Roth
5ecb47d46c Merge branch 'main' into feature/trigger-e2e-lamnda 2026-06-22 14:55:16 +00:00
Jun-te Kim
9c89a0e680 neighbouring properties added 2026-06-22 14:38:00 +00:00
Daniel Roth
d05e5bd1f3 new application to trigger e2e for a single property and scenario 2026-06-22 12:54:53 +00:00
Jun-te Kim
2afa7acea4
Merge pull request #1251 from Hestia-Homes/feature/hyde_make_it_more_accurate_with_tests
Feature/hyde make it more accurate with tests
2026-06-22 10:04:28 +01:00
Jun-te Kim
1b53c57a07 re run ddd tests 2026-06-22 08:52:45 +00:00
Khalim Conn-Kowlessar
a9632937d5 fix(ventilation): use lodged extract-fan count when known, not max(lodged, age default) (RdSAP 10 §4.1 Table 5, PDF p.28)
Table 5 reads "Number of extract fans if known; if number is unknown:
[age-band default]" — the default is an UNKNOWN-fallback, NOT a floor. The
cascade applied `max(lodged, table_5_default)`, flooring a genuinely-lodged
count up to the age-band minimum: e.g. an age H-M dwelling lodging 2 extract
fans was billed at the 6-8-room default of 3, over-counting ventilation line
(8) and the heat-loss coefficient. Fixed to `lodged if lodged > 0 else
default` (a lodged 0 is the Elmhurst/RdSAP "unknown" form → default; any
positive count is taken literally).

Surfaced by Khalim's Elmhurst stress worksheet (simulated case 46): this was
its last ventilation residual — our Jan effective ACH 9.14 -> 9.0748 (exact
match to the accredited worksheet), SAP 29 -> 30 = Elmhurst, cost £1496 vs
£1493. Corpus IMPROVED: within-0.5 71.6% -> 72.5%, MAE 0.819 -> 0.815 (the
max-flooring over-counted ventilation on every cert lodging fans below its
age default). Floor ratcheted 0.71 -> 0.72. pyright not installed locally.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 06:22:26 +00:00
Khalim Conn-Kowlessar
34e52a893c fix(heating): assume portable-electric secondary for unheated habitable rooms (SAP 10.2 Appendix A.2.2)
When the main heating system does not heat every habitable room (heated rooms
< habitable rooms), SAP 10.2 Appendix A.2.2 assumes the unheated rooms are
served by a portable-electric secondary heater, so the Table 11 secondary
fraction (0.10 for a boiler main) must be costed at the electricity tariff —
even when the cert lodges no explicit secondary system.

`_secondary_fraction` previously returned 0 unless a secondary was lodged or
the main was a forced-secondary electric-storage code, dropping the assumed
secondary and billing 100% of space heat to the (cheaper) main fuel — an
over-rate. Added an `unheated_habitable_rooms` trigger plus
`_has_unheated_habitable_rooms(epc)`, which prefers the lodged
`any_unheated_rooms` flag and guards the gov-API `heated_rooms_count == 0`
"not provided" sentinel. The secondary fuel/efficiency cascade already
defaults to portable electric (code 693) when no secondary code is lodged.

Worksheet-validated on simulated case 46 (heated 4 < habitable 7, no lodged
secondary): the assumed 10% electric secondary (2289 kWh, ~£260) lifted our
SAP 39 -> 29.35 vs accredited Elmhurst 30 (cost £1502 vs £1493, within 0.6%).

Corpus UNCHANGED (71.6% / MAE 0.819): all 17 corpus certs with heated <
habitable already lodge an explicit secondary description, so the gov-API
path was already costing it; this only adds the assumed secondary where none
is lodged (Elmhurst / reduced-field path). pyright not installed locally.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 06:15:37 +00:00
Khalim Conn-Kowlessar
d05fdbe6f2 fix(mapper): map truncated Elmhurst glazing label "Double between 2002" (RdSAP 10 Table 24 code 3)
The full RdSAP-Schema-21 label is "Double between 2002 and 2021" (double
glazing installed 2002-2021, SAP 10.2 Table 24 code 3). When the Elmhurst
Summary PDF wraps the trailing "and 2021" into an adjacent table cell that
the extractor joins away, the surfaced label truncates to "Double between
2002" — the same artifact already handled for "Triple post or during".
`_elmhurst_glazing_type_code` raised UnmappedElmhurstLabel on it, blocking
the whole Summary (surfaced on the simulated-case-46 multi-attribute
worksheet). Added the truncated form as a code-3 alias.

pyright not installed in this container.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 06:15:25 +00:00
Khalim Conn-Kowlessar
104e3725b8 test(corpus): ratchet SAP MAE 0.83->0.82 and PE 4.0->3.7 floors after stone-mechanism correction
The corrected stone branch (age-E-only cap, §3.5 Table-3 unknown-thickness
default, Scotland band-J override) improved the corpus gauge to SAP MAE 0.819
and PE MAE 3.6 kWh/m2/yr. Lock the gains in and record the corrected
mechanism in the provenance log; within-0.5 (71.6%) and CO2 (0.08) unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 13:34:31 +00:00
Jun-te Kim
a3e2566378 landlord override data added 2026-06-20 12:57:54 +00:00
Jun-te Kim
abd4bbc2d0 Merge branch 'feautre/hyde_upload_and_extend_landlord_overrides' into feature/hyde_make_it_more_accurate_with_tests 2026-06-20 07:26:35 +00:00
Jun-te Kim
3044c70202 sap score and elmhirst mapper optimsaiation 2026-06-20 07:25:42 +00:00
Jun-te Kim
c1ab179d51 Add Hyde property_overrides build script + creds-free smoke test 🟩
Two-pass org_ref-matched builder for property_overrides (classify via ChatGPT
into the landlord ledger, validate+apply user edits, write idempotently);
ephemeral-Postgres smoke proves the one-property chain without creds.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 19:02:42 +00:00
Jun-te Kim
85e43b3737 Silence untyped SAEnum.enums access in the consistency guard 🟪
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 18:19:32 +00:00
Jun-te Kim
e03fd27357 Type-clean the override_component consistency guard 🟪
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 18:19:07 +00:00
Jun-te Kim
f7ee47118b Add the 5 new components to the override_component pgEnum mirror 🟩
Caught live writing property_overrides on portfolio 796: the Python
override_component SAEnum lagged the DB enum, so reading a new-component row
back threw LookupError. Guard it with a consistency test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 18:18:12 +00:00
Jun-te Kim
3a0a122b7f Group landlord property-override enums under domain/epc/property_overrides 🟪
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:49:58 +00:00
Khalim Conn-Kowlessar
034d4b7cd5 fix(uvalues): bill known-insulation stone walls by the §5.6 thickness formula, uncapped (RdSAP 10 §5.6)
An uninsulated stone wall of lodged thickness, age bands A-E, is billed by
the RdSAP 10 §5.6 Table-12 formula on its measured thickness — sandstone /
limestone U = 54.876·W^-0.561, granite / whinstone U = 45.315·W^-0.513. Two
bugs suppressed it:

1. CAP: the as-built branch capped the formula result at the Table-6
   typical-thickness default (`if u0 >= 1.7: return 1.7`). But the formula
   only dips below 1.7 past ~488 mm (sandstone) / ~640 mm (granite), so the
   cap nullified §5.6 for essentially every real-thickness stone wall,
   under-counting fabric loss and over-rating. A measured 400 mm sandstone
   age-B wall is 1.90 (Elmhurst-confirmed), not 1.70.

2. NO INSULATION-STATE GATE: the branch fired for any stone wall with a
   lodged thickness, including ones whose insulation is "Unknown". RdSAP
   treats an unknown-insulation wall via the Table-6 typical-thickness
   default, NOT as bare stone of the lodged thickness — so a 50 mm granite
   wall with Unknown insulation must read 1.70 (worksheet), not the formula's
   6.09. Gated on `wall_insulation_type is not None`.

Fixes the 2 long-standing stone-U unit tests (granite 120 mm → 3.8871,
sandstone 120 mm → 3.7408 — they were correct red tests, not "known fails").
Corpus within-0.5 70.3% -> 71.6% (MAE 0.833 -> 0.822); ratcheted floors to
0.71 / 0.83. Worksheet fixture 000565 (granite 50 mm Unknown → 1.70) still
passes via the insulation gate. The two stone groups (sandstone/limestone vs
granite/whinstone) keep their distinct §5.6 coefficients.

pyright not installed in this codespace (strict gate not run locally).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:32:31 +00:00
Jun-te Kim
4fbd8d5999 Decode old/slimline/convector storage-heater heating overrides 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:20:41 +00:00
Jun-te Kim
49f783de45 Decode from-main oil/LPG/coal and gas boiler-circulator water heating 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:20:02 +00:00
Jun-te Kim
b7d2cff220 Decode bottled/special LPG, community electric/biomass, dual-fuel, smokeless coal 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:18:33 +00:00
Jun-te Kim
9d0d12c278 Lock heating-override value coverage and reader/overlay registry parity 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:10:02 +00:00
Jun-te Kim
ad3b1f15a8 Classify the landlord Hot Water and Heating columns into categories 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:07:14 +00:00
Jun-te Kim
a6f2ae99df Remap the primary heating system and compose the heating override trio 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:06:52 +00:00
Jun-te Kim
6c2d1dce34 Route a main_heating_system override row through its overlay mapper 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:06:04 +00:00
Jun-te Kim
2ede859158 Decode regular/CPSU/storage/direct-electric heating overrides 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:05:39 +00:00
Jun-te Kim
21afbefa9d Resolve a landlord gas-combi heating override to its SAP code 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:05:00 +00:00
Jun-te Kim
b5ef41ea62 Remap the EPC hot-water arrangement from a landlord override 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:04:33 +00:00
Jun-te Kim
55e83c7f9f Route a water_heating override row through its overlay mapper 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:03:51 +00:00
Jun-te Kim
a117e61b31 Decode more water-heating system/fuel overrides to their codes 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:03:25 +00:00
Khalim Conn-Kowlessar
fc5f10ea92 fix(rating): floor the continuous SAP score at 1 (SAP 10.2 §13 / RdSAP 10 §13)
The SAP rating is spec-floored at 1 ("if the result of the calculation is
less than 1, the rating is 1"). `sap_rating_integer` already clamps, but the
continuous `sap_score_continuous` did not — so a degenerate dwelling could
emit a physically impossible negative SAP. Apply the same max(1, …) floor to
the continuous value (the un-rounded part is for sensitivity near real
ratings, not for negative ratings).

Removes a -12.3 accuracy outlier on the committed corpus (cert 422000111926,
lodged at the floor of 1, was computing -11.3): within-0.5 70.2% -> 70.3%,
MAE 0.845 -> 0.833. Ratcheted the corpus MAE ceiling to 0.84. Unit-pinned in
test_calculator.

pyright not installed in this codespace (strict gate not run locally).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:03:13 +00:00
Jun-te Kim
8f7a344707 Resolve a landlord from-main-gas water-heating override to its codes 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:02:48 +00:00
Jun-te Kim
46d1f8bbb2 Guarantee every classifier age-band value maps to an overlay 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:45:31 +00:00
Jun-te Kim
fc591c6550 Classify the landlord Age column into a construction-age-band category 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:43:47 +00:00
Jun-te Kim
e71df5df99 Re-date a building part's EPC age band from a landlord override 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:43:28 +00:00
Jun-te Kim
55cdc1c777 Route a construction_age_band override row through its overlay mapper 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:42:19 +00:00
Jun-te Kim
cd14751fdb Route age-band overrides to the right part, normalise, reject unknown 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:42:02 +00:00
Jun-te Kim
0135f0f27b Resolve a landlord age-band override onto the main building part 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:41:20 +00:00
Jun-te Kim
5a6d9a4e5d Guarantee every classifier glazing value maps to an overlay code 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:38:07 +00:00
Jun-te Kim
0b782bd1a6 Classify the landlord Glazing column into a glazing category 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:35:39 +00:00
Jun-te Kim
8ce22a5ccb Remap every window's glazing from a landlord glazing override 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:21:06 +00:00
Jun-te Kim
1810f09240 Route a glazing override row through the glazing overlay mapper 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:20:09 +00:00
Jun-te Kim
71bdcabb73 Produce no overlay for an unresolvable landlord glazing value 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:19:56 +00:00
Jun-te Kim
917627c833 Decode single/double/triple glazing overrides to SAP codes 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:19:33 +00:00
Jun-te Kim
f1c6825cae Resolve a landlord double-glazing override to its glazing code 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:18:45 +00:00
Jun-te Kim
fd922a26c2 Satisfy strict type-checking for the main_fuel classifier wiring 🟪
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:01:07 +00:00
Jun-te Kim
cab69bbca9 Guarantee every classifier fuel value maps to an overlay code 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:45:22 +00:00
Jun-te Kim
04dd2dd222 Classify the landlord Main Fuel column into a fuel category 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:43:54 +00:00
Jun-te Kim
3dab6fc425 Remap the EPC primary fuel from a landlord fuel override 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:24:31 +00:00
Jun-te Kim
991eb74132 Route a main_fuel override row through the fuel overlay mapper 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:23:37 +00:00