Commit graph

6784 commits

Author SHA1 Message Date
Jun-te Kim
71bdcabb73 Produce no overlay for an unresolvable landlord glazing value 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:19:56 +00:00
Jun-te Kim
88a4badbf7 Decode single/double/triple glazing overrides to SAP codes 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:19:44 +00:00
Jun-te Kim
917627c833 Decode single/double/triple glazing overrides to SAP codes 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:19:33 +00:00
Jun-te Kim
0fb86771dd Resolve a landlord double-glazing override to its glazing code 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:19:02 +00:00
Jun-te Kim
f1c6825cae Resolve a landlord double-glazing override to its glazing code 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:18:45 +00:00
Jun-te Kim
fd922a26c2 Satisfy strict type-checking for the main_fuel classifier wiring 🟪
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:01:07 +00:00
Jun-te Kim
cab69bbca9 Guarantee every classifier fuel value maps to an overlay code 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:45:22 +00:00
Jun-te Kim
dbfe9820a5 Classify the landlord Main Fuel column into a fuel category 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:44:56 +00:00
Jun-te Kim
04dd2dd222 Classify the landlord Main Fuel column into a fuel category 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:43:54 +00:00
Jun-te Kim
3dab6fc425 Remap the EPC primary fuel from a landlord fuel override 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:24:31 +00:00
Jun-te Kim
b85db926ed Route a main_fuel override row through the fuel overlay mapper 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:23:52 +00:00
Jun-te Kim
991eb74132 Route a main_fuel override row through the fuel overlay mapper 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:23:37 +00:00
Jun-te Kim
35b48cc8fc Produce no overlay for an unresolvable landlord fuel value 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:23:25 +00:00
Jun-te Kim
07aea7ef9b Map community mains gas to its distinct fuel code 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:23:06 +00:00
Jun-te Kim
39950d410c Map community mains gas to its distinct fuel code 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:22:56 +00:00
Jun-te Kim
a402a3858e Decode landlord electricity/LPG/oil/coal overrides to fuel codes 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:22:44 +00:00
Jun-te Kim
7f8e2762d0 Decode landlord electricity/LPG/oil/coal overrides to fuel codes 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:22:32 +00:00
Jun-te Kim
0e85da1507 Resolve a landlord mains-gas override to the primary fuel code 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:15:54 +00:00
Jun-te Kim
240d7b1025 Resolve a landlord mains-gas override to the primary fuel code 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:07:32 +00:00
Jun-te Kim
360e069386 hubspot etl 20 in parrell 2026-06-19 11:07:30 +00:00
Jun-te Kim
37b0a38425 add more test cases 2026-06-19 10:59:51 +00:00
Jun-te Kim
e6a829aaea more examples 2026-06-19 09:51:49 +00:00
Jun-te Kim
269a7fdaa7 Merge branch 'main' of https://github.com/Hestia-Homes/Model into feature/hyde_make_it_more_accurate_with_tests 2026-06-18 16:24:26 +00:00
Jun-te Kim
4959b13eda
Merge pull request #1250 from Hestia-Homes/feature/per-cert-mapper-validation
Feature/per cert mapper validation
2026-06-18 16:26:45 +01:00
Khalim Conn-Kowlessar
2e416a0221 chore: untrack .claude/settings.json (machine-specific personal config)
`.claude/settings.json` was committed to main by mistake — it holds
per-developer permission allow-lists (npx cache paths, /tmp script paths,
and even hardcoded credentials), not shared project config. Mirror the
existing `.claude/settings.local.json` treatment: remove it from the index
and add it to .gitignore so each developer keeps their own local copy.
Claude Code merges settings.json + settings.local.json at runtime, so no
permissions are lost.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 15:01:26 +00:00
Khalim Conn-Kowlessar
a7de8c5c35 Merge remote-tracking branch 'origin/main' into feature/per-cert-mapper-validation 2026-06-18 15:00:32 +00:00
Khalim Conn-Kowlessar
8942d45772 fix(fuel): price secondary dual-fuel/anthracite at their own rate, not the colliding LPG code (RdSAP 10 Table 32)
The gov-API lodges secondary fuel as an enum whose value can COLLIDE with a
different same-valued RdSAP 10 Table 32 / SAP 10.2 Table 12 fuel code:
  - enum 9  = "dual fuel (mineral and wood)"  vs Table code 9  = LPG SC11F
  - enum 5  = "anthracite"                    vs Table code 5  = LPG (bulk)
The main-fuel boundary already canonicalises these (`_GOV_API_COLLISION_
FUELS`), but the SECONDARY-heating cost + CO2/PE paths never did — they took
the bare same-value lookup, so a dual-fuel room heater was priced as LPG
(3.48 vs dual-fuel 3.99 p/kWh) and emitted as LPG (CO2 0.241 vs 0.087),
and an anthracite secondary as bulk LPG (12.19 vs 3.64 p/kWh). The price
under-count over-rates SAP; the CO2 over-count inflates emissions.

Fix: add enum 9 to `_GOV_API_COLLISION_FUELS` (5 and 33 were already there)
and canonicalise the secondary fuel code on both the cost
(`_secondary_fuel_cost_gbp_per_kwh`) and factor (`_secondary_fuel_code`)
paths, mirroring the main-fuel boundary. canonical_fuel_code only touches
{5,9,33}, so genuinely Table-coded secondaries (House coal 11, wood logs 20,
community fuels 30-32) are left unchanged — confirmed by a full-map audit.

Corpus: within-0.5 69.7% -> 70.2% (MAE 0.854 -> 0.845; dual-fuel-secondary
cohort 42.9% -> 49.0%, signed +0.55 -> +0.41) and CO2 MAE 0.12 -> 0.08 t/yr
(bias +0.04 -> 0.00). Ratcheted the corpus floors (within 0.70, MAE 0.85,
CO2 0.09, PE 4.0). A prior session deferred enum 9 ("direction not
understood") while the EPC PE/CO2 lens was confounded by the climate-cascade
bug (fc7c4d2d); on the corrected lens the over-rate direction is clear.

pyright not installed in this codespace (strict gate not run locally).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 14:52:58 +00:00
Khalim Conn-Kowlessar
6950deae06 chore(scripts): triage PE/CO2 on the demand cascade in the corpus profilers
`profile_corpus_error.py` and `dive_cert.py` compared our PE/CO2 against
the lodged EPC figures using the UK-average RATING cascade, but the EPC
lodges CO2/PE on the postcode DEMAND cascade (SAP 10.2 Appendix U p.124,
now wired into Sap10Calculator.calculate in fc7c4d2d). That confounded the
DEMAND-vs-COST triage: a cert whose demand actually reproduced on local
weather looked "PE off" purely from the climate difference and was
mislabelled DEMAND-side. Switching the PE/CO2 lens to `cert_to_demand_
inputs` (SAP still from the rating cascade) re-classifies the corpus
outside-0.5 set 261/42 -> 211/92 DEMAND/COST — ~50 certs are genuinely
cost-side (e.g. 10091578598: SAP +7.81 but PE +1.6 / CO2 -0.04). Sharpens
the hunt for the subtle widespread SAP term.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 14:21:00 +00:00
Khalim Conn-Kowlessar
fc7c4d2d3b fix(climate): compute EPC CO2/PE on the postcode demand cascade (SAP 10.2 Appendix U p.124)
The SAP/EI rating is computed on UK-average weather (Appendix U Tables
U1-U3 region 0) so ratings are nationally comparable, but Appendix U
paragraph 1 (PDF p.124) requires that "other calculations (such as for
energy use and costs on EPCs) are done using local weather. Weather data
for each postcode district are taken from the PCDB". `Sap10Calculator.
calculate` ran ONE cascade (UK-average) and fed it to SAP, CO2 AND primary
energy, so every cert's EPC-displayed CO2/PE were computed on the wrong
climate. Because most of England is warmer than the UK-average, this
systematically OVER-counted heating demand on the emissions/PE outputs.

The two cascades (`cert_to_inputs` rating, `cert_to_demand_inputs`
postcode) already existed; this wires the demand cascade into the
production entry point and grafts its CO2/PE onto the rating result (SAP
unchanged). The corpus gauge's longstanding +5% CO2/PE over-estimate was
mostly this climate bug, NOT (as previously diagnosed) per-cert mapper
fidelity:
  CO2 MAE 0.26 -> 0.12 t/yr  (bias +0.18 -> +0.04)
  PE  MAE 13.6 -> 3.8 kWh/m2 (bias +9.0  -> +0.24)
  SAP within-0.5 = 69.7% (rating cascade, unchanged)

Worksheet-validated to 1e-4 on simulated case 45 (heat-pump ground-floor
flat, postcode W6): the P960 prints the current dwelling twice — Block 1
on UK-average weather (SAP 60.5318, CO2 692.13) and Block 2 on postcode
weather (CO2 626.78, PE 6581.59). Both reproduce exactly. Added a tracked
case-45 Summary fixture + two-cascade cascade pin as a permanent guard,
and ratcheted the corpus CO2/PE ceilings to 0.13 / 4.2. The e2e Elmhurst
suite (Block-1 line refs) now pins the rating cascade directly; the two
Vaillant overlay snapshots refreshed to demand-cascade CO2/PE.

pyright not installed in this codespace (strict gate not run locally);
change is type-trivial (dataclasses.replace over SapResult).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 14:15:34 +00:00
Khalim Conn-Kowlessar
26106505be chore(scripts): add corpus SAP-accuracy profiler + per-cert dive tools
profile_corpus_error.py buckets signed SAP error by raw-API feature and
lists worst over/under-raters with the PE/CO2-vs-cost split (COST-side vs
DEMAND-side triage). dive_cert.py dumps one cert's lodged-vs-ours
SAP/CO2/PE + full intermediate line refs + mapped inputs. Both run on the
committed RdSAP-21.0.1 corpus (no /tmp sample needed). Used to find the
stone-wall, per-part-roof, ground-floor-flat and HP-water fixes this session.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 13:48:12 +00:00
Khalim Conn-Kowlessar
72ef0f0e7b fix(water): don't apply heat-pump water SCOP to a separate immersion (SAP N3.7a)
When a heat-pump cert lodges a PCDB Table 362 record, the APM override
set BOTH the space efficiency (N3.6) and the water efficiency (N3.7a)
from the heat pump unconditionally. But the PCDB η_water applies only
when the DHW is heated BY the heat pump (water-heating code "from main":
901/902/914). A separate electric immersion (WHC 903) heats the water at
100% regardless of the space system, so applying the HP's water SCOP
(187.5% × 0.6 in-use = 112.5%) under-counted the immersion's hot-water
fuel.

Gate the η_water override on the DHW-from-main codes; a separate immersion
keeps its own 100% efficiency. Space η_space still always uses the APM
value (the heat pump is the space main).

Worksheet-validated to 1e-4 on simulated case 45 (HP space + WHC-903
immersion): water fuel (62) 1893.57 -> 2130.2639, total cost (255)
619.7433, CO2 692.13 — all matching the P960 exactly; SAP 60.53 -> rounds
to the worksheet's 61. RdSAP-21.0.1 corpus unchanged (no HP+WHC903 certs
in it). Pinned in test_cert_to_inputs (immersion fuel is main-independent).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 13:44:51 +00:00
Khalim Conn-Kowlessar
9b0c590bf8 fix(heat-transmission): bill a ground-floor flat's ground floor (RdSAP 10 §3.12)
The flat floor-exposure heuristic keys on dwelling_type: a flat defaults
to has_exposed_floor=False (assuming a heated dwelling below). The
Elmhurst Summary path lodges a ground-floor flat's vertical position as a
"Ground floor" floor_type rather than the API floor_heat_loss=1 exposed
code, and the mapper can label such a flat "Top-floor flat" — so the
cascade dropped the ground floor entirely (a ground floor is in contact
with the ground and carries heat loss).

Treat a "ground floor" floor_type as a heat-loss floor, overriding the
dwelling-level suppression upward — mirroring the existing "another
dwelling below" party override downward.

Worksheet-validated to 1e-4 on simulated case 45 (a ground-floor flat
the mapper labelled "Top-floor flat"): floor (28a) 0 -> 25.38 W/K,
fabric (33) 75.63 -> 101.0104, HTC (39) 112.93 -> 145.3579, all matching
the P960 exactly; SAP 67.81 -> 62.52. RdSAP-21.0.1 corpus within-0.5
69.5% -> 69.7% (MAE 0.859 -> 0.854). Floors ratcheted. Pinned in
test_heat_transmission (ground-floor billed + party-floor suppressed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 13:16:44 +00:00
Jun-te Kim
f0411b2cf1 Merge https://github.com/Hestia-Homes/Model into feature/hyde_make_it_more_accurate_with_tests 2026-06-18 10:33:27 +00:00
Jun-te Kim
0972090183
Merge pull request #1249 from Hestia-Homes/feature/landlord-overrides
Feature/landlord overrides
2026-06-18 11:29:07 +01:00
Jun-te Kim
b07472cf38 sap calculator variaince changes 2026-06-18 10:22:21 +00:00
Jun-te Kim
7f8bfa5d06 move overlas to its own thing 2026-06-18 10:04:14 +00:00
Jun-te Kim
3809dc4697 Merge branch 'main' of https://github.com/Hestia-Homes/Model into feature/landlord-overrides 2026-06-18 09:39:05 +00:00
Jun-te Kim
1d392d8d87 ll overrides@ 2026-06-17 18:56:57 +00:00
Jun-te Kim
1b070b6d8f Document all four mapped override components and the deferred (assumed) wall states
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 18:55:23 +00:00
Jun-te Kim
1c3b4a7b72 Correct ADR-0032 dec-4: property_type affects party-wall calc + eligibility, not just metadata
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 18:07:30 +00:00
Jun-te Kim
4219ef9d8b Overlay landlord property-type and built-form corrections onto the Effective EPC 🟩
Adds whole-dwelling property_type/built_form to EpcSimulation (folded by
apply_simulations) and maps those override components. property_type drives
party-wall heat loss + ASHP/solar/wall eligibility, so a landlord correction now
moves both the SAP calc and the measure menu; built_form has no calculator
consumer today (feeds the ML transform). Written as the landlord text value
(park-home check is text-only). Refines ADR-0032 dec-4.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 18:06:29 +00:00
Jun-te Kim
0305241ad3 Overlay more wall materials and roof loft-insulation depth from landlord overrides 🟩
Extends WallType coverage to timber/stone/system-built/cob/park-home/curtain and
adds RoofType "Pitched, N mm loft insulation" -> roof_insulation_thickness. The
"(assumed) insulated"/"partial" wall states stay deferred (ambiguous code, needs
Elmhurst validation per ADR-0032); property_type/built_form carry no SAP weight.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 17:15:50 +00:00
Jun-te Kim
5939520b0d Add a cell-by-cell inspector for landlord-override → effective-EPC mapping
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 17:04:44 +00:00
Jun-te Kim
86b5387a05 Show lodged vs effective main wall per property in the modelling e2e run
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 14:15:46 +00:00
Jun-te Kim
e58f6ce935
Merge pull request #1247 from Hestia-Homes/feature/landlord-overrides
Feature/landlord overrides
2026-06-17 14:41:51 +01:00
Jun-te Kim
3abf2f117a Merge branch 'main' of https://github.com/Hestia-Homes/Model into feature/landlord-overrides 2026-06-17 11:58:29 +00:00
Jun-te Kim
8abd5e8b54 better smoke tests to check what is failign 2026-06-17 11:53:33 +00:00
Jun-te Kim
a955a09e9c Pin uprn_10093116330 (full-SAP gas-combi 2-storey semi): engine 82 vs Elmhurst 78
7th sibling full-SAP cert; documented full-SAP→RdSAP +4 residual. Build clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 10:26:11 +00:00
Jun-te Kim
d08c35ee03 Merge branch 'feature/hyde_make_it_more_accurate_with_tests' into feature/landlord-overrides 2026-06-17 09:43:28 +00:00
Jun-te Kim
e519de26a4 Pin uprn_10093116336 (full-SAP gas-combi 2-storey semi): engine 83 vs Elmhurst 79
6th sibling full-SAP cert; same documented full-SAP→RdSAP +4 residual. Build clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 09:41:29 +00:00