Commit graph

6769 commits

Author SHA1 Message Date
Khalim Conn-Kowlessar
34e52a893c fix(heating): assume portable-electric secondary for unheated habitable rooms (SAP 10.2 Appendix A.2.2)
When the main heating system does not heat every habitable room (heated rooms
< habitable rooms), SAP 10.2 Appendix A.2.2 assumes the unheated rooms are
served by a portable-electric secondary heater, so the Table 11 secondary
fraction (0.10 for a boiler main) must be costed at the electricity tariff —
even when the cert lodges no explicit secondary system.

`_secondary_fraction` previously returned 0 unless a secondary was lodged or
the main was a forced-secondary electric-storage code, dropping the assumed
secondary and billing 100% of space heat to the (cheaper) main fuel — an
over-rate. Added an `unheated_habitable_rooms` trigger plus
`_has_unheated_habitable_rooms(epc)`, which prefers the lodged
`any_unheated_rooms` flag and guards the gov-API `heated_rooms_count == 0`
"not provided" sentinel. The secondary fuel/efficiency cascade already
defaults to portable electric (code 693) when no secondary code is lodged.

Worksheet-validated on simulated case 46 (heated 4 < habitable 7, no lodged
secondary): the assumed 10% electric secondary (2289 kWh, ~£260) lifted our
SAP 39 -> 29.35 vs accredited Elmhurst 30 (cost £1502 vs £1493, within 0.6%).

Corpus UNCHANGED (71.6% / MAE 0.819): all 17 corpus certs with heated <
habitable already lodge an explicit secondary description, so the gov-API
path was already costing it; this only adds the assumed secondary where none
is lodged (Elmhurst / reduced-field path). pyright not installed locally.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 06:15:37 +00:00
Khalim Conn-Kowlessar
d05fdbe6f2 fix(mapper): map truncated Elmhurst glazing label "Double between 2002" (RdSAP 10 Table 24 code 3)
The full RdSAP-Schema-21 label is "Double between 2002 and 2021" (double
glazing installed 2002-2021, SAP 10.2 Table 24 code 3). When the Elmhurst
Summary PDF wraps the trailing "and 2021" into an adjacent table cell that
the extractor joins away, the surfaced label truncates to "Double between
2002" — the same artifact already handled for "Triple post or during".
`_elmhurst_glazing_type_code` raised UnmappedElmhurstLabel on it, blocking
the whole Summary (surfaced on the simulated-case-46 multi-attribute
worksheet). Added the truncated form as a code-3 alias.

pyright not installed in this container.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 06:15:25 +00:00
Khalim Conn-Kowlessar
d7a60efcdf fix(uvalues): thread glazing gap into pre-2002 window U fallback (RdSAP 10 Table 24, PDF p.50)
`u_window` hard-coded the 12 mm gap row for pre-2002 double/triple glazing
(double 2.8, triple 2.1), ignoring the lodged glazing gap. Table 24 splits
the pre-2002 rows by gap: double 6mm=3.1 / 12mm=2.8 / 16mm+=2.7; triple
6mm=2.4 / 12mm=2.1 / 16mm+=2.0 (PVC/wooden), with a metal-frame column
(+0.5/+0.5/+0.5 ish). Added a `glazing_gap` parameter + `_glazing_gap_row`
helper and wired `w.glazing_gap` through the synthesised-window caller in
heat_transmission.

Corpus impact nil by design: the gov-API mapper already resolves per-window
U gap-aware via `_API_GLAZING_TYPE_GAP_TO_TRANSMISSION` (e.g. code 3 + gap
"16+" → 2.7), so corpus certs use that lodged per-window U, not this fallback.
This aligns the reduced-field / worksheet fallback path with the mapper and
Table 24. Unknown gap still defaults to the 12 mm row.

(Metal frames are not distinguishable on the gov-API path — only a `pvc_frame`
boolean exists and Table 24 groups PVC+wooden — so the PVC/wooden U stands
there; the metal column applies only where frame material is lodged.)

Spec-pinned: pre-2002 double + triple gap-row tests. pyright not installed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 14:14:31 +00:00
Khalim Conn-Kowlessar
600684f5df fix(uvalues): apply metal-frame U 1.6 for 2022-or-later windows (RdSAP 10 Table 24, PDF p.51)
Table 24 "Double or triple glazed, 2022 or later" row gives U = 1.4 for
PVC/wooden frames and 1.6 for metal frames. `u_window` returned 1.4 for both,
ignoring the metal-frame variant and under-counting metal-frame heat loss on
post-2022 windows. The 2002-2021 and pre-2002 rows already split PVC/metal
correctly; this aligns the 2022+ row.

Spec-pinned in test_u_window_post_2022_metal_returns_table24_1_6_not_pvc_1_4.
pyright not installed in this container.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 14:09:21 +00:00
Khalim Conn-Kowlessar
b22b27c0ff fix(uvalues): correct rafter-roof age-M default U 0.18->0.15 (RdSAP 10 Table 18 col 2, PDF p.46)
A full row-by-row audit of the roof U-value tables (16, 17, 18) and floor
tables (19, 20) against the PDF found one numeric error: Table 18 column (2)
"Pitched, insulation at rafters" band M is 0.15 W/m²K (footnote (1) only — no
country variation; the whole M row converges to 0.15), but _ROOF_RAFTERS_BY_AGE
carried 0.18. The rafters column diverges above the joist column at H-L
(0.35/0.35/0.20/0.20/0.18) and rejoins it at M=0.15.

Everything else in the roof/floor tables is exact: Table 16 joist + rafter
thickness ladders, Table 17 room-in-roof (all 6 columns), Table 18 cols (1)
joists / (3) flat / (4) room-in-roof, Table 19 (England & Wales) floor
insulation defaults, and Table 20 (England) exposed-floor U-values.

Known remaining gaps (NOT fixed — zero England-corpus reach, would need a new
roof country-override mechanism): Scotland Table 18 footnote (2) K=0.20 (flat/
RR/thatch) and (3) joists-L=0.15; Scotland/NI Table 19 thicknesses; Scotland/
Wales Table 20 L/M overrides. Logged for a future country pass.

Corpus unchanged (band-M rafter roofs essentially absent from the 1000-cert
England corpus). pyright not installed in this container.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 13:57:29 +00:00
Khalim Conn-Kowlessar
104e3725b8 test(corpus): ratchet SAP MAE 0.83->0.82 and PE 4.0->3.7 floors after stone-mechanism correction
The corrected stone branch (age-E-only cap, §3.5 Table-3 unknown-thickness
default, Scotland band-J override) improved the corpus gauge to SAP MAE 0.819
and PE MAE 3.6 kWh/m2/yr. Lock the gains in and record the corrected
mechanism in the provenance log; within-0.5 (71.6%) and CO2 (0.08) unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 13:34:31 +00:00
Khalim Conn-Kowlessar
381ecfda4c fix(uvalues): add missing Scotland band-J 0.30 wall override for all 7 as-built types (RdSAP 10 Table 7, PDF p.35)
A full code-vs-spec sweep of unknown-thickness as-built U-values across all
seven wall types found exactly one divergence: Scotland age band J. Table 7
(Scotland) gives 0.30 for every uninsulated wall type, but the England base
(Table 6) is 0.35 and the _COUNTRY_KLM_OVERRIDES[SCT] dicts listed H/K/L/M
while omitting J — so a Scotland band-J wall wrongly returned England's 0.35.

Added 'J': 0.30 to all 7 SCT as-built entries (granite, sandstone, solid
brick, cavity, timber frame, system built, cob).

The audit's companion finding — Scotland sandstone age E = 1.5 — needs NO
table override: the §5.6 stone branch intercepts all sandstone bands A-E and
already caps age E at 1.5 for Scotland (e_default in the prior commit), so a
table-E override would be dead code. England base tables and Isle of Man
(= England) are confirmed correct by the same sweep.

Corpus unchanged (England-only corpus has no Scotland band-J certs).
pyright not installed in this container — strict type gate not run locally.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 13:29:54 +00:00
Khalim Conn-Kowlessar
54ae05d04b fix(uvalues): default unknown-thickness stone to §3.5 Table 3 thickness, not flat 1.7 (RdSAP 10 Table 3, PDF p.20)
When a stone wall lodges no documentary thickness, RdSAP 10 §3.5 Table 3
supplies the default thickness to feed the §5.6 formula — it does NOT fall
to a flat 1.7. Table 3 stone row: A-D = 500 mm, E = 450, F-H = 420;
Scotland footnote (*) adds 200 mm for bands A,B and 100 mm otherwise.

This matches accredited Elmhurst: an England age-B granite/whinstone wall,
as built, unknown thickness defaults to 500 mm →
  U = 45.315 × 500^(-0.513) = 1.87   (sandstone → 1.68).
The previous flat-1.7 fallback was a setup error — Table 6 reads "According
to 5.6" for bands A-D with no 1.7 entry, so the formula (not 1.7) is the
spec target. The age-E footnote-(a) cap still applies on top.

Adds `_table_3_stone_thickness(band, country)`. Updated the stale
`..._returns_table_6_age_a_default` pin (was 1.7 → now 1.87 via Table-3)
and added sandstone (1.68) + Scotland-+200mm (700 mm → 1.39) pins.

Corpus: within-0.5 71.6% unchanged, SAP MAE 0.821 → 0.819, PE MAE 3.7 → 3.6.
pyright not installed in this container — strict type gate not run locally.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 13:26:36 +00:00
Khalim Conn-Kowlessar
7e187078b9 fix(uvalues): cap as-built stone U at age E only, not A-E; drop insulation-state gate (RdSAP 10 Tables 6-7 footnote a, PDF p.33-35)
Commit 034d4b7c corrected two stone cases but via the wrong mechanism: it
removed the 1.7 cap for ALL age bands and gated the §5.6 formula on
`wall_insulation_type is not None`. Both diverge from RdSAP 10 Tables 6-10:

- The stone rows read "According to 5.6" for bands A-D (uncapped formula)
  and "1.7 a" at band E, where footnote (a) = "Or from equations in 5.6 if
  the calculated U-value is less than 1.7" → U_E = min(formula, 1.7). The
  cap belongs ONLY at age E. Removing it for E let an as-built 50 mm granite
  age-E wall return the formula's 6.09 instead of 1.70.
- Scotland sandstone/limestone age E defaults to 1.5 (Table 7, PDF p.35);
  granite/whinstone stays 1.7.
- The insulation-state gate is not a spec rule. It sent age-A-D stone with
  "insulation Unknown" (wall_insulation_type None) to the flat-1.7 table
  instead of the §5.6 formula; it only "worked" for fixture 000565 Ext1
  because that wall is age E, where 1.7 is correct anyway. Removed: the
  age-E cap, not the gate, is what produces 000565 Ext1's worksheet 1.70.

Bands A-D stay uncapped (sandstone 400 mm → 1.90, granite 120 mm → 3.89).
Unknown-thickness handling unchanged in this commit (still flat-table; the
§3.5 Table-3 default follows separately).

pyright not installed in this container — strict type gate not run locally.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 13:24:31 +00:00
Khalim Conn-Kowlessar
034d4b7cd5 fix(uvalues): bill known-insulation stone walls by the §5.6 thickness formula, uncapped (RdSAP 10 §5.6)
An uninsulated stone wall of lodged thickness, age bands A-E, is billed by
the RdSAP 10 §5.6 Table-12 formula on its measured thickness — sandstone /
limestone U = 54.876·W^-0.561, granite / whinstone U = 45.315·W^-0.513. Two
bugs suppressed it:

1. CAP: the as-built branch capped the formula result at the Table-6
   typical-thickness default (`if u0 >= 1.7: return 1.7`). But the formula
   only dips below 1.7 past ~488 mm (sandstone) / ~640 mm (granite), so the
   cap nullified §5.6 for essentially every real-thickness stone wall,
   under-counting fabric loss and over-rating. A measured 400 mm sandstone
   age-B wall is 1.90 (Elmhurst-confirmed), not 1.70.

2. NO INSULATION-STATE GATE: the branch fired for any stone wall with a
   lodged thickness, including ones whose insulation is "Unknown". RdSAP
   treats an unknown-insulation wall via the Table-6 typical-thickness
   default, NOT as bare stone of the lodged thickness — so a 50 mm granite
   wall with Unknown insulation must read 1.70 (worksheet), not the formula's
   6.09. Gated on `wall_insulation_type is not None`.

Fixes the 2 long-standing stone-U unit tests (granite 120 mm → 3.8871,
sandstone 120 mm → 3.7408 — they were correct red tests, not "known fails").
Corpus within-0.5 70.3% -> 71.6% (MAE 0.833 -> 0.822); ratcheted floors to
0.71 / 0.83. Worksheet fixture 000565 (granite 50 mm Unknown → 1.70) still
passes via the insulation gate. The two stone groups (sandstone/limestone vs
granite/whinstone) keep their distinct §5.6 coefficients.

pyright not installed in this codespace (strict gate not run locally).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:32:31 +00:00
Khalim Conn-Kowlessar
fc5f10ea92 fix(rating): floor the continuous SAP score at 1 (SAP 10.2 §13 / RdSAP 10 §13)
The SAP rating is spec-floored at 1 ("if the result of the calculation is
less than 1, the rating is 1"). `sap_rating_integer` already clamps, but the
continuous `sap_score_continuous` did not — so a degenerate dwelling could
emit a physically impossible negative SAP. Apply the same max(1, …) floor to
the continuous value (the un-rounded part is for sensitivity near real
ratings, not for negative ratings).

Removes a -12.3 accuracy outlier on the committed corpus (cert 422000111926,
lodged at the floor of 1, was computing -11.3): within-0.5 70.2% -> 70.3%,
MAE 0.845 -> 0.833. Ratcheted the corpus MAE ceiling to 0.84. Unit-pinned in
test_calculator.

pyright not installed in this codespace (strict gate not run locally).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:03:13 +00:00
Khalim Conn-Kowlessar
2e416a0221 chore: untrack .claude/settings.json (machine-specific personal config)
`.claude/settings.json` was committed to main by mistake — it holds
per-developer permission allow-lists (npx cache paths, /tmp script paths,
and even hardcoded credentials), not shared project config. Mirror the
existing `.claude/settings.local.json` treatment: remove it from the index
and add it to .gitignore so each developer keeps their own local copy.
Claude Code merges settings.json + settings.local.json at runtime, so no
permissions are lost.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 15:01:26 +00:00
Khalim Conn-Kowlessar
a7de8c5c35 Merge remote-tracking branch 'origin/main' into feature/per-cert-mapper-validation 2026-06-18 15:00:32 +00:00
Khalim Conn-Kowlessar
8942d45772 fix(fuel): price secondary dual-fuel/anthracite at their own rate, not the colliding LPG code (RdSAP 10 Table 32)
The gov-API lodges secondary fuel as an enum whose value can COLLIDE with a
different same-valued RdSAP 10 Table 32 / SAP 10.2 Table 12 fuel code:
  - enum 9  = "dual fuel (mineral and wood)"  vs Table code 9  = LPG SC11F
  - enum 5  = "anthracite"                    vs Table code 5  = LPG (bulk)
The main-fuel boundary already canonicalises these (`_GOV_API_COLLISION_
FUELS`), but the SECONDARY-heating cost + CO2/PE paths never did — they took
the bare same-value lookup, so a dual-fuel room heater was priced as LPG
(3.48 vs dual-fuel 3.99 p/kWh) and emitted as LPG (CO2 0.241 vs 0.087),
and an anthracite secondary as bulk LPG (12.19 vs 3.64 p/kWh). The price
under-count over-rates SAP; the CO2 over-count inflates emissions.

Fix: add enum 9 to `_GOV_API_COLLISION_FUELS` (5 and 33 were already there)
and canonicalise the secondary fuel code on both the cost
(`_secondary_fuel_cost_gbp_per_kwh`) and factor (`_secondary_fuel_code`)
paths, mirroring the main-fuel boundary. canonical_fuel_code only touches
{5,9,33}, so genuinely Table-coded secondaries (House coal 11, wood logs 20,
community fuels 30-32) are left unchanged — confirmed by a full-map audit.

Corpus: within-0.5 69.7% -> 70.2% (MAE 0.854 -> 0.845; dual-fuel-secondary
cohort 42.9% -> 49.0%, signed +0.55 -> +0.41) and CO2 MAE 0.12 -> 0.08 t/yr
(bias +0.04 -> 0.00). Ratcheted the corpus floors (within 0.70, MAE 0.85,
CO2 0.09, PE 4.0). A prior session deferred enum 9 ("direction not
understood") while the EPC PE/CO2 lens was confounded by the climate-cascade
bug (fc7c4d2d); on the corrected lens the over-rate direction is clear.

pyright not installed in this codespace (strict gate not run locally).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 14:52:58 +00:00
Khalim Conn-Kowlessar
6950deae06 chore(scripts): triage PE/CO2 on the demand cascade in the corpus profilers
`profile_corpus_error.py` and `dive_cert.py` compared our PE/CO2 against
the lodged EPC figures using the UK-average RATING cascade, but the EPC
lodges CO2/PE on the postcode DEMAND cascade (SAP 10.2 Appendix U p.124,
now wired into Sap10Calculator.calculate in fc7c4d2d). That confounded the
DEMAND-vs-COST triage: a cert whose demand actually reproduced on local
weather looked "PE off" purely from the climate difference and was
mislabelled DEMAND-side. Switching the PE/CO2 lens to `cert_to_demand_
inputs` (SAP still from the rating cascade) re-classifies the corpus
outside-0.5 set 261/42 -> 211/92 DEMAND/COST — ~50 certs are genuinely
cost-side (e.g. 10091578598: SAP +7.81 but PE +1.6 / CO2 -0.04). Sharpens
the hunt for the subtle widespread SAP term.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 14:21:00 +00:00
Khalim Conn-Kowlessar
fc7c4d2d3b fix(climate): compute EPC CO2/PE on the postcode demand cascade (SAP 10.2 Appendix U p.124)
The SAP/EI rating is computed on UK-average weather (Appendix U Tables
U1-U3 region 0) so ratings are nationally comparable, but Appendix U
paragraph 1 (PDF p.124) requires that "other calculations (such as for
energy use and costs on EPCs) are done using local weather. Weather data
for each postcode district are taken from the PCDB". `Sap10Calculator.
calculate` ran ONE cascade (UK-average) and fed it to SAP, CO2 AND primary
energy, so every cert's EPC-displayed CO2/PE were computed on the wrong
climate. Because most of England is warmer than the UK-average, this
systematically OVER-counted heating demand on the emissions/PE outputs.

The two cascades (`cert_to_inputs` rating, `cert_to_demand_inputs`
postcode) already existed; this wires the demand cascade into the
production entry point and grafts its CO2/PE onto the rating result (SAP
unchanged). The corpus gauge's longstanding +5% CO2/PE over-estimate was
mostly this climate bug, NOT (as previously diagnosed) per-cert mapper
fidelity:
  CO2 MAE 0.26 -> 0.12 t/yr  (bias +0.18 -> +0.04)
  PE  MAE 13.6 -> 3.8 kWh/m2 (bias +9.0  -> +0.24)
  SAP within-0.5 = 69.7% (rating cascade, unchanged)

Worksheet-validated to 1e-4 on simulated case 45 (heat-pump ground-floor
flat, postcode W6): the P960 prints the current dwelling twice — Block 1
on UK-average weather (SAP 60.5318, CO2 692.13) and Block 2 on postcode
weather (CO2 626.78, PE 6581.59). Both reproduce exactly. Added a tracked
case-45 Summary fixture + two-cascade cascade pin as a permanent guard,
and ratcheted the corpus CO2/PE ceilings to 0.13 / 4.2. The e2e Elmhurst
suite (Block-1 line refs) now pins the rating cascade directly; the two
Vaillant overlay snapshots refreshed to demand-cascade CO2/PE.

pyright not installed in this codespace (strict gate not run locally);
change is type-trivial (dataclasses.replace over SapResult).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 14:15:34 +00:00
Khalim Conn-Kowlessar
26106505be chore(scripts): add corpus SAP-accuracy profiler + per-cert dive tools
profile_corpus_error.py buckets signed SAP error by raw-API feature and
lists worst over/under-raters with the PE/CO2-vs-cost split (COST-side vs
DEMAND-side triage). dive_cert.py dumps one cert's lodged-vs-ours
SAP/CO2/PE + full intermediate line refs + mapped inputs. Both run on the
committed RdSAP-21.0.1 corpus (no /tmp sample needed). Used to find the
stone-wall, per-part-roof, ground-floor-flat and HP-water fixes this session.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 13:48:12 +00:00
Khalim Conn-Kowlessar
72ef0f0e7b fix(water): don't apply heat-pump water SCOP to a separate immersion (SAP N3.7a)
When a heat-pump cert lodges a PCDB Table 362 record, the APM override
set BOTH the space efficiency (N3.6) and the water efficiency (N3.7a)
from the heat pump unconditionally. But the PCDB η_water applies only
when the DHW is heated BY the heat pump (water-heating code "from main":
901/902/914). A separate electric immersion (WHC 903) heats the water at
100% regardless of the space system, so applying the HP's water SCOP
(187.5% × 0.6 in-use = 112.5%) under-counted the immersion's hot-water
fuel.

Gate the η_water override on the DHW-from-main codes; a separate immersion
keeps its own 100% efficiency. Space η_space still always uses the APM
value (the heat pump is the space main).

Worksheet-validated to 1e-4 on simulated case 45 (HP space + WHC-903
immersion): water fuel (62) 1893.57 -> 2130.2639, total cost (255)
619.7433, CO2 692.13 — all matching the P960 exactly; SAP 60.53 -> rounds
to the worksheet's 61. RdSAP-21.0.1 corpus unchanged (no HP+WHC903 certs
in it). Pinned in test_cert_to_inputs (immersion fuel is main-independent).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 13:44:51 +00:00
Khalim Conn-Kowlessar
9b0c590bf8 fix(heat-transmission): bill a ground-floor flat's ground floor (RdSAP 10 §3.12)
The flat floor-exposure heuristic keys on dwelling_type: a flat defaults
to has_exposed_floor=False (assuming a heated dwelling below). The
Elmhurst Summary path lodges a ground-floor flat's vertical position as a
"Ground floor" floor_type rather than the API floor_heat_loss=1 exposed
code, and the mapper can label such a flat "Top-floor flat" — so the
cascade dropped the ground floor entirely (a ground floor is in contact
with the ground and carries heat loss).

Treat a "ground floor" floor_type as a heat-loss floor, overriding the
dwelling-level suppression upward — mirroring the existing "another
dwelling below" party override downward.

Worksheet-validated to 1e-4 on simulated case 45 (a ground-floor flat
the mapper labelled "Top-floor flat"): floor (28a) 0 -> 25.38 W/K,
fabric (33) 75.63 -> 101.0104, HTC (39) 112.93 -> 145.3579, all matching
the P960 exactly; SAP 67.81 -> 62.52. RdSAP-21.0.1 corpus within-0.5
69.5% -> 69.7% (MAE 0.859 -> 0.854). Floors ratcheted. Pinned in
test_heat_transmission (ground-floor billed + party-floor suppressed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 13:16:44 +00:00
Jun-te Kim
0972090183
Merge pull request #1249 from Hestia-Homes/feature/landlord-overrides
Feature/landlord overrides
2026-06-18 11:29:07 +01:00
Jun-te Kim
b07472cf38 sap calculator variaince changes 2026-06-18 10:22:21 +00:00
Jun-te Kim
7f8bfa5d06 move overlas to its own thing 2026-06-18 10:04:14 +00:00
Jun-te Kim
3809dc4697 Merge branch 'main' of https://github.com/Hestia-Homes/Model into feature/landlord-overrides 2026-06-18 09:39:05 +00:00
Jun-te Kim
1d392d8d87 ll overrides@ 2026-06-17 18:56:57 +00:00
Jun-te Kim
1b070b6d8f Document all four mapped override components and the deferred (assumed) wall states
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 18:55:23 +00:00
Jun-te Kim
1c3b4a7b72 Correct ADR-0032 dec-4: property_type affects party-wall calc + eligibility, not just metadata
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 18:07:30 +00:00
Jun-te Kim
4219ef9d8b Overlay landlord property-type and built-form corrections onto the Effective EPC 🟩
Adds whole-dwelling property_type/built_form to EpcSimulation (folded by
apply_simulations) and maps those override components. property_type drives
party-wall heat loss + ASHP/solar/wall eligibility, so a landlord correction now
moves both the SAP calc and the measure menu; built_form has no calculator
consumer today (feeds the ML transform). Written as the landlord text value
(park-home check is text-only). Refines ADR-0032 dec-4.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 18:06:29 +00:00
Jun-te Kim
0305241ad3 Overlay more wall materials and roof loft-insulation depth from landlord overrides 🟩
Extends WallType coverage to timber/stone/system-built/cob/park-home/curtain and
adds RoofType "Pitched, N mm loft insulation" -> roof_insulation_thickness. The
"(assumed) insulated"/"partial" wall states stay deferred (ambiguous code, needs
Elmhurst validation per ADR-0032); property_type/built_form carry no SAP weight.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 17:15:50 +00:00
Jun-te Kim
5939520b0d Add a cell-by-cell inspector for landlord-override → effective-EPC mapping
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 17:04:44 +00:00
Jun-te Kim
86b5387a05 Show lodged vs effective main wall per property in the modelling e2e run
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 14:15:46 +00:00
Jun-te Kim
e58f6ce935
Merge pull request #1247 from Hestia-Homes/feature/landlord-overrides
Feature/landlord overrides
2026-06-17 14:41:51 +01:00
Jun-te Kim
3abf2f117a Merge branch 'main' of https://github.com/Hestia-Homes/Model into feature/landlord-overrides 2026-06-17 11:58:29 +00:00
Jun-te Kim
8abd5e8b54 better smoke tests to check what is failign 2026-06-17 11:53:33 +00:00
Jun-te Kim
d08c35ee03 Merge branch 'feature/hyde_make_it_more_accurate_with_tests' into feature/landlord-overrides 2026-06-17 09:43:28 +00:00
Jun-te Kim
e519de26a4 Pin uprn_10093116336 (full-SAP gas-combi 2-storey semi): engine 83 vs Elmhurst 79
6th sibling full-SAP cert; same documented full-SAP→RdSAP +4 residual. Build clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 09:41:29 +00:00
Jun-te Kim
317220beba Pin uprn_10093116334 (full-SAP gas-combi bungalow): engine 81 vs Elmhurst 77
5th sibling full-SAP cert validated against Elmhurst (semi-detached bungalow,
same street/boiler PCDB 17505 as 10093116324). Engine 81 (lodged 82); Elmhurst
worksheet 77. The +4 is the documented full-SAP→RdSAP residual. Build verified
clean (storeys=1, no phantom conservatory). Worklist strategy note added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 09:32:57 +00:00
Jun-te Kim
8d4cbcccea
Merge pull request #1242 from Hestia-Homes/feature/landlord-overrides
Feature/landlord overrides
2026-06-17 10:31:52 +01:00
Jun-te Kim
f226570b0f upgraded python version 2026-06-17 09:28:24 +00:00
Jun-te Kim
3eb0022034 Pin uprn_10093116324 (full-SAP gas-combi bungalow): engine 79 vs Elmhurst 74
4th sibling full-SAP cert validated against Elmhurst. Engine 79 (lodged 80);
Elmhurst worksheet 74. The +5 is the documented full-SAP→RdSAP residual — engine
uses the cert's measured U-values (wall 0.19/floor 0.12/roof 0.12) + PCDB combi
17505 (88.5%); Elmhurst uses RdSAP band-L defaults + generic 84% BGW combi. Build
verified clean (single-storey bungalow, no phantom conservatory, TFA 52/51.9).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 09:10:36 +00:00
Jun-te Kim
d87718f316 Merge remote-tracking branch 'origin/main' into feature/hyde_make_it_more_accurate_with_tests
# Conflicts:
#	datatypes/epc/domain/mapper.py
2026-06-17 09:05:37 +00:00
Jun-te Kim
70ca56415e Merge branch 'main' of https://github.com/Hestia-Homes/Model into feature/landlord-overrides 2026-06-17 08:50:52 +00:00
Jun-te Kim
74c1aff530
Merge pull request #1245 from Hestia-Homes/feature/per-cert-mapper-validation
Feature/per cert mapper validation
2026-06-17 09:49:36 +01:00
Jun-te Kim
b11864a990
Merge pull request #1244 from Hestia-Homes/feature/e2e-runs
Feature/e2e runs
2026-06-17 09:48:55 +01:00
Khalim Conn-Kowlessar
c57ee578de fix(modelling): mirror the FE-owned property_baseline_performance columns
The SQLModel had drifted to a `bill_` prefix on the Bill Derivation block, but
the FE-owned Drizzle table uses unprefixed names (`heating_kwh`, `hot_water_kwh`
… `total_annual_bill_gbp`) plus a nullable `fuel_rates_period`. INSERTs failed
with UndefinedColumn. Rename the columns to mirror the live table column-for-
column (the prefix's anti-clash purpose is moot: `heating_kwh` != the recorded
`space_heating_kwh`), and add the `fuel_rates_period` column — left None until
Bill Derivation threads the snapshot period through.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 01:45:42 +00:00
Khalim Conn-Kowlessar
edf1003dcf fix(epc): hydrate recorded performance, RHI, and dates on read
The Baseline stage is the first consumer to read these off a persisted EPC
end-to-end, surfacing three gaps that only manifest on real API data:

- Only the 21.0.1 mapper copied through the recorded current-performance
  scalars (SAP rating, CO2, PEUI) and *no* mapper mapped the EPC band, so
  Lodged Performance raised for 17.x/18.0/19.0/20.0.0 certs. Overlay all four
  from the raw payload in `from_api_response`, once, for every schema version.
- Likewise the `renewable_heat_incentive` block (baseline space/water-heating
  kWh) was only mapped by the 21.x paths. Gap-fill it centrally from the raw
  payload when a mapper left it unset.
- The FE-owned `epc_property` date columns are Postgres `timestamp`s while the
  SQLModel mirror types them `str`, so a read hands back a `datetime` and
  `date.fromisoformat()` raised. Normalise via `_as_date()`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 01:45:18 +00:00
Khalim Conn-Kowlessar
e136e937d6 fix(heat-transmission): match roof description per part by kind (RdSAP 10 §5.11)
The deduplicated `epc.roofs[]` list cannot be indexed 1:1 against the
building parts (190/329 multi-part certs have len(roofs) != len(parts)),
so every part's `u_roof` consumed a SINGLE join of all roof descriptions.
That leaked one part's insulation state onto another: a "Flat, no
insulation" extension dragged a "Pitched, insulated (assumed)" main roof
to the uninsulated 2.30, ~3x over-stating its heat loss. 3-part certs
systematically under-rated (56% within-0.5, mean -0.79 SAP).

Partition the non-RR roof descriptions into flat vs pitched/sloping and
match each part to its own kind (`_main_roof_descriptions_by_kind`),
falling back to the global join when a part's kind has no matching entry.

Corpus cert 100010129331: roof 110.5 -> 31.3 W/K, +13.10 -> -0.05 SAP.
RdSAP-21.0.1 within-0.5 68.8% -> 69.5% (MAE 0.888 -> 0.859; PE 13.9 ->
13.6); 3-part cohort 56% -> 61%. Floors/ceilings ratcheted. Pinned in
test_heat_transmission (by_kind split + mixed-roof no-contamination).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 00:48:50 +00:00
Khalim Conn-Kowlessar
c5aa5620ca fix(uvalues): apply §5.8 insulation R to stone walls (RdSAP 10 p.41-42)
The §5.8 Table-14 added-insulation R-value adjustment was gated to
WALL_SOLID_BRICK, so a stone (granite/sandstone) wall lodging
wall_insulation_type 1/3 ("External"/"Internal") + a thickness fell
through the §5.6 thin-wall branch and was billed at its UNINSULATED U
(e.g. sandstone 520 mm + 100 mm internal: 1.64 instead of 0.30 → ~5×
the wall heat loss). Mirror the brick insulation branch into the stone
block, feeding the RAW §5.6 U₀ into the §5.8 chain per the same rule the
brick branch and the dry-lined granite pin 000565 already follow (the
Table-6 footnote (a) 1.7 cap does not apply on the insulated path).

Corpus cert 100052159386 (sandstone 520 mm + 100 mm internal): -26.20 ->
-4.08 SAP, walls 300 -> 55 W/K. RdSAP-21.0.1 corpus within-0.5 68.6% ->
68.8% (SAP MAE 0.942 -> 0.888; PE MAE 14.3 -> 13.9; CO2 0.27 -> 0.26);
floors/ceilings ratcheted. Unit-pinned in test_rdsap_uvalues.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 00:26:25 +00:00
Khalim Conn-Kowlessar
ea72ee97bf feat(scripts): add full AraFirstRunPipeline local runner
scripts/run_first_run_e2e.py runs the real Ingestion -> Baseline -> Modelling
pipeline against the DB by composing build_first_run_pipeline + dispatch_first_run
with the live source clients (the Lambda handler can't run locally — its
_source_clients_from_env still raises, #1136). Unlike run_modelling_e2e it runs
real ingestion (persists EPC/spatial/solar) and has no inspect-only mode, so it's
gated behind --confirm (preview otherwise); measure scoping comes only from the
Scenario's exclusions (the pipeline threads no --measures), and the modelling
batch is all-or-nothing, both documented.

Extract the shared env/engine/S3 plumbing into scripts/e2e_common.py (public
load_env/build_engine/s3_parquet_reader) so both runners share one source and
neither imports the other's privates.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 23:45:23 +00:00
Khalim Conn-Kowlessar
0d24f5b13a Merge branch 'main' of https://github.com/Hestia-Homes/Model into feature/per-cert-mapper-validation 2026-06-16 23:43:16 +00:00
Khalim Conn-Kowlessar
d501535cbc fix(mapper): map dropped §6.1 non-separated conservatory (API)
The gov API lodges a NON-SEPARATED conservatory (conservatory_type=4) as a
glazed "building part" carrying only {floor_area, room_height,
double_glazed, glazed_perimeter} — no fabric, no floor dimensions. The
four fields were undeclared on the 21.0.1 SapBuildingPart, so `from_dict`
dropped them and the conservatory was silently lost: it billed no §6.1
window/rooflight/floor and added nothing to TFA (5 corpus certs over-rated
— too little heat loss → SAP too high).

Fix (21.0.1 schema + mapper):
- declare the four glazed fields on `SapBuildingPart`;
- `_api_sap_conservatory` builds `EpcPropertyData.sap_conservatory` from
  the glazed BP (identified by a lodged `glazed_perimeter`; only type-4
  conservatories lodge it — separated ones, §6.2, lodge nothing);
- exclude the glazed BP from the fabric building-part loop (it is billed
  by the §6.1 cascade, not as a dwelling part);
- `_total_floor_area_from_building_parts` adds the conservatory floor area
  to TFA (drives occupancy → §4/§5 demand).

Validation is cross-mapper parity, NOT a corpus back-solve: the API mapper
feeds the SAME worksheet-validated §6.1 cascade (`conservatory_geometry`,
pinned to 1e-4 against the case-44 Summary) as the Elmhurst path — so the
API conservatory fabric is correct by construction. `from_api_response`
on an injected type-4 cert reproduces the glazed wall (perimeter × ground-
floor room height = 22.05), glazed roof (floor/cos20 = 12.77) and Table 25
double U_eff (2.758 wall / 2.993 roof); a separated (type 2/3) cert lodges
no glazed BP → disregarded per §6.2.

Gauges: corpus within-0.5 67.9% → 68.6% (MAE 0.959 → 0.942; floor 0.67→0.68,
ceiling 0.97→0.95); /tmp eval mean|err| 0.822 → 0.817. Harness 47/47
0-raised; regression = the 3 pre-existing fails; pyright net-zero (65=65).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 23:37:25 +00:00