Commit graph

645 commits

Author SHA1 Message Date
Khalim Conn-Kowlessar
dc55d3b899 Price an off-peak meter line day/night by its high-rate fraction 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 17:24:50 +00:00
Khalim Conn-Kowlessar
1acfc08fce Load the off-peak day/night rate from the committed snapshot 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 17:20:19 +00:00
Khalim Conn-Kowlessar
bda8a3f276 Expose an off-peak meter's standing charge and full day/night blend range 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 17:14:51 +00:00
Khalim Conn-Kowlessar
4641717c90 Blend an off-peak meter's day/night rate by high-rate fraction 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 16:55:10 +00:00
Daniel Roth
aa1a834280
Merge pull request #1303 from Hestia-Homes/fix/modelling-e2e-not-modellable-errors
Fail unmodellable properties with a specific, debuggable error
2026-06-24 17:20:25 +01:00
KhalimCK
422c943748
Merge pull request #1305 from Hestia-Homes/feature/e2e-runs
Feature/e2e runs
2026-06-24 16:25:22 +01:00
Jun-te Kim
04ee16488e Fail unmodellable properties with a specific, debuggable error 🟩
_predict_epc returned None for three unrelated causes — unresolved
property_type, an empty same-type cohort, and a degenerate (no MAIN part)
prediction — which the handler collapsed into one generic "not predictable"
string. The SubTask output could not say which cause fired or which data to
fix.

Raise a specific PropertyNotModellableError subclass per cause, each carrying
the property's identity (property_id, uprn, postcode, portfolio_id) and
cause-specific context. The unresolved-property-type message points at the
likely missing/contradictory Landlord Override. All subclass ValueError, so the
per-property failure boundary keeps catching them and records str(exc).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 14:13:28 +00:00
Jun-te Kim
dd06b19c77 Fail unmodellable properties with a specific, debuggable error 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 14:10:55 +00:00
Khalim Conn-Kowlessar
f3a164c371 Guard EpcPropertyData round-trip field coverage 🟩
Fail if any EpcPropertyData field is neither reconstructed by _compose nor on a
documented allow-list, turning latent persistence gaps into explicit decisions
(would have caught the conservatory and roof-window drops). ADR-0036.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 13:56:05 +00:00
Khalim Conn-Kowlessar
03a0d9c1ca Round-trip the non-separated conservatory through persistence 🟩
Persist SapConservatory as five nullable conservatory_* columns on epc_property
(1:1 with the dwelling) and rebuild it in _compose, so the §6.1 fold survives
save -> reload -> score. Without this the scored (re-hydrated) EPC silently
dropped the conservatory (persist != score) — a latent gap shared with the
21.0.1 path. Adds a deep-equality round-trip test. ADR-0036.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 13:56:05 +00:00
Daniel Roth
f6e5e11d88
Merge pull request #1300 from Hestia-Homes/feature/notnullviolation-optional-fields-21-0-1
fix: coalesce omitted 21.0.1 optional count fields to 0 to prevent NotNullViolation on persist
2026-06-24 14:36:04 +01:00
Daniel Roth
10c975b1ae Fix test asserting None for cfl_fixed_lighting_bulbs_count on 21.0.1 cert 🟩
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 13:31:02 +00:00
Jun-te Kim
e2915d8042 Prefer a MAIN-bearing prediction template so EPC-less dwellings predict 🟩
A predicted EPC is seeded by deep-copying one representative neighbour's
structure. _template chose the member whose floor area was closest to the
cohort median, ignoring building-part labels. When that member's only part
was lodged with a null identifier (mapped to OTHER), the prediction had no
MAIN part and the modelling_e2e handler rejected it as "not predictable" —
discarding an otherwise-rich same-type cohort.

Restrict the template to MAIN-bearing members (median still over the whole
cohort); fall back to closest-on-size only when none are MAIN-bearing, so an
all-unlabelled cohort is left for the handler's MAIN-part guard to reject
rather than silently relabelling real data.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 13:20:43 +00:00
Jun-te Kim
abb822c1c7 Prefer a MAIN-bearing prediction template so EPC-less dwellings predict 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 13:16:54 +00:00
Jun-te Kim
fc7cfc70e8
Merge pull request #1295 from Hestia-Homes/fix/fuel-code-39-electricity
Map fuel code 39 (electricity, any tariff) to standard electricity
2026-06-24 12:37:25 +01:00
Daniel Roth
ecd9a5c622
Merge pull request #1296 from Hestia-Homes/feature/e2e-subtasks
Separate subtask for each property ID when e2e modelling is triggered
2026-06-24 12:36:12 +01:00
Jun-te Kim
782c686969 Map fuel code 39 (electricity, any tariff) to standard electricity
modelling_e2e properties with main fuel 39 failed at the price boundary
(UnpricedFuelCode since #44fff767; previously mis-rated as non-electric →
the ~14-SAP over-rating flagged in earlier review).

Code 39 is SAP Table 12 "electricity, any tariff" (epc_codes.csv main_fuel 39 =
"electricity, unspecified tariff"; spec footnote (j): defines an electric system,
cost/CO2/PE = standard electricity). It was absent from API_FUEL_TO_TABLE_32, so
to_table_32_code(39) was None → is_electric_fuel_code(39) False and pricing
raised.

Fix: map API_FUEL_TO_TABLE_32[39] = 30 (standard electricity) — the canonical
place Khalim's fuel work added codes. One line makes classification, pricing,
CO2/PE and the billing carrier all agree (39 → 30 → ELECTRICITY).

Tests: to_table_32_code(39)==30, is_electric_fuel_code(39) True, price == standard
electricity, and the billing carrier resolves to ELECTRICITY. 0 corpus impact
(no lodged corpus cert uses 39); accuracy + mapper-corpus gates green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 11:20:32 +00:00
Daniel Roth
e1bc5360f2 skipped cohort certs are logged without failing the coordinator SubTask 🟩
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 11:11:19 +00:00
Daniel Roth
fc2732550c skipped cohort certs are logged without failing the coordinator SubTask 🟥
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 11:09:30 +00:00
Khalim Conn-Kowlessar
f6ec96fbcd Merge remote-tracking branch 'origin/main' into feature/e2e-runs
# Conflicts:
#	domain/sap10_calculator/tables/table_12a.py
2026-06-24 11:07:42 +00:00
Daniel Roth
c2a84d3ee8 per-property failure fails child SubTask without raising from handler 🟩
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 11:07:34 +00:00
Daniel Roth
39f028f03a per-property failure fails child SubTask without raising from handler 🟥
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 11:04:29 +00:00
KhalimCK
f7f74ea72b
Merge pull request #1288 from Hestia-Homes/feature/per-cert-mapper-validation
Feature/per cert mapper validation
2026-06-24 12:04:00 +01:00
Jun-te Kim
46fc8f338c
Merge pull request #1289 from Hestia-Homes/fix/solar-missing-max-array-panels
Treat solar block without array sizing as no-solar (fix KeyError maxArrayPanelsCount, 2 e2e failures)
2026-06-24 11:59:57 +01:00
Jun-te Kim
e9a8ad9cd9
Merge pull request #1290 from Hestia-Homes/fix/building-part-int-identifier
Tolerate non-string building-part identifier (fix TypeError, cohort cert)
2026-06-24 11:59:31 +01:00
Daniel Roth
2d2abc016b handler creates one child SubTask per property ID in the batch 🟥
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 10:56:13 +00:00
Jun-te Kim
41ebf01969
Merge pull request #1287 from Hestia-Homes/fix/dual-fuel-billing-carrier
Add dual-fuel (mineral+wood) billing carrier (fix UnmappedSapCode fuel_code 10, 10 e2e failures)
2026-06-24 11:49:18 +01:00
Daniel Roth
3a66819a86 task_handler passes orchestrator and task_id to wrapped function when flag is true 🟥 2026-06-24 10:47:05 +00:00
Khalim Conn-Kowlessar
db3bf00602 fix(synthesis): Coherent Heating System — synthesisers own the whole boundary
A dwelling's heating is one conceptual system, but its fields are scattered
across EpcPropertyData (a gov-API schema mirror): the cluster on sap_heating, the
electricity tariff on sap_energy_source.meter_type, hot-water flags loose at top
level. Three places synthesise a heating system — Measure Options, Landlord
Overrides, EPC Prediction's donor — and each hand-copied a different ad-hoc
subset. The override and donor both dropped meter_type, so an electric-storage
system landed on the template's single-rate meter and billed overnight heat at
the peak rate: property 713406 scored SAP 13 (G) vs ~50 (E), inflating the HHRSH
measure to +45.8 and overshooting the plan to band A.

Establish a single Coherent Heating System boundary (CONTEXT.md) that every
synthesiser must cover, with a source-appropriate fill policy (ADR-0035):

- Override overlay *completes* the partial system the landlord named. Companion
  fields are now DERIVED from the SAP code, not hand-attached per archetype: the
  off-peak meter from the calculator's single off-peak classification (new
  OFF_PEAK_IMPLYING_HEATING_CODES = SAP §12 Rules 1-2), and an unobserved storage
  charge control defaults to the conservative manual control (Table 4e 2401). So
  adding a heating archetype is just adding its code — companions can't be
  forgotten. A contract test guards it (every off-peak code drags a Dual meter).
- Prediction's heating donor now *carries* the donor's meter_type alongside its
  sap_heating cluster — the donor is already coherent.

Coherence is a synthesis-time obligation only; the calculator still scores a real
lodged cert exactly as lodged.

Verified on 713406: baseline 13 -> 47.8 (E), matching its recorded rating; the
phantom HHRSH recommendation is gone and the plan no longer overshoots to A.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 10:46:11 +00:00
Jun-te Kim
6d98ecf375 Tolerate non-string building-part identifier (fix TypeError)
A SAP-16.x cohort cert (9258-4062-7265-2844-7954) lodges a bare int building-
part identifier (the second part as `1` after "Main Dwelling").
`BuildingPartIdentifier.from_api_string` regex-matched it assuming a string and
raised "TypeError: expected string or bytes-like object, got 'int'", failing the
whole property.

Fix: guard the match on `isinstance(api_identifier, str)` so a non-string
identifier falls to OTHER, matching the documented "anything unrecognised ->
OTHER" contract. The baseline SAP fabric sums all building parts regardless of
identifier, so OTHER is SAP-neutral; the identifier only labels parts for
measure targeting. Fixes every mapper (all route through from_api_string).

Cert now maps + calculates (sap 63). Regression test added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 10:33:23 +00:00
Jun-te Kim
eecea8301e
Merge pull request #1286 from Hestia-Homes/fix/predicted-baseline-rhi-fallback
Fall back to computed kWh for predicted-EPC baselines (fix 16 e2e failures)
2026-06-24 11:26:33 +01:00
Jun-te Kim
79f89d872e Treat solar block without array sizing as no-solar (fix KeyError)
2 modelling_e2e properties failed with KeyError: 'maxArrayPanelsCount'.

Google returns a `solarPotential` block with no array-level sizing fields
(`maxArrayPanelsCount` / `panelCapacityWatts`) for buildings with no usable
solar estimate. `SolarPotential.from_building_insights` hard-indexed those keys
and crashed the whole property.

Fix: the projection now returns Optional and yields None when those fields are
absent — the established "no solar potential" outcome (the orchestrator and
recommendation path already type it Optional and skip solar on None). Existing
callers (`_solar_potential_for`, harness) already assign to Optional.

Regression test + `assert is not None` narrowing on the valid-fixture tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 10:22:06 +00:00
Jun-te Kim
d97b8e87a4 Add dual-fuel (mineral+wood) billing carrier (fix UnmappedSapCode 10)
10 modelling_e2e properties failed with "unmapped SAP code in fuel_code: 10":
the billing layer (`sap_code_to_fuel`) had no carrier for Table-32 code 10
(dual fuel, mineral + wood) and raised rather than guess one.

SAP 10.2 treats dual fuel as its OWN fuel (its own Table-12 factors), so model
it as its own billing carrier rather than collapsing onto wood or coal:

- New `Fuel.DUAL_FUEL_MINERAL_AND_WOOD`.
- `_CODE_TO_FUEL[10]` -> that carrier.
- Fuel Rates snapshot prices it at 7.69 p/kWh — the midpoint of the COAL proxy
  (7.13) and WOOD_LOGS (8.25). This mirrors SAP's own construction: Table-32
  dual fuel (3.99) ~= midpoint of house coal (3.67) and wood logs (4.23).
  Marked `derived` with a documented _note/_gap/_assumption (like the COAL and
  HEAT_NETWORK proxies), since there is no retail blend price.

A dedicated carrier + rate (vs a one-line map to an existing carrier) keeps the
fuel identity faithful to SAP and avoids mispricing dual fuel as pure wood/coal.

Tests: code 10 -> DUAL_FUEL carrier; snapshot prices it at 7.69; grid-export
codes (36/60) still raise (the genuine no-carrier case).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 10:07:57 +00:00
Khalim Conn-Kowlessar
80985865a2 Revert "guard(modelling_e2e): quarantine predicted Properties the calculator mis-scores"
This reverts commit 0c70280dea.
2026-06-24 09:33:54 +00:00
Khalim Conn-Kowlessar
0c70280dea guard(modelling_e2e): quarantine predicted Properties the calculator mis-scores
TEMPORARY guard (remove once the SAP calculator's oil-heating under-score is
fixed): a predicted oil-boiler picture scores SAP 13/G against its own
synthesised recorded SAP of 50/E, so the optimiser overshoots goal C all the
way to band A and publishes nonsense.

A predicted EpcPropertyData carries its recorded SAP (energy_rating_current).
When the calculator baseline diverges from it by more than ~one band (20 SAP
points), withhold the Plan: raise inside the per-property loop so the existing
failure isolation drops just that property into `failures` and fails the
subtask, while every other property still models and persists. Lodged
Properties are untouched — they have a real recorded cert and the Rebaseliner
already owns this check.

Verified end-to-end against property 713406 (UPRN 100061849247): baseline 13.2
vs recorded 50 -> quarantined, no Plan written.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 09:07:24 +00:00
Khalim Conn-Kowlessar
4e2f2bdcc7 test(worksheet): pin simulated case 50 — MVHR + dual-immersion all-electric
Adds the mapper-driven e2e cascade pin for "simulated case 50" (000565 semi,
electric storage main SAP 402 + portable electric secondary + MVHR + whc-903
DUAL electric immersion + 160 L cylinder, Economy-7). Routes the Summary PDF
through extractor + mapper + calculator like the other 000565 fixtures.

Locks in two off-peak fixes this case ground-truthed:
- the Table 13 HW high/low split applied to CO2/PE (commit 39ae2cf0), and
- the Table 12a Grid 2 MVHR fan fraction 0.71/0.58 (commit cd5113ab).

All 11 SAP-result fields reconcile to the U985 worksheet EXACTLY, including
the (272) rating CO2 2397.1237 — SAP 38.8426 (=39), cost £1317.0116, water
1668.0788 kWh, fans 315.6384 kWh.

Summary mirrored to the tracked fixtures dir so the test doesn't depend on
the unstaged `sap worksheets/` workspace.

pyright strict gate not run locally (pyright not installed in this container).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 09:07:16 +00:00
Khalim Conn-Kowlessar
39ae2cf0c2 fix(water-heating): split whc-903 immersion HW CO2/PE on off-peak tariffs
SAP 10.2 Table 12d/12e: electric water heating on a 7-/10-hour tariff bills
CO2/PE at the high-rate code (32/34) and low-rate code (31/33), kWh-weighted
by the Table 13 high-rate fraction. The cost path already applied this split;
the CO2/PE factors did not — they used the flat annual Table 12 figure
(0.136 CO2 / 1.501 PE) for ALL dual-rate electric HW.

That flat-annual behaviour (slice S0380.163) was validated only against
HW-from-main "low-rate cost" certs (100% low, no high-rate split). It is NOT
how Elmhurst bills a whc-903 ELECTRIC IMMERSION: the hand-built case-50
worksheet (000565 + dual immersion, 7-hour) splits HW CO2/PE into "high rate
cost" (CO2 0.1475 / PE 1.5514) + "low rate cost" (CO2 0.1238 / PE 1.4429)
weighted by the Table 13 fraction 0.1009. So flat-0.136 for immersion HW was
a spec gap on our side, not an Elmhurst divergence.

Fix: `_electric_immersion_hw_high_rate_fraction` threads the Table 13 fraction
(scoped to whc-903, 7-/10-hour, cylinder data present) into the HW CO2 + PE
factor helpers, which then blend the Table 12d/12e high/low codes. The flat
rule is unchanged for HW-from-main and 18-/24-hour (no Table 12d split), so
the S0380.163 41-variant cases and the existing pin are untouched.

Case 50: rating CO2 2413.48 -> 2397.1237 = Elmhurst EXACT; demand CO2 2007.1384
EXACT; demand PE +111 -> +32.5 residual (within corpus PE noise). Corpus
unchanged 73.3% / MAE 0.774 / CO2 0.08 / PE 3.4 (62 whc-903 off-peak certs;
aggregate gauges hold). SAP unaffected (cost-based).

Pin: test_whc903_immersion_hw_co2_pe_factors_split_high_low_on_off_peak; doc
updated in SAP_CALCULATOR.md §8.1.
pyright strict gate not run locally (pyright not installed in this container).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 09:05:21 +00:00
Jun-te Kim
6679d8c621 Fall back to computed kWh for predicted-EPC baselines (no lodged RHI)
16 modelling_e2e properties failed with "Effective EPC is missing
renewable_heat_incentive; cannot read baseline space-heating / hot-water kWh".

Baseline runs for predicted properties too (ADR-0031), reading space/water-
heating kWh off the EPC's lodged RHI block. Predicted EPCs deep-copy a neighbour
template that may carry no RHI, so `_require_rhi` hard-failed the whole subtask.

Fix: when the EPC has no RHI, fall back to the property's OWN computed figures
from the scored SapResult (space_heating_kwh_per_yr / hot_water_kwh_per_yr) —
more representative than a neighbour's lodged numbers. Only when there is also no
SapResult (the rebaseliner scored nothing) is there genuinely no demand to
record, and we still fail noisily. Lodged certs are unchanged (RHI still wins).

Regression tests: fallback-to-computed, and the no-RHI/no-result raise.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 08:58:24 +00:00
Khalim Conn-Kowlessar
f3e3494bf7 test(worksheet): pin simulated case 52 — regular gas boiler + cylinder
Adds the mapper-driven e2e cascade pin for "simulated case 52" (000565
semi + regular non-combi mains-gas boiler SAP 102 + 160 L foam cylinder
heated from the main, no cylinder stat, uninsulated primary pipework,
standard tariff). Routes the Summary PDF through extractor + mapper +
calculator like the other 000565 / 001431_case* fixtures.

This closes the last untested branch of the cylinder/water chain: the
SAP 10.2 §4 cylinder storage loss (Table 2/2a/2b lines 51-55) + the
Table 3 PRIMARY circuit loss (59, uninsulated pipework + no stat) that
combi/immersion fixtures don't reach. All 11 SAP-result fields reconcile
to the U985 worksheet EXACTLY with no calculator change — SAP 57.2904
(=57), cost £911.1973, water 3929.7635 kWh — confirming the cylinder-loss
derivation is correct.

Summary mirrored to the tracked fixtures dir so the test doesn't depend
on the unstaged `sap worksheets/` workspace.

pyright strict gate not run locally (pyright not installed in this container).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 08:50:19 +00:00
Khalim Conn-Kowlessar
01bc93ed33 Merge remote-tracking branch 'origin/main' into feature/e2e-runs 2026-06-24 08:39:31 +00:00
Khalim Conn-Kowlessar
7a970ea393 fix(repositories): fold landlord overrides when re-hydrating a Property via the UoW
PostgresUnitOfWork built its PropertyPostgresRepository without an overrides
reader, so a Property re-hydrated through the unit silently dropped its
Landlord Overrides (ADR-0032). The Baseline orchestrator runs through the UoW,
so it scored the bare lodged EPC while the Plan modelled the override-folded
Effective EPC — the two diverged (e.g. baseline effective 71/C vs plan
baseline 62/D), producing "already at band C yet recommends reaching C".

Wire PropertyOverridesPostgresReader into the unit's property repo (uow-
independent committed reference data, read via the same session factory) so
every re-hydration folds overrides, matching the live modelling path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 08:34:30 +00:00
Khalim Conn-Kowlessar
80d617aa5b modelling: glazing overlay models draught-proofing + frame-factor re-lodge
Fitting sealed glazing units changes two things beyond the pane's U/g
that the cascade reads, which the overlay didn't model — leaving the
double/secondary before→after pins ~0.7 SAP short (xfail):

1. Draught-proofing (RdSAP 10 §8.1). Sealed units draught-proof the panes
   they replace, re-lodging the dwelling-level `percent_draughtproofed`
   (cert 001431: 84 → 100). The §2 cascade reads that dwelling-level
   value, so the overlay now carries it. `_recompute_percent_draughtproofed`
   anchors on the lodged before-% — `after = round((round(before%/100 × N)
   + flips) / N × 100)`, N = openable windows (vertical + roof) + doors,
   flips = upgraded panes that were not draught-proofed — so it's robust
   to incomplete window extraction (unchanged openings are already in the
   aggregate). ~0.3 SAP.

2. Frame factor (§6 solar gains). A replacement unit re-lodges its own
   FF=0.70, overriding the pane it replaced — the two "single glazing,
   known data" panes lodge FF 1.00 / 0.50 (one is 6.6 m²), so leaving them
   unchanged understated solar gains by ~+150 kWh space heating. `WindowOverlay`
   now carries `frame_factor`, written flat onto the window. ~0.4 SAP.

Wiring: `EpcSimulation.percent_draughtproofed` + `WindowOverlay.frame_factor`
new fields; `apply_simulations` / `_fold_window` write them; the glazing
generator computes both from the upgraded set and cert 001431's after.

Un-xfails `test_{double,secondary}_glazing_overlay_reproduces_the_relodged_after`
— both now pin SAP/CO2/PE to the relodged after within tolerance. Updates
the two `test_glazing_recommendation` overlay expectations for the new
`frame_factor`. 96 modelling tests pass; zero new pyright errors.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 08:25:08 +00:00
Jun-te Kim
22cb47a280
Merge pull request #1285 from Hestia-Homes/feature/e2e-runs
fix(modelling_e2e): persist predicted EPC + baseline for predicted pr…
2026-06-24 09:08:06 +01:00
Khalim Conn-Kowlessar
cd5113abf2 fix(tariff): include MVHR fan electricity in the off-peak Grid 2 fan split
SAP 10.2 Table 12a Grid 2 (PDF p.191) bills "Fans for mechanical ventilation
systems" at 0.71 (7-hour) / 0.58 (10-hour), distinct from "All other uses"
(0.90 / 0.80) which covers circulation pumps, flue fans and the solar HW pump.

The cost-split mech-vent kWh (`mev_kwh_for_cost_split`) only summed the
decentralised-MEV (230b) fans, not the (230a) MVHR fan electricity — even
though the total pumps/fans bucket adds both. So an MVHR dwelling on an
off-peak tariff billed its fan electricity at the 0.90/0.80 "all other uses"
rate instead of 0.71/0.58. The comment already said "MEV/MVHR-fan portion";
only the MEV term was wired when MVHR landed. Fixed to mirror both
mechanical-ventilation fan terms summed into the total.

Worksheet-proven on simulated case 50 (000565 semi + MVHR Vent Axia + dual
electric immersion, Unknown meter -> 7-hour via the §12 dual-immersion
trigger): the fan bucket (315.64 kWh, 100% MVHR per worksheet line 230a) was
billing at 14.311 p/kWh (0.90) vs Elmhurst's 12.451 p/kWh (0.71) — +£5.87/yr,
-0.23 SAP. After the fix our existing-dwelling rating reconciles to Elmhurst
EXACTLY: SAP value 38.8426 (=Elmhurst 38.8426 -> 39), total cost £1317.0116
(=Elmhurst £1317.0116 to the penny).

Same `mev_kwh_for_cost_split` feeds the CO2 + PE cascades, so all three split
consistently. 0 corpus impact (all 3 corpus MVHR certs are standard tariff);
gauge unchanged 73.3% / MAE 0.774 / CO2 0.08 / PE 3.4.

Pin: test_mvhr_fan_electricity_bills_at_grid2_fan_fraction_on_off_peak.
pyright strict gate not run locally (pyright not installed in this container).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 22:16:58 +00:00
Khalim Conn-Kowlessar
eea5d3a5a8 fix(tariff): map electric boilers 191/193-196 to their Table 12a Grid 1 rows
SAP 10.2 Table 4a electric boilers (PDF p.170) split across three distinct
Table 12a Grid 1 SH rows (PDF p.191), not one "direct-acting" family as the
stale TODO in `_table_12a_system_for_main` implied:

  - 191 Direct-acting electric boiler   -> "Direct-acting electric boiler (a)"
    row: 7-hour 0.90, 10-hour 0.50 (NOT the 1.00/0.50 "Other direct-acting
    electric heating" room-heater row).
  - 193/194/195/196 Electric dry core / water storage boiler -> "Electric dry
    core or water storage boiler" row: 7-hour 0.00 (charged wholly off-peak =
    100% low rate, identical to the None fallback).
  - 192 Electric CPSU -> Appendix F; left falling through to None (off-peak
    low) until the Appendix-F high-rate cascade is implemented.

The enum + fractions already existed in table_12a.py; only the code->enum
mapping was missing. Resolves the TODO and pins the spec-correct 0.00 for the
storage boilers so 195 can't be mis-"fixed" up to a direct-acting fraction.

Forward guard, 0 corpus impact: storage boilers already billed 100% low via
the None fallback, and all corpus 191 certs are on standard tariff (Table 12a
off-peak split never fires). Corpus gauge unchanged 73.3% / MAE 0.774.

Pin: test_electric_boilers_191_195_map_to_distinct_table_12a_grid1_rows.
pyright strict gate not run locally (pyright not installed in this container).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 21:48:59 +00:00
Khalim Conn-Kowlessar
9694650abe fix(water-heating): derive combi keep-hot from the PCDB record, default no-keep-hot
SAP 10.2 Table 3a (PDF p.160) additional combi loss (61)m. Two coupled
defects, both surfaced by simulated case 49 (000565 + gas combi, U985
"Combi keep hot type = None") sitting at SAP 71.43 vs the worksheet's 72:

1. The cascade defaulted EVERY non-PCDB combi to the flat keep-hot
   time-clock row (600 × n/365). A combi WITHOUT a keep-hot facility uses
   row 1 (600 × fu × n/365, fu = V_d/100 when daily HW < 100 L/day) —
   over-counting (61)m for the no-keep-hot cohort. `water_heating_from_
   cert` now defaults to the "without keep-hot" row.

2. `pcdb_combi_loss_override` returned None for keep_hot_facility=1/
   timer=1, leaning on the OLD flat-600 default. So flipping the default
   silently turned 190 corpus PCDB keep-hot-time-clock combis into
   no-keep-hot. Fixed to return the flat keep-hot row EXPLICITLY.

Key insight (the Summary is the input echo; the U985 keep-hot line is a
computed OUTPUT, so it must be derivable): keep-hot rides on the PCDB
boiler record (Table 105 keep_hot_facility/timer), resolved by
`pcdb_combi_loss_override`. A generic SAP-code combi with no PCDB record
(case 49, PCDF ref 0) has no keep-hot by construction → row 1. So the
default is not a guess — it is the spec-correct value for non-PCDB combis.

Worksheet-proven: case 49 → cost £726.696, SAP 72 — matching the
accredited worksheet to the digit (continuous 71.6945 = the worksheet's
own 71.6945). 000516 (keep-hot None) also exact (£860.716, SAP 63);
000490 (PCDB 10328, keep_hot_facility=1/timer=1) keeps its flat-600 via
the PCDB path. Masked until now because every prior combi-loss worksheet
fixture was keep-hot (000490/000474/000480 time-clock) or had V_d >= 100
every month (001431, rows coincide); case 49 is the first no-keep-hot one.

Corpus within-0.5 72.7% -> 73.3%, MAE 0.781 -> 0.774, PE 3.5 -> 3.4;
ratcheted _MAX_SAP_MAE 0.785 -> 0.775, _MAX_PE_PER_M2_MAE 3.6 -> 3.5.

Note: pyright strict type gate not run locally (pyright not installed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 21:22:02 +00:00
Khalim Conn-Kowlessar
b7455aabe5 fix(ventilation): MVHR takes lodged 0 intermittent fans, not the Table 5 default
Extends the dMEV intermittent-fan fix (4db05e84) to MVHR. A balanced
whole-house MVHR system IS the dwelling's ventilation, so the lodged (7a)
intermittent-extract-fan count is explicit — a lodged 0 means 0, not the
RdSAP 10 Table 5 age-band "unknown" default. The cascade was substituting
the default (here 20 m³/h) into worksheet line (8) openings, inflating
(16/18) infiltration → (21) → (22b) → (25) effective ach → (38)
ventilation heat loss → the space-heating demand.

Worksheet-proven on simulated case 49 (000565 + Vent Axia 500140 MVHR,
lodged (7a)=0): our (8) openings 0.0723 -> 0.0000, (18) 0.7223 -> 0.6500,
(25)m Jan 0.9423 -> 0.8571, all now matching Elmhurst exactly; space-
heating demand 7857 -> 7528 kWh (worksheet 7546). SAP 70.90 -> 71.43
continuous. (The residual to the worksheet's 72 is its own continuous SAP
71.69 rounding up, driven by a separate gas-combi water-heating-loss gap,
not ventilation.)

Scoped to EXTRACT_OR_PIV_OUTSIDE + MVHR only — MV-without-HR
(mechanical_ventilation=1) stays on the default-substitution path
(forcing its lodged 0 regressed 47 Howsman / 18 Jutland and is not
worksheet-validated). Corpus within-0.5 holds 72.7%, MAE 0.782 -> 0.781.

Note: pyright strict type gate not run locally (pyright not installed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 20:15:32 +00:00
Khalim Conn-Kowlessar
7b30b464e5 feat(ventilation): credit MVHR (24a) heat recovery via PCDB Table 323 + 329
MVHR (24a) heat-recovery support, part 2: the mapper + cascade wiring.

Both source paths now resolve balanced whole-house MV with heat recovery
to the MVHR kind:
- gov-API: `_API_MECHANICAL_VENTILATION_TO_KIND` 4 → "MVHR" (was None /
  treated as natural — under-stated ventilation heat loss, over-rating).
- Elmhurst Summary: `_ELMHURST_MV_TYPE_TO_KIND` "Mechanical ventilation
  with heat recovery (MVHR)" → "MVHR" (was UnmappedElmhurstLabel, which
  blocked the whole Summary for MVHR dwellings).

cert_to_inputs resolves the in-use heat-recovery efficiency + SFP for an
MVHR cert (`_mvhr_system_values`): pick the PCDB Table 323 data point by
the lodged wet-room count (SAP 10.2 §2.6.4), multiply the raw efficiency
by the Table 329 ducts-inside-envelope in-use factor (0.90) and the raw
SFP by the per-duct-type factor (rigid 1.4), and feed:
- the §2.6.6 eq (2) effective-air-change credit (23c) → (24a)/(25)m;
- the (230a) fan electricity (in-use SFP × 1.22 × V), costed but NOT
  added to the Table 5a gains (its effect is in the efficiency).
An MVHR lodged with no PCDF index falls back to the SAP 10.2 Table 4g
default (raw efficiency 66% × 0.70, raw SFP 2.0 × 2.5).

Worksheet-proven on simulated case 49 (000565 semi + Vent Axia Sentinel
Kinetic B 500140 + gas combi → Elmhurst Current SAP 72): every MVHR line
matches Elmhurst exactly — (33) fabric heat loss 100.5923, (23c) in-use
efficiency 81.9% = 91 × 0.90, (25)m Jan 0.8571, (230a) fan electricity
415.9325, (231) total pumps/fans 501.9325. The residual SAP 71 vs 72 is
the known 000565-family space-heating-demand artifact (same -1/-2 seen on
cases 47/48), not the MVHR logic.

Corpus: within-0.5 72.6% -> 72.7%, MAE 0.788 -> 0.782, PE 3.6 -> 3.5.
The 3 gov-API MVHR certs: Flat 1 +6 -> 0 (Table 4g default path) and
12a Princes Gate +3 -> +1 (heat-recovery credit); Apartment 707 -4 -> -6
is a separate baseline under-rate (it under-rated as natural too — the
MVHR credit correctly adds ventilation loss per Elmhurst's method).
Ratcheted _MAX_SAP_MAE 0.79 -> 0.785, _MAX_PE_PER_M2_MAE 3.7 -> 3.6.

Note: pyright strict type gate not run locally (pyright not installed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 19:45:37 +00:00
Khalim Conn-Kowlessar
34cbd7d66c feat(pcdb): parse Table 323 (Centralised MEV / MVHR) + Table 329 efficiency IUF
MVHR (24a) heat-recovery support, part 1: the PCDB data layer.

PCDB Table 323 (PCDF Spec Rev 6b §A.18, Format 426; pcdb10.dat carries
Format 431, header `$323,431,...`) holds the per-wet-room SFP + heat-
exchanger efficiency for centralised MEV / MVHR units. Added
`MvhrRecord` / `MvhrDataPoint`, `parse_centralised_mv_row` /
`parse_table_323`, the ETL step, the committed jsonl, and the
`mvhr_record(pcdb_id)` runtime lookup (mirrors Table 322).

SAP 10.2 §2.6.4/§2.6.6: "MVHR ... SFP is a single value depending on the
number of wet rooms" — each test group's leading field is the wet-room
count; callers select the group matching the dwelling lodgement.
Worksheet-proven on simulated case 49 (000565, 2 wet rooms, Vent Axia
Sentinel Kinetic B 500140 → flow 21.0, SFP 0.88, efficiency 91%).

Also decoded the MVHR heat-recovery efficiency in-use factor from Table
329 (Format 432): system_type 3 ducts-inside-envelope = 0.90 (case-49
(23c) = 91 × 0.90 = 81.9%), cross-checked against system_type 10 = 0.70
(= SAP 10.2 Table 4g default heat-recovery in-use factor). "Table 4h is
no longer used – data now stored in the PCDB" (SAP 10.2 p.176).

The outside-envelope efficiency columns + with-scheme SFP blocks are
preserved verbatim in `raw` (no fixture exercises them yet).

Note: pyright strict type gate not run locally (pyright not installed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 19:29:17 +00:00
Jun-te Kim
af5b2b5f80 Mark unmappable cohort certs as a subtask failure
Skipped cohort certs were previously surfaced only as outputs.result on a
completed subtask, so they were easy to miss. Treat them as a failure too:
once the batch has run to completion (so every modellable property is already
written to DB), raise if there were any per-property errors OR any skipped
certs. The run gets flagged for debugging without discarding the work done.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 18:03:04 +00:00