Chasing the space-heating demand gap on "simulated case 48" (main 691 + Unknown
meter + 903 dual immersion): our SAP 55 vs Elmhurst 57. Every §10a cost line
already matched to the penny; the residual was demand — our space-heating
energy 3849.8 kWh vs Elmhurst 3513.8 (+9.6%). Traced through the worksheet: our
ventilation heat loss (38) ran ~35.5 W/K vs Elmhurst 27.76 — we were adding 20
m3/h of intermittent extract fans (the Table 5 age-band default) on a dwelling
with a decentralised mechanical extract (dMEV) system that lodges 0 fans.
SAP 10.2 §2 (PDF p.13): a whole-house mechanical EXTRACT system provides
extraction via the (23a) 0.5 system air-change rate; the lodged intermittent
extract-fan count (7a) is then explicit — a lodged 0 means 0 (the dMEV is the
ventilation), NOT "unknown". The Table 5 default is an unknown-fallback for
NATURALLY ventilated dwellings only, so it must not be substituted here.
Fix: for EXTRACT_OR_PIV_OUTSIDE, take vc.intermittent_fans as-is (no age-band
default). Worksheet-proven on two dMEV builds of cert 000565: "case 48" lodges
(7a)=0 -> our SAP 55 -> 57 EXACT; the original 000565 fixture lodges (7a)=2 and
keeps 2 (its e2e pins are unchanged). An earlier draft that forced fans=0 broke
000565 (which legitimately has 2) — corrected to "lodged as-is".
within-0.5 72.5% -> 72.6%, MAE 0.789 -> 0.788; CO2/PE unchanged. The fix also
reduces a systematic under-rating bias in the 21-cert dMEV cohort (median dSAP
-0.22 -> -0.08). Scoped to EXTRACT_OR_PIV_OUTSIDE; balanced MVHR/MV kinds left
untouched pending their own worksheet. SAP-schema regression
test_18_0_0 pin 80 -> 81 (closer to its lodged 84, same cause). Spec-pinned in
test_cert_to_inputs (dMEV-lodged-0 vs natural-default). pyright not installed
in this container -- strict type gate not run locally.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Closes the follow-up from 22fe4f41. RdSAP 10 §12's third Unknown-meter
exception bullet — "main heating is ground source or water source heat pump" —
was unimplemented. Add `_GROUND_OR_WATER_SOURCE_HEAT_PUMP_CODES` (Table 4a,
SAP 10.2 PDF p.176-177: ground 211/215/221/225 + warm-air 521/525; water
213/216/223/226 + warm-air 523/526) to the Unknown-meter off-peak triggers;
once dual, Rules 1-4 (Rule 3) resolve it to 10-hour.
AIR-source heat pumps (214/217/224/227, 524/527) are deliberately EXCLUDED —
the spec names only ground/water source. Verified the only Unknown-meter heat
pumps in the corpus are "3/10 Bedford House" (main 214 = AIR source), which
correctly KEEP STANDARD. 0 corpus certs carry a GSHP/WSHP on an Unknown meter,
so this is a spec-completeness forward guard (gauge unchanged 72.5% / 0.789),
same family as the Scotland-J wall / rafters-M roof 0-impact spec fixes.
Coverage gap noted in-code: a database-index heat pump without a 211/213-style
SAP code can't have its source type read from the code alone (rare).
Spec-pinned (test_unknown_meter_ground_or_water_source_heat_pump_triggers_off_
peak). Gates green: corpus 72.5%/0.789, batch worksheet 0 raised/0 diverge,
suite 2989 passed (2 known pre-existing fails). pyright not installed locally.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
MAE improved 0.793 -> 0.789 via the Unknown-meter + dual-electric-immersion
off-peak trigger (commit 22fe4f41). Ratchet the ceiling so the gain can't
silently regress. within-0.5 unchanged (72.5%).
pyright not installed in this container — strict type gate not run locally.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Supersedes the previous "verified non-fix" doc (3548f1f3): the spec DOES make
this a fix — Khalim was right that the Unknown-meter branch is driven by the
heating/water system, not a blanket STANDARD.
RdSAP 10 §12 (PDF p.62): "If the electricity meter is unknown, treat as single
meter EXCEPT where main heating OR WATER HEATING are intended to run off an
off-peak tariff (per systems listed in the text box above) ... If that results
in a dual meter, assign tariff per rules 1 to 4." The text-box off-peak systems
include DUAL ELECTRIC IMMERSION. Our `rdsap_tariff_for_cert` only triggered the
Unknown→off-peak exception on a storage/CPSU MAIN — it ignored the
dual-electric-immersion WATER-heating trigger, so an Unknown-meter dwelling
with a non-storage main (e.g. room heaters) + dual immersion was billed
STANDARD (13.19p flat) when it should be dual → Rules 1-4 on the main.
Fix: thread `water_is_off_peak_dual_immersion` (whc 903 + immersion lodged dual
via `_immersion_is_single is False`) into the Unknown-meter branch; when any
text-box trigger is present, resolve via the same Rules 1-4 dispatch (room
heaters → Rule 3 → 10-hour). Single-immersion / instantaneous (whc 909) certs
correctly stay STANDARD (no text-box system).
Worksheet-validated on "simulated case 48" (main 691 + Unknown meter + 903 dual
immersion): Elmhurst 10-Hour Off Peak, SAP 57; ours 45 → 55 (7-hour gives 45,
confirming 10-hour). Flips exactly ONE corpus cert — Apartment 241 (the genuine
-5.38 under-rater, main 691 + dual immersion) -5.38 → -1.05; every other
Unknown+dual-immersion cert already has a storage main (Rule 2). Corpus
within-0.5 holds 72.5%, MAE 0.793 → 0.789 (improved). CO2/PE unchanged.
GSHP/WSHP-main trigger (the other §12 Unknown exception bullet) is a separate
follow-up. Gates green: corpus 72.5%/0.789, batch worksheet 0 raised/0 diverge,
000565 e2e 11/11, suite 2987 passed (2 known pre-existing fails). pyright not
installed in this container — strict type gate not run locally.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two second-main fuel errors mis-cost a dual-main dwelling whose two main systems
burn different fuels (SAP 10.2 §10a worksheet (213) bills main 2 at its own fuel):
1. Off-peak/legacy scalar cost path (calculator.py + cert_to_inputs.py): main 2's
kWh was priced at main 1's `space_heating_fuel_cost_gbp_per_kwh` scalar. Split
main 1 vs main 2 and price main 2 at its OWN rate via the new
`_main_2_space_heating_fuel_cost_gbp_per_kwh` (+ CalculatorInputs field).
Scoped to a NON-electric second main (wood/oil/coal) — an electric second
main keeps main 1's scalar (its off-peak Table 12a split is the deferred §10a
slice; per-system splitting it regresses the off-peak electric cohort, certs
13 Parkers Hill / 34 Dunley Road). 0 corpus impact (no corpus cert has a
non-electric second main on an off-peak meter).
2. Elmhurst Summary mapper (mapper.py): when §14.1 omits the Fuel Type cell, a
fuel-fired second main (room-heater SAP code) inherited main 1's fuel. Derive
it from the SAP code's Table 4a category (solid 631-636 -> house coal, gas ->
mains gas, liquid -> oil) before the main-1 inherit, mirroring
`_elmhurst_secondary_fuel_from_sap_code` (same modal sub-fuel caveat). Boiler
codes (<601) still inherit main 1 (case 6 oil rads+UFH).
simulated case 47 (electric room heaters + solid room heaters 633): our SAP
37.81 -> 55.09 vs Elmhurst current 57 (residual is the wood-vs-coal sub-fuel the
Summary export does not carry). Corpus unchanged 72.5% / MAE 0.793; batch 0
raised / 0 diverge; 000565 e2e green. (mapper.py also carries an earlier,
behaviour-free roof-window doc comment.) Spec-cited unit pins added (AAA).
pyright not installed locally — strict type gate not run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`table_32.unit_price_p_per_kwh` silently returned the mains-gas default
(3.48 p/kWh) for any fuel code it could not resolve to a Table 32 price or a
translatable gov-API enum. An unhandled fuel billed at the gas rate mis-costs
the dwelling (same failure mode as the dual-main wood-vs-electric over-cost).
Raise `UnpricedFuelCode` (new, mirrors MissingMainFuelType / UnmappedSapCode)
so the gap surfaces at the price boundary. `None` (no fuel lodged) still
defaults — callers resolve "no system" upstream.
0 corpus impact: all 1000 certs compute (every lodged fuel resolves), so this
is a forward guard against future/unmapped fuels. Unit pin added; existing
None-default test docstring tightened. pyright not installed locally — strict
type gate not run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Main heating system 2's space-heating fuel cost (worksheet (213)) was billed
at main system 1's Table 32 unit price (`main_2_high_rate_gbp_per_kwh` reused
`main_1_high_rate_gbp_per_kwh`). For a dual-FUEL pair this grossly mis-costs the
second main: cert 10032957680 "Copse Cottage" (main 1 electric room heaters
fuel 30, main 2 wood logs fuel 6) charged its 9481 kWh of wood at 13.19 p/kWh
instead of 4.23 p/kWh — +£850/yr → SAP 21.75 vs lodged 45.
Route main 2 through its own fuel code (`_main_fuel_code(details[1])`), mirroring
the existing secondary-fuel handling. Copse Cottage 21.75 -> 45.94. Corpus
within-0.5 holds 72.5%, SAP MAE 0.815 -> 0.793 (ratcheted ceiling 0.82 -> 0.80);
CO2/PE unchanged. Same-fuel dual mains (gas+gas) unaffected. Off-peak-tariff
dual-fuel mains still defer to the legacy scalar path (separate slice).
Spec-cited unit pin added (AAA). pyright not installed locally — strict type
gate not run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make run_modelling_e2e the single script that does everything for a portfolio,
so the 291-property run needs one invocation with per-property recovery (no
all-or-nothing chunking):
- On --persist, a lodged-EPC Property now also gets its Baseline Performance
row written via PropertyBaselineOrchestrator (per Property, so one bad cert
does not abort the batch). Predicted (EPC-less) Properties have no lodged
figures, so they get a Plan but no baseline row.
- The run CSV gains api_sap (register) vs baseline_sap (calculator) + sap_delta,
so calculator-vs-API divergence is reviewable per property.
Fill the off-catalogue overlay for the measures the live material catalogue
cannot price, so they stop crashing the run:
- double_glazing (£550/window) and secondary_glazing (£400/window): priced
per window (the generator multiplies by single-glazed window count, matching
the legacy window_glazing). Grounded in 2025/26 UK installed costs; per-window
is the right unit for windows (fixed per-unit install dominates) — per-m2 fits
walls/floors, not glazing.
- gas_boiler_upgrade / system_tune_up / system_tune_up_zoned: these are priced
off the heating rate sheet (Products()), with get() reading the catalogue only
for an id — so the overlay entry exists to satisfy that lookup (material_id
stays None, as with ASHP); the rate sheet remains authoritative.
Validated on a 12-property sample (incl. a secondary-glazing case and a
SAP-Schema-16.2 cert): 12/12 baseline rows + plans, 0 errors.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Table 5 reads "Number of extract fans if known; if number is unknown:
[age-band default]" — the default is an UNKNOWN-fallback, NOT a floor. The
cascade applied `max(lodged, table_5_default)`, flooring a genuinely-lodged
count up to the age-band minimum: e.g. an age H-M dwelling lodging 2 extract
fans was billed at the 6-8-room default of 3, over-counting ventilation line
(8) and the heat-loss coefficient. Fixed to `lodged if lodged > 0 else
default` (a lodged 0 is the Elmhurst/RdSAP "unknown" form → default; any
positive count is taken literally).
Surfaced by Khalim's Elmhurst stress worksheet (simulated case 46): this was
its last ventilation residual — our Jan effective ACH 9.14 -> 9.0748 (exact
match to the accredited worksheet), SAP 29 -> 30 = Elmhurst, cost £1496 vs
£1493. Corpus IMPROVED: within-0.5 71.6% -> 72.5%, MAE 0.819 -> 0.815 (the
max-flooring over-counted ventilation on every cert lodging fans below its
age default). Floor ratcheted 0.71 -> 0.72. pyright not installed locally.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When the main heating system does not heat every habitable room (heated rooms
< habitable rooms), SAP 10.2 Appendix A.2.2 assumes the unheated rooms are
served by a portable-electric secondary heater, so the Table 11 secondary
fraction (0.10 for a boiler main) must be costed at the electricity tariff —
even when the cert lodges no explicit secondary system.
`_secondary_fraction` previously returned 0 unless a secondary was lodged or
the main was a forced-secondary electric-storage code, dropping the assumed
secondary and billing 100% of space heat to the (cheaper) main fuel — an
over-rate. Added an `unheated_habitable_rooms` trigger plus
`_has_unheated_habitable_rooms(epc)`, which prefers the lodged
`any_unheated_rooms` flag and guards the gov-API `heated_rooms_count == 0`
"not provided" sentinel. The secondary fuel/efficiency cascade already
defaults to portable electric (code 693) when no secondary code is lodged.
Worksheet-validated on simulated case 46 (heated 4 < habitable 7, no lodged
secondary): the assumed 10% electric secondary (2289 kWh, ~£260) lifted our
SAP 39 -> 29.35 vs accredited Elmhurst 30 (cost £1502 vs £1493, within 0.6%).
Corpus UNCHANGED (71.6% / MAE 0.819): all 17 corpus certs with heated <
habitable already lodge an explicit secondary description, so the gov-API
path was already costing it; this only adds the assumed secondary where none
is lodged (Elmhurst / reduced-field path). pyright not installed locally.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The full RdSAP-Schema-21 label is "Double between 2002 and 2021" (double
glazing installed 2002-2021, SAP 10.2 Table 24 code 3). When the Elmhurst
Summary PDF wraps the trailing "and 2021" into an adjacent table cell that
the extractor joins away, the surfaced label truncates to "Double between
2002" — the same artifact already handled for "Triple post or during".
`_elmhurst_glazing_type_code` raised UnmappedElmhurstLabel on it, blocking
the whole Summary (surfaced on the simulated-case-46 multi-attribute
worksheet). Added the truncated form as a code-3 alias.
pyright not installed in this container.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The corrected stone branch (age-E-only cap, §3.5 Table-3 unknown-thickness
default, Scotland band-J override) improved the corpus gauge to SAP MAE 0.819
and PE MAE 3.6 kWh/m2/yr. Lock the gains in and record the corrected
mechanism in the provenance log; within-0.5 (71.6%) and CO2 (0.08) unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two-pass org_ref-matched builder for property_overrides (classify via ChatGPT
into the landlord ledger, validate+apply user edits, write idempotently);
ephemeral-Postgres smoke proves the one-property chain without creds.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Caught live writing property_overrides on portfolio 796: the Python
override_component SAEnum lagged the DB enum, so reading a new-component row
back threw LookupError. Guard it with a consistency test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The FE-owned `material.type` pgEnum cannot carry `secondary_heating_removal`,
so pricing it through the DB catalogue raises a DataError that poisons the
session — the modelling pipeline crashed on any property with a lodged
secondary heater unless the measure was excluded on the Scenario.
Realise the `ProductRepository` docstring's intent (DB catalogue today, a JSON
file for costs the ETL does not yet supply, behind the same port): add a
`CompositeProductRepository` that resolves an override source first, then the
catalogue. Checking the override first keeps that Measure Type away from the DB
entirely; every other type misses the override and falls through unchanged.
- off_catalogue_costs.json prices it at £270 flat per-dwelling — the legacy
`Costs.heater_removal` ported to the new flat model (ADR-0028):
(£25 + £200 baseline) x 1.2 VAT, for the single fixed secondary a cert lodges.
Contingency (0.25) is joined from config, not the file.
- Wire the composite into PostgresUnitOfWork.product and run_modelling_e2e, so
the first-run pipeline and the local runner both honour the overlay.
- Integration test: drop the unrealistic seeded secondary_heating_removal DB
rows (the pgEnum can't hold the type) and assert it is JSON-sourced
(material_id is None, cost £270) end-to-end through a real Unit of Work.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An uninsulated stone wall of lodged thickness, age bands A-E, is billed by
the RdSAP 10 §5.6 Table-12 formula on its measured thickness — sandstone /
limestone U = 54.876·W^-0.561, granite / whinstone U = 45.315·W^-0.513. Two
bugs suppressed it:
1. CAP: the as-built branch capped the formula result at the Table-6
typical-thickness default (`if u0 >= 1.7: return 1.7`). But the formula
only dips below 1.7 past ~488 mm (sandstone) / ~640 mm (granite), so the
cap nullified §5.6 for essentially every real-thickness stone wall,
under-counting fabric loss and over-rating. A measured 400 mm sandstone
age-B wall is 1.90 (Elmhurst-confirmed), not 1.70.
2. NO INSULATION-STATE GATE: the branch fired for any stone wall with a
lodged thickness, including ones whose insulation is "Unknown". RdSAP
treats an unknown-insulation wall via the Table-6 typical-thickness
default, NOT as bare stone of the lodged thickness — so a 50 mm granite
wall with Unknown insulation must read 1.70 (worksheet), not the formula's
6.09. Gated on `wall_insulation_type is not None`.
Fixes the 2 long-standing stone-U unit tests (granite 120 mm → 3.8871,
sandstone 120 mm → 3.7408 — they were correct red tests, not "known fails").
Corpus within-0.5 70.3% -> 71.6% (MAE 0.833 -> 0.822); ratcheted floors to
0.71 / 0.83. Worksheet fixture 000565 (granite 50 mm Unknown → 1.70) still
passes via the insulation gate. The two stone groups (sandstone/limestone vs
granite/whinstone) keep their distinct §5.6 coefficients.
pyright not installed in this codespace (strict gate not run locally).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The SAP rating is spec-floored at 1 ("if the result of the calculation is
less than 1, the rating is 1"). `sap_rating_integer` already clamps, but the
continuous `sap_score_continuous` did not — so a degenerate dwelling could
emit a physically impossible negative SAP. Apply the same max(1, …) floor to
the continuous value (the un-rounded part is for sensitivity near real
ratings, not for negative ratings).
Removes a -12.3 accuracy outlier on the committed corpus (cert 422000111926,
lodged at the floor of 1, was computing -11.3): within-0.5 70.2% -> 70.3%,
MAE 0.845 -> 0.833. Ratcheted the corpus MAE ceiling to 0.84. Unit-pinned in
test_calculator.
pyright not installed in this codespace (strict gate not run locally).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
