SAP 10.2 §9.4.11 (PDF p.30) "Boiler interlock":
For the purposes of the SAP, an interlocked system is one in which
both the space and stored water heating are interlocked. If either
is not, the 5% seasonal efficiency reduction is applied to both
space and water heating; if both are interlocked no reductions are
made.
Table 4c (PDF p.169-170) lodges -5 for both Space and DHW columns on
the "No boiler interlock — regular boiler" row. Pre-slice the cascade
applied the -5pp adjustment ONLY to the `water_eff` scalar fallback
(`cert_to_inputs.py:4354`) and missed:
(a) the SH efficiency path (cascade kept the raw PCDB winter eff for
space heating);
(b) the PCDB Equation D1 monthly cascade (Eq D1 received raw
winter/summer values without the -5pp adjustment).
RdSAP §3 (PDF p.57) defines boiler interlock as "Assumed present if
there is a room thermostat and (for stored hot water systems heated
by the boiler) a cylinder thermostat. Otherwise not interlocked."
Cert pcdb 1 (Potterton KOA PCDB 716 + 110 L cylinder + Cylinder Stat:
No) reproduces the pattern: worksheet (210) = 60% = PCDB winter
65 - 5; worksheet (217)m monthly Eq D1 pivots on (winter 60,
summer 48) not (65, 53).
The SH path is further gated on `pcdb_main is not None` because
§9.4.11 only applies to "gas and liquid fuel boilers" — cert 000565
(ASHP Main 1) keeps its raw SH eff. The combi-fed-cylinder DHW path
(cert 000565 WHC 914 to PCDB combi Main 2) continues to receive its
existing -5pp via the `water_pcdb_main` gate (unchanged).
Corpus impact: pcdb 1 SAP residual +6.95 → +3.40; cost -£157.61 →
-£75.68; CO2 -845.81 → -397.02; PE -3135.30 → -1601.74. No other
variant has PCDB main + cylinder + no thermostat, so the other 24
corpus pins are unchanged.
Extended handover suite: 884 pass, 0 fail (was 883 + 1 new AAA test
pinning the §9.4.11 SH eff path).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Probed all three variants (electric 3, oil 1, solid fuel 2) in this
thread. Each has a different driver despite the matching magnitude:
- electric 3: §9 useful-demand calc for ctrl=3 storage heaters
- oil 1: HW efficiency for Table 4b oil boiler (cascade 86% vs ws ~65%)
- solid fuel 2: HW kWh lodged in different line ref (re-probe needed)
Tested combined-R hypothesis (effective_R = (1-frac)·R_main + frac·R_sec
per SAP 10.2 §9b) — the cascade currently DOES NOT pass secondary_fraction
to mean_internal_temperature_monthly, so effective_R = R_main. Monkey-
patching to inject combined R REGRESSES electric 3 (+2.55 → +3.17)
because raising R lowers cascade demand — opposite of needed direction.
Recommends taking the three variants as separate per-variant slices.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two compounding bugs were over-counting the SAP 10.2 §4 (56)m cylinder
storage loss by ~76 kWh/yr across all 17 cylinder-with-immersion
corpus variants (cascade HW kWh 2460.40 vs worksheet 2384.12):
(1) **Extractor gap.** Elmhurst Summary §15.1 "Hot Water Cylinder"
block lodges `Cylinder Size` / `Insulation Thickness` but NOT
`Cylinder Thermostat`. The thermostat is lodged separately in
§16 "Recommendations" as `Cylinder thermostat (Already installed)`.
The extractor only searched §15.1, so `cylinder_thermostat`
resolved to None for every variant on property 001431. The
cascade then defaulted `has_cylinder_thermostat=False`, applying
SAP 10.2 Table 2b's ×1.3 "no thermostat" multiplier.
(2) **Cascade spec gap.** `_separately_timed_dhw` returned True for
any cylinder-lodged cert regardless of HW fuel. Per SAP 10.2
Table 2b note b) (PDF p.159):
> "Multiply Temperature Factor by 0.9 if there is separate time
> control of domestic hot water (boiler systems, warm air systems
> and heat pump systems)"
Electric immersion is NOT in the bracketed list — the ×0.9
reduction is restricted to boiler / warm-air / HP systems. Pre-
slice the cascade over-applied ×0.9 on electric-immersion certs.
Combined, the cascade computed TF = 0.60 × 1.3 × 0.9 = 0.702 vs the
worksheet's TF = 0.60 (base — thermostat present, immersion exempt).
After both fixes the cascade HW kWh matches the worksheet's (64) at
1e-3 precision (2384.116 vs 2384.12).
Corpus impact (16 cylinder-with-immersion variants on 18-hour meter):
| variant | SAP_c shift | Cost shift |
|--------------|------------:|-----------:|
| electric 1 | -0.20 → -0.06 | -£3.34 |
| electric 2 | -1.27 → +0.47 | -£4.44 |
| electric 3 | +2.42 → +2.55 | -£2.91 |
| electric 5 | -0.06 → +0.07 | -£3.06 |
| electric 6 | +1.19 → +1.33 | -£3.20 |
| electric 7 | +1.14 → +1.29 | -£3.35 |
| electric 8 | -0.41 → -0.26 | -£3.50 |
| electric 9 | -0.24 → -0.12 | -£2.91 |
| solid fuel 4-11 | -0.45..-0.09 → -0.29..+0.10 | -£3 to -£4 |
The HW kWh line closes cleanly; some SAP residuals sign-flip slightly
because the cascade's now-correct HW kWh exposes the SH+Sec demand
mismatch for storage heaters (electric 3/6/7 — open driver is the
Table 11 `main_heating_category=None` default for codes 401/402,
queued for a mapper-side slice).
Tests:
- new AAA test `test_separately_timed_dhw_excludes_electric_immersion_per_table_2b_note_b`
- 16 corpus pins re-tightened (8 electric + 8 solid fuel)
Extended handover suite: 883 pass (was 882; +1 new test), 0 fail.
Pyright net-zero on touched files (43 → 43 errors, all pre-existing).
Per [[feedback-spec-citation-in-commits]] +
[[feedback-spec-floor-skepticism]] (the "HW +76 kWh uniform overcount"
across 17 variants traced to TWO spec-citable defaults the cascade
was getting wrong, not a precision floor).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Pre-slice `_is_off_peak_meter` carried its own string-dispatch that
only recognised the RdSAP 10 long form `"off-peak 18 hour"`. The bare
`"18 Hour"` lodging (Elmhurst Summary §14.2 surface form, lodged by
41/41 corpus variants) fell into the catch-all `return False` branch.
That mis-classified every 18-hour cert as non-off-peak for the
secondary / PV cost paths and billed electric secondary heating at
standard 13.19 p/kWh (Table 32 code 30) instead of the 18-hour low
rate 7.41 p/kWh (Table 32 code 40).
The fix routes `_is_off_peak_meter` through `tariff_from_meter_type`
so every lodging form already recognised there (int 1/4/5, `"18 Hour"`,
`"off-peak 18 hour"`, `"Dual"`, `"Dual (24 hour)"`, numeric strings)
is consistently classified. Single (code 2) stays standard; Unknown
(code 3) retains the heuristic "electric end-uses on Unknown meters
typically come from E7-eligible dwellings whose tariff the assessor
couldn't pin down — apply off-peak". Per
[[feedback-zero-error-strict]] the now-dead `_RDSAP_DEFINITELY_OFF_PEAK`
frozenset is deleted (canonical dispatch covers the same codes).
Spec citation per [[feedback-spec-citation-in-commits]]:
> RdSAP 10 §17 page 85 row 10-2 (Electricity meter): "Dual / single /
> 10-hour / 18-hour / 24-hour / unknown"
> RdSAP 10 §12 page 62: "if the meter is dual 18-hour/24-hour it is
> 18-hour/24-hour tariff"
Corpus impact (6 storage-heater / underfloor variants on forced
secondary):
| variant | SAP code | old ΔSAP | new ΔSAP |
|---|---:|---:|---:|
| electric 3 | 401 | -0.10 | +2.42 |
| electric 5 | 402 | -2.48 | -0.06 |
| electric 6 | 404 | -1.14 | +1.19 |
| electric 7 | 408 | -1.08 | +1.14 |
| electric 8 | 409 | -2.54 | -0.41 |
| electric 9 | 421 | -2.76 | -0.24 |
Total absolute SAP residual across the cluster: 10.10 → 5.46.
The 3 sign-flipped variants (electric 3/6/7) surface a separate
cascade bug: `_secondary_heating_fraction_for_category` defaults to
0.10 when the mapper leaves `main_heating_category=None` for electric
storage, but the worksheet for codes 401/402 uses 0.15 = Table 11
Cat 7. Mapper-side fix queued.
Tests:
- new AAA test `test_is_off_peak_meter_recognises_bare_18_hour_lodging`
covers 7 lodging forms (bare, lowercase, long-form, Single, standard,
Unknown+electric, Unknown+non-electric)
- 6 corpus pins re-tightened (electric 3/5/6/7/8/9)
Extended handover suite: 882 pass (was 881; +1 new test), 0 fail.
Pyright net-zero on touched files (43 → 43 errors, all pre-existing).
Per [[reference-unmapped-sap-code]] strict-dispatch routing.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Pre-slice every off-peak callsite in `cert_to_inputs.py` —
`_space_heating_fuel_cost_gbp_per_kwh`, `_hot_water_fuel_cost_gbp_per_kwh`,
`_secondary_fuel_cost_gbp_per_kwh`, `_pv_dwelling_import_price_gbp_per_kwh` —
hardcoded `prices.e7_low_rate_p_per_kwh = 5.50` p/kWh (Table 32 code 31,
the 7-hour low rate) regardless of the cert's actual tariff. Every
18-hour cert was thereby under-charged 1.91 p/kWh × off-peak kWh on
its space-heating, hot-water, and secondary-heating cost rows.
Per RdSAP 10 §19 Table 32 (p.95):
> "Electricity ... 7-hour tariff (low rate / off-peak) — code 31 5.50 p/kWh
> ... 10-hour tariff (low rate) — code 33 7.50 p/kWh
> ... 18-hour tariff (low rate) — code 40 7.41 p/kWh
> ... 24-hour tariff — code 35 6.61 p/kWh"
The fix routes through a new `_off_peak_low_rate_gbp_per_kwh(tariff)`
helper that reads the existing per-tariff Table 32 lookup
(`_TARIFF_HIGH_LOW_RATES_P_PER_KWH`). A companion
`_off_peak_low_rate_gbp_per_kwh_via_meter_heuristic(meter_type)` covers
the secondary / PV paths that detect off-peak via the
`_is_off_peak_meter` heuristic (RdSAP meter code 3 = Unknown is treated
as off-peak for electric end-uses), falling back to the SEVEN_HOUR rate
when the meter resolves to STANDARD — codifying the heuristic that the
literal 5.50 constant used to embed.
Per [[feedback-zero-error-strict]] the now-dead
`PriceTable.e7_low_rate_p_per_kwh` field is deleted (no fallback can
silently re-introduce the 5.50 hardcode); the field's docstring +
RDSAP_10_TABLE_32_PRICES instantiation update to point at the new
helpers.
Corpus closure (all 18-hour cohort):
- 8 electric variants — SAP +5.85..+9.64 → -0.10..-2.76; cost
-£135..-£222 → +£2..+£64
- ashp +5.67 → +0.24 SAP (-£131 → -£5.57)
- gshp +5.16 → +1.15 SAP (-£119 → -£26)
- solid fuel 4..11 — SAP +1.59..+2.04 → ±0.45 (cost ±£10)
Golden 0240 PV path also closes (was raising UnmappedSapCode on
Unknown-meter probe — surfaced an unreachable PV literal that the
meter-heuristic helper now resolves).
Tests:
- new AAA test `test_space_heating_off_peak_fallback_uses_actual_tariff_low_rate_not_e7`
exercising the EIGHTEEN_HOUR fallback at the helper level
- 19 corpus pins re-tightened (8 electric + ashp + gshp + 8 solid-fuel
+ golden 0240's implicit pin)
Extended handover suite: 881 pass (was 880; +1 new test), 0 fail.
Pyright net-zero on touched files (43 → 43 errors, all pre-existing).
Per [[feedback-spec-citation-in-commits]] +
[[feedback-worksheet-not-api-reference]] +
[[reference-unmapped-sap-code]].
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Continuation of S0380.135's Table 4a per-heating-system responsiveness
dispatch (`_RESPONSIVENESS_BY_SAP_CODE` in cert_to_inputs.py). The
solid-fuel coverage closed 10 corpus variants; this slice extends the
dispatch to the electric heating SAP code ranges from SAP 10.2 Table
4a (PDF p.170):
401 Old (large volume) storage heaters R=0.00
402 Slimline storage heaters R=0.20
403 Convector storage heaters R=0.20
404 Fan storage heaters R=0.40
405 Slimline storage heaters + Celect-type ctrl R=0.40
407 Fan storage heaters + Celect-type ctrl R=0.60
408 Integrated storage+direct-acting heater R=0.60
409 High heat retention storage heaters (§9.2.8) R=0.80
421 In concrete slab (off-peak only) R=0.00
422 Integrated (storage+direct-acting) R=0.25
423 Integrated with low off-peak R=0.50
424 In screed above insulation R=0.75
425 In timber floor / immediately below covering R=1.00
515 Electricaire system R=0.75
691 Panel, convector or radiant heaters R=1.00
694 Water- or oil-filled radiators R=1.00
701 Electric ceiling heating R=0.75
A few electric storage codes (402, 403, 405, 407) carry a *different*
R value in the 24-hour tariff section of Table 4a vs the off-peak
section (e.g. Slimline 402 = R=0.20 off-peak / R=0.40 24-hour). This
dict captures the off-peak value as the default because the 24-hour
tariff is rare in the corpus (no variant lodges it). If a 24-hour-
tariff cert surfaces with one of these codes the dispatch needs to be
promoted to a (sap_code, tariff) lookup; until then the off-peak
default applies.
Heating-systems corpus impact — 6 electric corpus variants re-pinned:
variant SAP R ΔSAP was ΔPE was
electric 3 401 0.00 +9.43 +14.70 -1059 -3189
electric 5 402 0.20 +6.76 +10.97 -96 -1798
electric 6 404 0.40 +7.82 +10.97 -494 -1770
electric 7 408 0.60 +7.58 +9.68 -428 -1277
electric 8 409 0.80 +5.84 +6.89 +200 -224
electric 9 421 0.00 +6.77 +12.03 +154 -1976
3/6 PE residuals close to ±200 kWh (electric 5/8/9). The remaining
+5..+9 SAP residuals across all electric variants suggest a separate
shared cascade gap (likely Table 12a high/low-rate fraction or pumps/
fans electric handling — fuel cost is consistently under-counted by
~£100-£220 across the cluster). Queued for follow-up.
electric 1 (SAP 191 Direct acting electric boiler) and electric 2
(SAP 524 Air source heat pump) unchanged — both have spec R=1.0
already (matched the Table 4d emitter fallback).
Extended handover suite: 880 pass / 0 fail (+1 new AAA test
covering the 17 electric R-dispatch entries).
Pyright net-zero on touched files (43 → 43).
No golden fixture impact — no golden cert lodges a covered electric
SAP code via the cascade path that would shift residuals.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`_is_electric_main` and `_is_electric_water` hand-rolled a literal set
check `code in {10, 25, 29}` ∪ `{30..40}` to classify a fuel code as
electricity. The set conflated two enums:
- {10, 25, 29} — API enum codes (epc_codes.csv row main_fuel):
10 = electricity (backwards compat)
25 = electricity (community)
29 = electricity (not community)
- {30, 31, ..., 40} — Table 32 codes (RdSAP 10 spec p.95):
30 = standard tariff
31/32 = 7-hour low/high
33/34 = 10-hour low/high
35 = 24-hour heating
38/40 = 18-hour high/low
API enum codes 1-29 collide with Table 32 codes 1-29 for unrelated
fuels — API 10 = "electricity" vs Table 32 10 = "dual fuel (mineral +
wood)". S0380.135's EES dispatch sets `main_fuel_type` to Table 32
codes (BDI → 10 for dual fuel), so a dual-fuel main was silently
mis-classified as electric. The `_space_heating_fuel_cost_gbp_per_kwh`
tariff branch then re-routed solid fuel 6's space heating cost through
the 18-hour-low electric rate (5.50 p/kWh) instead of dual-fuel 3.99
p/kWh — solid fuel 6 SAP residual −7.38 → −11.37 in S0380.135.
The fix promotes the existing `table_32._is_electric_code` to public
`is_electric_fuel_code` and routes both `_is_electric_main` and
`_is_electric_water` through it. The canonical helper normalises a
fuel code via T32-first then API-translate fallback (same convention
as `unit_price_p_per_kwh`), so a Table-32-code-10 dual-fuel main
classifies as non-electric correctly.
Subtle behaviour change: API enum code 25 ("electricity (community)")
maps via API_FUEL_TO_TABLE_32 to Table 32 code 41 ("heat from electric
heat pump (community)") which is a heat network billed at the heat-
network rate (4.24 p/kWh single rate), not at the off-peak electric
tariff. Pre-S0380.136 the literal-set check would have treated this
as direct electric and applied the Table 12a high/low-rate split —
that was wrong; community heat networks don't have an off-peak split.
The new canonical helper correctly excludes code 41 from
_ELECTRIC_FUEL_CODES.
Heating-systems corpus impact:
solid fuel 6 (Dual Fuel Anthracite Wood, SAP 160):
ΔSAP −11.3731 → +1.9493 (now in cluster with other solid-fuel)
Δcost +£268.44 → −£44.91
ΔPE unchanged (PE wasn't affected by the cost mis-routing)
No other corpus variants moved — none have `main_fuel_type` in the
ambiguous API/T32 collision range that was previously mis-classified.
Extended handover suite: 879 pass / 0 fail (+2 from new AAA tests
covering both `_is_electric_main` and `_is_electric_water` dual-fuel
non-electric classification + API code 29 → electric / API code 25 →
heat-network non-electric semantics).
Pyright net-zero on touched files (43 → 43).
No golden fixture impact — no golden cert lodges `main_fuel_type=10`
(dual fuel) on the cascade path.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
SAP 10.2 spec line 15271:
"R = responsiveness of main heating system (Table 4a or Table 4d)"
The cascade's `_responsiveness` was keyed solely on `heat_emitter_type`
(Table 4d), which is correct for systems whose responsiveness is
determined by the emitter (gas / oil / HP boilers feeding radiators or
UFH). But for systems with intrinsically low responsiveness — solid-
fuel room heaters, range cookers, independent solid-fuel boilers —
the spec lodges R directly in Table 4a against the heating-system SAP
code, and that value overrides any emitter-based lookup.
For solid fuel 8 (SAP code 160 = "Range cooker boiler (integral oven
and boiler)", lodged with radiators emitter), pre-slice the cascade
returned R = 1.0 (radiators) instead of the spec-correct R = 0.50
(Table 4a p.169). The Table 9b mean-internal-temperature adjustment
then over-estimated heating-system response, under-estimating space
heating demand by ~10% (cascade demand 6874.80 kWh vs worksheet EPC
implied 7566 kWh).
The fix adds a new dispatch `_RESPONSIVENESS_BY_SAP_CODE` consulted
first in `_responsiveness`; SAP codes not in the dict fall through to
the existing Table 4d emitter lookup.
Table 4a entries added (SAP 10.2 PDF p.169-170):
151 Manual feed independent boiler R=0.75
153 Auto (gravity) feed independent boiler R=0.75
155 Wood chip/pellet independent boiler R=0.75
156 Open fire with back boiler to radiators R=0.50
158 Closed room heater with boiler to radiators R=0.50
159 Stove (pellet-fired) with boiler to radiators R=0.75
160 Range cooker boiler (integral oven+boiler) R=0.50
161 Range cooker boiler (independent oven+boiler) R=0.50
631 Open fire in grate R=0.50
632 Open fire with back boiler (no radiators) R=0.50
633 Closed room heater R=0.50
634 Closed room heater with boiler (no radiators) R=0.50
635 Stove (pellet fired) R=0.75
636 Stove (pellet fired) with boiler (no rads) R=0.75
Heating-systems corpus impact — 10 solid-fuel variants re-pinned:
variant ΔSAP was Δcost was ΔPE was
solid fuel 2 +2.64 +4.79 -£60 -£110 -1211 -2292
solid fuel 3 +1.32 +4.43 -£30 -£102 -935 -2496
solid fuel 4 +1.59 +4.13 -£37 -£95 +151 -1097
solid fuel 5 +1.70 +2.71 -£39 -£62 +160 -331
solid fuel 6 -11.37 -7.38 +£268 +£168 +87 -1313 ← see below
solid fuel 7 +2.04 +5.82 -£47 -£131 +44 -1638
solid fuel 8 +1.81 +4.24 -£42 -£98 +88 -1308
solid fuel 9 +1.71 +3.44 -£39 -£79 +155 -510
solid fuel 10 +1.75 +5.14 -£40 -£118 +120 -1315
solid fuel 11 +1.62 +4.35 -£37 -£100 +171 -962
7/10 PE residuals close to ±220 kWh (down from -331..-2496). 9/10 SAP
residuals tighten to +1.32..+2.64 (down from +2.71..+5.82).
solid fuel 6 (Dual Fuel Anthracite Wood, SAP 160) SAP residual
regresses -7.38 → -11.37 while PE closes +87. The dual-fuel cascade
has a separate bug now exposed by the more-accurate demand calc;
queued for a follow-up slice.
Non-solid-fuel variants (15) unchanged — their SAP codes aren't in
the new dispatch dict so they fall through to Table 4d as before.
Electric storage Table 4a rows (193-196, 422-424, 515, 701) and the
spec's other low-responsiveness codes can be added in follow-up
slices as electric corpus variants are unblocked.
Extended handover suite: 877 pass / 0 fail (+1 new responsiveness
AAA test). Pyright net-zero on touched files (43 → 43).
No golden fixture impact — no golden cert lodges a solid-fuel SAP
code via the cascade path.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The SAP 10.2 worksheet computes each existing-dwelling metric in two
distinct blocks:
1. "ENERGY RATING" block — uses Table 12 regulated prices + UK-
average climate. Produces SAP score (Block 11a), total fuel
cost (255), total CO2 (272).
2. "EPC COSTS, EMISSIONS AND PRIMARY ENERGY" block — uses Table 32
prices + postcode-specific climate. Produces total CO2 (272)
again with different value, total PE (286).
The two blocks operate on different space-heating demand kWh per
SAP 10.2 §13 (e.g. solid fuel 8: 21097 kWh in rating block vs
16813 kWh in EPC block for London W6).
The corpus regression test was extracting all four pins and asserting
against the cascade's rating-mode result (`cert_to_inputs`). That was
apples-to-apples for SAP/cost/CO2 (the first `(255)` and `(272)`
matches the regex finds ARE in the rating block) but apples-to-
oranges for PE: the `(286)` Total PE only exists in the EPC block,
so every PE pin was comparing rating-mode cascade output against
EPC-block worksheet output. The mismatch inflated every PE residual
by 10-15% of total PE.
The fix runs both cascade modes in the Act phase and assigns:
- rating-mode result → SAP / cost / CO2 residuals
- demand-mode result (`cert_to_demand_inputs`) → PE residual
25 corpus _CorpusExpectation entries re-pinned. Some closed
dramatically (apples-to-apples reveals the cascade was actually
correct):
ashp +1467.90 → -11.80 ← effectively closed
oil pcdb 1/2 +2086.75 → -83.82
oil pcdb 3 +1897.43 → -271.44
electric 1 +2837.14 → +164.91
electric 8 +2113.83 → -224.46
solid fuel 5 +2359.85 → -330.84
Others surfaced larger demand-mode gaps that the block mismatch had
been hiding — these are real cascade gaps the next slices will
address:
electric 3 -850.93 → -3189.22
electric 5/6 +540/+568 → -1797.96 / -1769.84
pcdb 1 -171.70 → -3135.30
solid fuel 2/3 +440.75 / +1451.79 → -2292.47 / -2496.20
The corpus test docstring + per-block-attribution comment now make
the rating-vs-EPC block distinction explicit so future reviewers
don't repeat the same conflation.
Extended handover suite at HEAD post-slice: 876 pass / 0 fail
(unchanged — no test count change, just per-pin value updates).
Pyright net-zero on touched file (0 → 0).
No cascade behaviour change. No golden / unit-test impact (the bug
was specific to the corpus test's pin-extraction logic).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Final slice of ADR-0012: collapse the per-property read round-trips a batch
made (Baseline hydrated ~8 queries x 30 properties one at a time) into a
handful of per-table IN queries.
- EpcPostgresRepository: extracted a shared `_compose(rows)` from `get` (the
windows + floor-dim fetches are now passed in, not fetched inline), so both
`get` and the new `get_for_properties(property_ids)` build EpcPropertyData
from pre-fetched rows. `get_for_properties` fetches each child table once
(`WHERE epc_property_id IN ...`), groups in memory, and composes — load-whole
per ADR-0002.
- PropertyRepository.get_many(property_ids) -> Properties: one query for the
property rows + one bulk EPC hydration, composed in input order.
- BaselineOrchestrator / IngestionOrchestrator read the batch via get_many
instead of N x get.
- Ports + fakes gain the bulk methods.
The #1129 round-trip fidelity test stays green (the compose extraction is
behaviour-preserving). New tests: bulk hydration correctness + round-trips are
constant w.r.t. batch size (one-per-table, proven by query count). 123 pass;
pyright strict clean; AAA.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Elmhurst Summary §14.0 "Main Heating EES Code" is a three-letter
identifier that resolves to the specific fuel for solid-fuel main
heating systems. The §14.0 "Main Heating SAP Code" alone can't
disambiguate because Table 4a categorises solid-fuel systems by
appliance type rather than fuel — SAP code 160 ("Closed room heater
with boiler") is shared by anthracite, wood chips, dual fuel and
smokeless across the heating-systems corpus.
Three changes land together:
1. `MainHeating` dataclass (`elmhurst_site_notes.py`) gains a
`main_heating_ees: str = ""` field for the §14.0 EES code.
2. `ElmhurstSiteNotesExtractor._extract_main_heating` reads "Main
Heating EES Code" from §14.0.
3. `_map_elmhurst_sap_heating` adds a fourth fuel-derivation
fallback (after the existing electric-SAP-code + §15.0-liquid-
fuel branches): when `main_fuel_int is None` and the §14.0 EES
code is in `_ELMHURST_MAIN_HEATING_EES_TO_FUEL_CODE`, use that
dict's value as the main fuel.
Dict (corpus-derived, 10 entries → 7 distinct Table 32 fuels):
BAF, BAI, RAM → 15 anthracite (3.64 / 0.395 / 1.064)
BCC → 11 house coal (3.67 / 0.395 / 1.064)
BDI → 10 dual fuel (3.99 / 0.087 / 1.049)
BKI → 12 smokeless (4.61 / 0.366 / 1.261)
BQI → 21 wood chips (3.07 / 0.023 / 1.046)
RPS → 22 wood pellets bags (5.81 / 0.053 / 1.325)
RUN → 23 bulk pellets (5.26 / 0.053 / 1.325)
RWN → 20 wood logs (4.23 / 0.028 / 1.046)
Dict values are Table 32 fuel codes, NOT API `main_fuel` enum codes
— the API codes 1-9 collide with Table 32 codes for unrelated fuels
(e.g. API 5 = "anthracite" vs Table 32 5 = "bottled LPG main
heating"). `unit_price_p_per_kwh` / `co2_factor_kg_per_kwh` /
`primary_energy_factor` all check the Table 32 dict before falling
through to the API translation, so using Table 32 codes here avoids
the collision and routes cost/CO2/PE through the correct fuel row.
Heating-systems corpus impact — all 10 solid-fuel variants move
from `_BLOCKED_BY_MISSING_MAIN_FUEL_TYPE` (assert-on-raise) back
onto the residual-pin grid in `_EXPECTATIONS`:
variant ΔSAP Δcost ΔCO2 ΔPE
solid fuel 2 +4.79 -£110 -484 kg +441 kWh anthracite
solid fuel 3 +4.43 -£102 -1206 +1452 anthracite
solid fuel 4 +4.13 -£95 -714 +1655 anthracite
solid fuel 5 +2.71 -£62 -301 +2360 house coal — smallest
solid fuel 6 -7.38 +£168 -154 +2519 dual fuel — only negative
solid fuel 7 +5.82 -£131 -758 +2968 smokeless
solid fuel 8 +4.24 -£98 -15 +2513 wood chips
solid fuel 9 +3.44 -£79 -8 +2428 wood pellets bags
solid fuel 10 +5.14 -£118 -53 +1849 wood pellets bulk
solid fuel 11 +4.35 -£100 -9 +1536 wood logs
Remaining residuals trace to heating-system efficiency / control
type — separate slices. 16 variants still in `_BLOCKED`: community
heating ×5, electric storage ×4, no system, oil non-Heating-oil ×5,
Bulk LPG ×1. Each is its own derivation slice.
Extended handover suite at HEAD post-slice: 876 pass / 0 fail (was
875 + 1 new EES wiring AAA test).
Pyright net-zero on touched files (45 → 45 — all pre-existing).
No golden fixture impact — no golden cert lodges an EES code via
the Elmhurst path.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replaces the handler's whole-pipeline Session (one transaction across all
three stages, connection pinned during Ingestion's external IO) with a
Unit-of-Work per stage (ADR-0012, added here). Each stage runs its batch in
one unit and commits once; any property raising aborts the batch and the
subtask fails noisily.
- BaselineOrchestrator(unit_of_work, rebaseliner): one unit for the batch,
commit once. Raise on a pre-SAP10 property leaves the unit uncommitted.
- IngestionOrchestrator(unit_of_work, epc_fetcher, geospatial_repo,
solar_fetcher): fetch/write split — phase 1 fetches the whole batch (EPC /
coords / solar) with NO unit open; phase 2 writes in one unit and commits.
The connection is never held during external IO. Geospatial S3 repo stays
injected (reference data, not transactional).
- Handler: module-scoped engine (pool reused across warm invocations) + a UoW
factory; whole-pipeline `with Session` gone. `build_first_run_pipeline`
composes on the factory. Source clients still behind the raising seam.
- ADR-0012 records the decision (per-stage boundary, all-or-nothing batch,
idempotent re-run, fetch/write split, module-scoped engine). Modelling stub
left untouched (no-op, no DB) per the ADR.
Tests: orchestrators on a shared FakeUnitOfWork (assert persisted batch +
exactly-once commit + no-commit-on-raise). New real-DB E2E integration test:
real PostgresUnitOfWork, Ingestion writes the EPC → Baseline reads it back
through the repo → re-run replaces, not duplicates (1 EPC row, 1 baseline row
after two runs). 121 pass in tests/; pyright strict clean; AAA.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The cascade's `_main_fuel_code` previously returned None when
`MainHeatingDetail.main_fuel_type` was anything other than an int
(empty string, None, or an unmapped string label). The downstream
`table_32.unit_price_p_per_kwh(None)` then silently defaulted to mains
gas (3.48 p/kWh / CO2 0.21 kg/kWh / η 0.45 / PE 1.22) — a misleading
fallback where cost may happen to be close but CO2 / PE / efficiency
are completely wrong for the actual heating system.
Probe of the heating-systems corpus surfaced 26 of 41 controlled-
variable variants with `main_fuel_type=''`:
Community heating 1/2/3/4/6 (Table 4a 301-304) 5
Electric 11/12/13/14 (Table 4a 5xx/6xx/7xx) 4
No system (SAP code 699) 1
Oil 2 (HVO) / oil 3 (FAME) / oil 4 (FAME) /
oil 5 (bioethanol) / oil 6 (B30K) (Table 4b) 5
Solid fuel 2..11 (Table 4a 150-160 + 600-636) 10
pcdb 3 (lodges 'Bulk LPG' string — mapper dict gap) 1
Each pre-slice carried a residual pin in `_EXPECTATIONS` encoding the
broken mains-gas-default state. Solid fuel 8's +0.87 ΔSAP — the
"smallest open residual" the user asked to investigate next — turned
out to be the net of compensating cost/efficiency errors; the CO2
delta was +3525 kg/yr and PE +4103 kWh/yr because the cascade was
costing wood chips as mains gas.
Two changes land together:
1. Add `MissingMainFuelType(ValueError)` to
`domain/sap10_calculator/exceptions.py`. Semantics distinct from
the sibling `UnmappedSapCode` (which is for unmapped int dispatch
codes; this is for "value not resolvable to a SAP fuel code at
all"). The error message names the lodged value + the
`sap_main_heating_code` hint so the upstream mapper fix is
obvious.
2. `_main_fuel_code` in `cert_to_inputs.py` now raises
`MissingMainFuelType` when `main_fuel_type` is not an int.
`main is None` still returns None (genuinely no main heating).
The 26 blocked corpus variants are lifted out of the
`_EXPECTATIONS` residual-pin grid into a new tuple
`_BLOCKED_BY_MISSING_MAIN_FUEL_TYPE` driving a new parametrised test
`test_heating_systems_corpus_blocked_variant_raises_missing_main_fuel_type`
that asserts the raise for each blocked variant. As mapper-side fixes
land (deriving fuel from `sap_main_heating_code` via SAP 10.2 Table
4a/4b/4f, or extending `_ELMHURST_MAIN_FUEL_TO_SAP10`), variants move
back onto the residual-pin grid.
Mirrors the [[reference-unmapped-sap-code]] / [[reference-unmapped-
api-code]] strict-raise pattern: forcing function for spec/mapper
completion at the cascade boundary instead of silently producing
wrong outputs.
Extended handover suite at HEAD post-slice: 875 pass / 0 fail (was
874; +1 from the new `_main_fuel_code` strict-raise unit test;
26 blocked corpus pins replaced 1:1 by 26 assert-on-raise tests).
Pyright net-zero (43 → 43 — all pre-existing `pytest.approx` flags).
No golden fixture impact — every golden cert carries an int
`main_fuel_type`.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Re-runs of a First Run batch re-save a property's data; that must replace,
not duplicate (ADR-0012 idempotent batch writes).
- `EpcPostgresRepository.save` deletes the property's existing EPC graph
(parent + all child tables, floor-dims via their building parts) before
inserting, when a `property_id` is given. Anonymous saves still insert.
- `BaselinePostgresRepository.save` deletes the existing row for the
`property_id` before inserting — no more unique-constraint violation on
re-save; also what the re-score-on-override path needs.
- Solar already upserts, so it's unchanged.
The #1129 round-trip fidelity test stays green (delete-first is a no-op on
a first save). 2 new tests (re-save replaces, not duplicates). pyright
strict clean; AAA.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
First slice of the per-stage batch-transaction refactor (ADR-0012). A
UnitOfWork is the single transaction a stage runs its batch in: a context
manager exposing the DB repos bound to one session, committing once on
`commit()` and rolling back on exception or exit-without-commit
(all-or-nothing per batch, fail noisily).
- `UnitOfWork` (port): `property` / `epc` / `solar` / `baseline` repos +
`commit()` / `rollback()`; `__exit__` rolls back uncommitted work.
- `PostgresUnitOfWork(session_factory)`: opens a Session from an injected
factory (a module-scoped engine + sessionmaker in prod, so the pool is
reused across warm invocations), binds the Postgres repos to it, closes
on exit.
Not yet wired into any orchestrator — that lands in the Baseline /
Ingestion refactor slices. 3 tests against ephemeral PG (commit durable
across units; exception rolls back; no-commit persists nothing). pyright
strict clean; AAA.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The published RdSAP 10 Specification 10-06-2025 PDF Table 32 (p.95)
lists heating oil at 7.64 p/kWh. Two independent operational sources
both use 5.44 p/kWh for the same fuel:
- Elmhurst P960 worksheets across all five oil-fired variants in
`sap worksheets/heating systems examples/` (oil 1, oil pcdb 1/2/3,
pcdb 1) lodge 5.4400 p/kWh on (240) "Space heating - main system 1"
and (247) "Water heating (other fuel)" for every "FuelType: Heating
oil" worksheet.
- The gov.uk EPC register's lodging software back-solves to ~5.48
p/kWh from cert 0240-0200-5706-2365-8010's lodged SAP 73 (oil + PV
detached, age J). With heating-oil at 5.44 in the cascade this cert
closes to ΔSAP = 0 exactly against its lodged value.
The BRE technical papers (`docs/specs/sap10 technical papers/`) carry
no Table 32 errata or fuel-price update, so the change is grounded in
empirical cross-source evidence rather than a spec citation — the
worksheet PDF is the source of truth per the project convention.
Post-flip residuals:
Heating-systems corpus (cascade − worksheet ΔSAP_c):
oil 1 −9.7030 → +2.6578
oil pcdb 1 −11.6343 → +0.4239 ← within 1 SAP of closure
oil pcdb 2 −11.6343 → +0.4239
oil pcdb 3 −10.8674 → +1.1597
pcdb 1 −9.4083 → +6.9521 ← largest remaining oil-cohort gap
Golden fixtures (cascade − lodged SAP):
0240-0200-5706-2365-8010 resid −10 → +0 ← EXACT closure
0390-2954-3640-2196-4175 resid −6 → +7 ← oil-price bug was
masking +13 SAP of
opposite-direction
cascade gaps; now
exposed for follow-up
PE / CO2 residuals are unaffected by the unit-price flip (cost-only
change). The 41-variant corpus regression guard (S0380.129) holds; all
other golden cohorts pass unchanged. Extended handover suite: 874 pass.
Bio-FAME (code 73) shows the inverse divergence on oil 3/4 worksheets
(worksheet 7.64 vs spec 5.44 — possible row-swap typo in the spec PDF)
but flipping it has no measurable cascade effect today, so deferred
until a cert that exercises it surfaces.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Captures the heating-systems corpus closure work, the new permanent
residual-pin regression test, and the queued S0380.131 candidate
(heating-oil unit price spec-vs-worksheet divergence).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Elmhurst Summary §14.0 Main Heating1 leaves "Fuel Type" empty for
Table 4b liquid-fuel boilers (heating oil / HVO / FAME / B30K /
bioethanol — SAP codes 120-141). Unlike gas boilers (codes 101-119)
where Elmhurst explicitly lodges "Mains gas", liquid-fuel boilers
take the fuel from §15.0 "Water Heating Fuel Type" since the same
boiler heats space + water.
Pre-slice:
- `_elmhurst_main_fuel_int(mh.fuel_type)` returned None for the
empty §14.0 fuel string.
- The electric-SAP-code inference (`_ELECTRIC_SAP_MAIN_HEATING_CODES`)
didn't fire because SAP 127 is a Table 4b oil boiler, not electric.
- `main_fuel_type` fell through to the raw empty string.
- `cert_to_inputs._main_fuel_code` returned None.
- `table_32.unit_price_p_per_kwh(None)` defaulted to mains gas
(3.48 p/kWh).
- The cascade therefore priced ~13.7k kWh/yr of oil space + water
heating at the gas tariff — a 56% under-count vs the worksheet's
Table 32 oil rate.
Two complementary fixes:
1. Add "Heating oil" → 28 ("oil (not community)" per epc_codes.csv
row main_fuel,28) to `_ELMHURST_MAIN_FUEL_TO_SAP10`. The existing
`API_FUEL_TO_TABLE_32` then routes API 28 → Table 32 code 4
(heating oil — 7.64 p/kWh / 0.298 kg CO2/kWh / 1.180 PE factor
per RdSAP 10 spec p.95). This fix handles pcdb 1 directly because
pcdb 1 lodges §14.0 "Fuel Type: Heating oil" explicitly.
2. Thread a §15.0-fuel fallback for the main_fuel inference: when
`mh.fuel_type` is empty AND `mh.main_heating_sap_code` is in the
Table 4b liquid-fuel range (120-141 per SAP 10.2 Table 4b
"Seasonal efficiency for gas and liquid fuel boilers"), use the
§15.0 water_heating_fuel as the main fuel too. Gated on the SAP
code range so this can't accidentally fire on solid-fuel-mains
+ electric-HW certs (where §15.0 lodges "Electricity" for the
immersion but the SH fuel is the solid fuel implicit in the SAP
code). This fix handles oil 1 + oil pcdb 1/2/3 (where §14.0 is
silent but §15.0 lodges "Heating oil").
Residual shifts at HEAD post-slice (5 variants legitimately re-pinned):
oil 1 +13.67 SAP → -9.70 SAP (cascade now over-counts at the
spec's 7.64 p/kWh — vs worksheet's 5.44)
oil pcdb 1/2 +11.17 → -11.63
oil pcdb 3 +11.87 → -10.87
pcdb 1 +21.90 → -9.41
Remaining negative residuals are the price-spec-vs-worksheet gap
queued for slice S0380.131 (5.44 vs 7.64 p/kWh oil). The mapper now
correctly identifies the fuel; what's left is the cascade tariff.
The other 36 corpus variants are unchanged — restricting the §15.0
fallback to SAP 120-141 keeps solid-fuel-mains and electric-mains
certs at their existing pins.
Extended handover suite at HEAD post-slice: **874 pass, 0 fail**
(was 873 + 1 new AAA test).
Pyright net-zero on touched files (45 → 45 — pre-existing errors
unrelated).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The 001431 corpus at `sap worksheets/heating systems examples/` now
has a permanent test module pinning cascade-vs-worksheet residuals
across all 41 populated heating-system variants. The corpus is a
controlled-variable test set — same dwelling (semi-detached, TFA 90 m²,
age G, W6 9BF, Elmhurst P960 worksheet format) under different heating
configurations — so every cascade-vs-worksheet residual is fully
attributable to the heating subsystem.
`test_heating_systems_corpus_residual_matches_pin` is parametrised by
variant folder name. Per variant it:
1. Extracts Block 11a (individual) or Block 11b (community) pins
from the P960 PDF — continuous SAP (`SAP value` row), total fuel
cost (255)/(355), CO2 (272/372/382/383), PE (286/386/486/483).
2. Routes the Summary PDF through ElmhurstSiteNotesExtractor →
EpcPropertyDataMapper.from_elmhurst_site_notes → cert_to_inputs
→ calculate_sap_from_inputs.
3. Asserts each of the four cascade outputs sits within an absolute
tolerance of the pinned residual.
Tolerances are tight (SAP ±0.001, cost ±£0.01, CO2 ±0.1 kg/yr, PE
±0.1 kWh/yr) — the *expected residual* moves toward 0 as heating-
cascade gaps close; the *tolerance* never widens. Per
[[feedback-zero-error-strict]] + [[feedback-golden-residuals-near-zero]].
Pins captured at HEAD `729ee29c` (post-S0380.128). All 41 pass.
Smallest residual: `solid fuel 8` +0.87 SAP / −£20 cost (closest to
closure). First negative ΔSAP: `community heating 6` −6.87 SAP / +£158
cost (heat-pump heat network — only variant where cascade UNDERshoots
the worksheet).
Extended handover suite at HEAD post-slice: **873 pass, 0 fail**
(was 832 + 41 new parametrised cases).
Pyright net-zero on new file (0 → 0).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Elmhurst Summary §14.0 Main Heating1 normally closes at "14.1 Main
Heating2", but community-heated dwellings and "no system" certs lodge
§14.0 followed directly by "14.1 Community Heating/Heat Network" (no
second main system exists on a community-heated dwelling). Pre-slice
the extractor's `_between("14.0 Main Heating1", "14.1 Main Heating2")`
returned an empty string for these shapes — every §14.0 field
(including `Main Heating SAP Code`) came back None, then the mapper
strict-raised `UnmappedElmhurstLabel` with "§14.0 Main Heating1 has
neither PCDF boiler reference (None) nor SAP code (None)".
The fix adds a `_section_lines_first_end(start, ends)` helper that
accepts a tuple of end-marker candidates and uses whichever appears
first after `start`. `_extract_main_heating` now closes §14.0 at
either "14.1 Main Heating2" or "14.1 Community Heating" — whichever
Summary lodges.
Impact on heating-systems corpus 001431 at `sap worksheets/heating
systems examples/`:
Variant Pre-S0380.128 -> Post-S0380.128
------------------------ ------------------ -----------------
community heating 1 mapper-raise -> SAP code 301 OK
community heating 2 mapper-raise -> SAP code 302 OK
community heating 3 mapper-raise -> SAP code 304 OK
community heating 4 mapper-raise -> SAP code 302 OK
community heating 6 mapper-raise -> SAP code 302 OK
no system mapper-raise -> SAP code 699 OK
Corpus tally: **35/41 -> 41/41 cascade-OK**. With all populated
variants now executing, the cascade-vs-worksheet residual cluster is
fully visible for the first time. Notably community heating 6 surfaces
the FIRST negative ΔSAP in the corpus (-6.87 — cascade undershooting
the worksheet rather than overshooting), a distinct diagnostic shape
worth investigating next.
The fix is structural (extractor section bracketing) — no spec rule
to cite. RdSAP 10 §17 page 85 row 1.0 ("Main Heating") + §17 row
10-1a ("Community Heat Source") confirm that community-heated certs
have only one main heating system (no Main 2 block).
Extended handover suite at HEAD post-slice: **832 pass, 0 fail**
(was 831 + 1 new AAA test).
Pyright net-zero on touched files (13 → 13 — pre-existing errors
unrelated).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Elmhurst Summary §15.1 sometimes lodges "Cylinder Size: No Access" (the
inaccessible-cylinder lodging form). Pre-slice the mapper strict-raised
`UnmappedElmhurstLabel` because `_ELMHURST_CYLINDER_SIZE_LABEL_TO_SAP10`
only carried the three lodged-size labels (Normal/Medium/Large).
Per RdSAP 10 Specification Table 28 page 55 ("Cylinder size"):
> "Inaccessible:
> - if off-peak electric dual immersion: 210 litres
> - if from solid fuel boiler: 160 litres
> - otherwise: 110 litres"
And per §10.5.1 page 53:
> "An electric immersion is assumed dual in the following cases:
> - cylinder is inaccessible and electricity tariff is dual"
So the 210-L "off-peak electric dual immersion" branch fires automatically
when both (a) cylinder is inaccessible AND (b) water heating is electric
AND (c) meter type is dual / off-peak (no separate dual-immersion lodging
required).
New helper `_resolve_elmhurst_inaccessible_cylinder_size` keys off
§15.0 "Water Heating Fuel Type" + §14.2 "Electricity meter type":
- solid fuel water heating fuel (Anthracite, House coal, Wood, etc.)
→ 160 L → SAP10 cylinder_size enum 3 (Medium)
- "Electricity" + dual/18-hour/24-hour/off-peak meter
→ 210 L → SAP10 cylinder_size enum 4 (Large)
- otherwise → 110 L → SAP10 cylinder_size enum 2 (Normal)
`_elmhurst_cylinder_size_code` extended with optional water_heating_fuel
+ meter_type kwargs; the single call site at line 4459 threads
`survey.water_heating.water_heating_fuel_type` and
`survey.meters.electricity_meter_type`.
Property 001431 (the heating-systems corpus dwelling) lodges `pcdb 1`
with §14.0 Potterton oil boiler (PCDF 716) + §15.0 "Water Heating Fuel
Type: Heating oil" + §14.2 "Electricity meter type: 18 Hour" — water
fuel is oil (not electric, not solid fuel) → "otherwise" branch → 110 L
→ enum 2 (Normal). `pcdb 1` now cascade-executes (corpus tally 34 → 35
OK / 41 populated).
Extended handover suite at HEAD post-slice: **831 pass, 0 fail**
(was 830 + 1 new AAA test).
Pyright net-zero on touched files (45 → 45 — pre-existing errors
unrelated).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Elmhurst Summary §14.0 Main Heating1 sometimes lodges the bare form
"Heat Emitter: Underfloor Heating" without a subtype qualifier (in
screed / timber floor). The mapper's `_ELMHURST_HEAT_EMITTER_TO_SAP10`
dict only carried the qualified forms, so the bare lodging fell through
to None and was passed as a raw string into `MainHeatingDetail.heat_
emitter_type` — causing `_responsiveness` to strict-raise
`UnmappedSapCode` on every cert with this lodging (2 variants on the
heating-systems corpus: `electric 1` + `oil 6`).
Per RdSAP 10 Specification §10.11 Table 29 page 56 ("Heating and hot
water parameters"):
> "Underfloor heating: If dwelling has a ground floor, then according
> to the floor construction (see Table 19 if unknown):
> - solid, main property age band A to E: concrete slab
> - solid, main property age band F to M: in screed
> - suspended timber: in timber floor
> - suspended, not timber: in screed
> Otherwise (i.e. upper floor flats), take floor as suspended"
New helper `_resolve_elmhurst_underfloor_subtype` keys off the main BP's
`floor.floor_type` + `construction_age_band` and returns:
- SAP10.2 Table 4d emitter code 2 (in screed) → R=0.75 — for
solid + age F-M, suspended-not-timber, and upper-floor-flat cases
- SAP10.2 Table 4d emitter code 3 (timber floor) → R=1.0 — for
suspended-timber
The solid + age A-E "concrete slab" branch (R=0.25) has no cert-side
enum entry yet, so the helper strict-raises `UnmappedElmhurstLabel`
when that combination lands — the next variant lodging an A-E solid
underfloor will surface the gap loudly per
[[reference-unmapped-sap-code]].
Property 001431 (the heating-systems corpus dwelling) lodges §9.0
"Type: S Solid" + §3.0 "Date Built: G 1983-1990" (age band G ∈ F-M)
→ "in screed" → code 2 → R=0.75. Both `electric 1` and `oil 6` now
cascade-execute (corpus tally 32 → 34 OK / 41 populated).
Extended handover suite at HEAD post-slice: **830 pass, 0 fail**
(was 829 + 1 new AAA test).
Pyright net-zero on touched files (45 → 45 — pre-existing errors
unrelated).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The Elmhurst Summary §14.2 Meters section lodges the electricity meter
type as the bare RdSAP enum form "18 Hour", but `_METER_STR_TO_INT`
only carried the legacy "off-peak 18 hour" alias. All 41 P960-format
heating-system fixtures at `sap worksheets/heating systems examples/`
lodge meter_type "18 Hour", so `cert_to_inputs` strict-raised on every
one of them before this slice.
Per RdSAP 10 Specification §17 page 85 (Electricity meter row 10-2):
> "Electricity meter: Dual/single/10-hour/18-hour/24-hour/unknown"
Per RdSAP 10 §12 page 62:
> "if the meter is dual 18-hour/24-hour it is 18-hour/24-hour tariff"
So the bare "18 Hour" lodging routes directly to enum 5 (Off-peak 18
hour) → `Tariff.EIGHTEEN_HOUR`, bypassing the §12 Rules 1-4 dispatch
(which only fires for Dual meters that aren't 18-hour or 24-hour).
After this slice the heating-system corpus probe (`/tmp/probe_*.py`
across 41 variants of the same property × different heating systems)
shifts from "32 raises + 7 mapper gaps + 2 emitter gaps" to
"32 cascade-OK + 7 community-heating + 2 underfloor-emitter + 1
cylinder-size 'No Access'". The 32 newly-OK variants surface a
positive ΔSAP cluster (cascade SAP_c > worksheet SAP_c by +0.87..+30
across boiler types) — that residual layer is queued for the next
slice.
Extended handover suite at HEAD post-slice: **829 pass, 0 fail**
(baseline 775 + test_table_12a.py's 54 incl. the new "18 Hour" entry).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Completes the First Run spine. Replaces the #1130 stub FirstRunPipeline
with the real three-stage composition and wires it into the handler.
- `FirstRunPipeline.run(command)` sequences Ingestion → Baseline →
Modelling, threading **only** `property_ids` between stages (and
`scenario_ids` into Modelling, off the command — never a prior stage's
output). Stages are injected behind thin `IngestionStage` /
`BaselineStage` / `ModellingStage` Protocols (the EpcFetcher/SolarFetcher
idiom), so the handler owns wiring and tests substitute fakes (ADR-0011).
- `ModellingOrchestrator` stub + `ScenarioRepository` / `MaterialsRepository`
seam ports — `run(property_ids, scenario_ids)` reads through repos, does
no scoring yet. Method shapes deferred to the Modelling per-service grills
(Scenario / Scenario Phase / Snapshot / Optimised Package / Plans are rich
— not pre-empted here).
- Handler delegates to the real pipeline via `build_first_run_pipeline`
(Postgres-backed repos off the session). The Ingestion source clients
(EPC API / Google Solar / geospatial S3) are isolated behind one
`_source_clients_from_env` seam that raises until the deploy/Terraform
config settles — out of scope for this slice. Subtask complete/failed +
CloudWatch URL still come from `@subtask_handler`.
Integration test (the criterion's centrepiece): wires REAL Ingestion +
REAL Baseline + stub Modelling through a shared fake EPC repo, with a
repo-backed PropertyRepo composing the Property from that slice. Proves
Baseline reads the very EPC Ingestion persisted — the through-repos
hand-off, no in-memory coupling. Plus a composition test pinning stage
order + only-property_ids threading.
TDD, one test → one impl. pyright strict clean; AAA layout. 116 pass in
the tests/ tree, no regressions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`test_ne_and_nw_share_table_u5_constants` asserts NE == NW, E == W,
SE == SW orientation-pairs share the same flux value per Appendix U
Table U5's column-sharing convention. The cascade looks up both via
the same dictionary key — the values are bit-identical, not
approximately equal. Tightened from `pytest.approx(..., abs=0.01)` to
exact `==` equality; abs=0.01 masked the fact that the cascade
returns the same float object.
Net pyright: unchanged.
Tests: 17 pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
17 hand-crafted ventilation tests had abs=0.001-0.01 tolerances that
masked the actual diff (always 0 or 1e-16 for these direct-arithmetic
formulas). Tightened to abs=1e-12 (essentially exact).
10 cohort cert pins (`LINE_8`/`LINE_10`/.../`LINE_25` against U985 PDF)
had mixed abs=0.0001-0.0005; standardised to abs=1e-4 (PDF 4-d.p.
display floor per [[feedback-e2e-validation-philosophy]]). The looser
0.0005 pins on (8), (16), (18), (21), (22b), (25) admitted up to half
a 4-d.p. unit of drift that the cascade isn't producing — actual
cascade diffs are ~5e-5 (one notch under display precision).
Test movement: all 26 tests pass at the new tolerances. Net pyright
change: 69 → 69.
Per [[feedback-zero-error-strict]] tolerance widening is forbidden;
this slice goes the other way — every pin tightened to its actual
precision floor.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Stage 2 of First Run. Establishes each Property's Baseline Performance
from persisted source data and writes it back — reads only from repos,
never a Fetcher or HTTP (ADR-0003), so it is byte-identical whether
Ingestion ran milliseconds ago or last week.
Domain (`domain/baseline/`):
- `Performance` VO — the four rated quantities: SAP / EPC Band / CO2 /
Primary Energy Intensity. `lodged_performance(epc)` reads them off the
EPC's recorded fields (PEUI = `energy_consumption_current`).
- `BaselinePerformance` (ADR-0004) — the paired `lodged` + `effective`
Performance + `rebaseline_reason`, plus the no-derivation part of the
energy block (`space_heating_kwh` / `water_heating_kwh`, off the RHI,
deterministic per ADR-0006). Both halves always populated.
- `Rebaseliner` port + `StubRebaseliner`: the re-score-on-override seam
(ADR-0011). SAP10 certs pass through (effective == lodged, reason
"none"); a pre-SAP10 cert raises `RebaselineNotImplemented` rather
than fabricating a plausible-but-wrong "none" — ML rebaselining is not
wired yet. Mirrors the repo's strict-raise culture.
Persistence: new `BaselineRepository` port + `BaselinePostgresRepository`
+ flat-column `baseline_performance` SQLModel (one row per Property). Per
ADR-0004's amendment this is a standalone table, NOT columns on the
retiring `property_details_epc`. Production migration is FE-owned
(Drizzle) — docs/migrations/baseline-performance-table.md.
Docs (grill-with-docs): corrected CONTEXT.md Lodged/Effective Performance
to Primary Energy Intensity (the term collided with its own _Avoid_ entry
under "heat demand") + fixed stale RHI field names; amended ADR-0004
Consequences for the standalone-table decision.
Fuel split + bills (rest of EPC Energy Derivation) deferred to a
follow-up — they need a Fuel Rates source (Ofgem-cap ETL) that does not
exist yet.
TDD, one test -> one impl: 7 tests (lodged read, rebaseliner pass-through
+ raise, orchestrator establish-and-persist + pre-SAP10 raise, Postgres
round-trip + absent). pyright strict clean; AAA layout.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The API mapper's `_API_FLOOR_CONSTRUCTION_TO_STR` dispatch covered
codes 1 and 2 only. Basement smoke-test fixture
`fixtures/basement/0712-3058-2202-3816-8204.json` lodges code 4 on
two BPs (paired with `floor_insulation=0` and global floor
descriptions "Solid" + "Solid, no insulation (assumed)"). Per the
[[reference-unmapped-api-code]] strict-raise pattern, that surfaced
as `UnmappedApiCode: floor_construction code: 4` on
`test_real_corpus_basement_cert_has_part_with_has_basement_true`.
Code 4 is the no-insulation solid-floor variant — semantically a
solid floor. The cascade's `u_floor` only distinguishes "Suspended"
prefix from everything-else (solid-branch is the fall-through), so
the additional code maps to the same "Solid" string as code 1.
Test movement: `test_real_corpus_basement_cert_has_part_with_has_basement_true`
→ PASS. No SAP/PE/CO2 cascade behaviour changes (the smoke test
only asserts basement detection from the alt-wall code).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
RdSAP 10 §5.11.4 (PDF p.44):
"If retrofit insulation present of unknown thickness use 50 mm."
The cascade encoded "unknown thickness" via the cert's "NI" (Not-
Indicated) sentinel which `_parse_thickness_mm` collapses to int(0).
But that conflates two structurally different signals:
(a) explicit int(0) — `_api_resolve_sloping_ceiling_thickness`
returns this for cert 001479 Ext2 PS sloping ceiling age C, a
per-BP "uninsulated" override of the dwelling-level description
("Pitched, insulated" from another BP).
(b) string "NI" — the cert lodgement marker for "thickness not
indicated; defer to description"; §5.11.4 should fire when the
description carries an "insulated" signal.
Pre-slice the heat_transmission cascade dropped `roof_description`
whenever `roof_thickness == 0`, killing the §5.11.4 path in `u_roof`
(line 711) for the (b) case. 346 corpus certs lodge the NI +
"insulated (assumed)" pattern per the §5.11.4 test's arrange comment.
Fix: inspect the raw `part.roof_insulation_thickness` value (pre-
parse) — drop the description only when the lodgement is the literal
int(0), keep it for the "NI" string sentinel so `u_roof`'s §5.11.4
branch fires (`_described_as_insulated` + thickness=0 → return 0.68).
Test movement:
test_roof_insulated_assumed_with_ni_thickness_uses_50mm_per_section_5_11_4 → PASS
test_summary_001479_full_chain_sap_matches_worksheet_pdf_exactly → PASS (cohort safe)
cert 000565 e2e — 11/11 PASS (unaffected — explicit per-BP thicknesses)
Golden corpus impact: cert 0240 had this exact pattern (BP[1] NI + global
description includes "Pitched, insulated (assumed)"). The fix drops its
roof U from 2.30 → 0.68 for that BP, closing massive mapper-gap residuals:
expected_sap_resid: -14 → -10 (Δ +4 SAP)
expected_pe_resid_kwh_per_m2: +12.49 → +0.054 (Δ −12.43 kWh/m²)
expected_co2_resid_tonnes_per_yr: +0.696 → +0.063 (Δ −0.633 t/yr)
Re-pinned per [[feedback-golden-residuals-near-zero]]: "Re-pin to the
new (smaller) value when a gap closes". The remaining 0240 residuals
(SAP -10 / PE +0.05 / CO2 +0.06) are tiny — the bulk of 0240's mapper
gap is now closed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The §5 test EPC builder threaded sap_windows from the fixture but
discarded `sap_roof_windows` — passing them through `make_minimal_sap10
_epc(...)`. Pre-S0380.110 the `_daylight_factor_from_cert` cascade
read a single aggregate `rooflight_total_area_m2` kwarg + bulk g_L,
so the test EPC builder's omission was masked. Post-S0380.110 the
cascade reads per-rooflight glazing via `epc.sap_roof_windows`
(Appendix L §L2a per-window g_L sum) — Triple / Double / Single
distinctions matter.
For cohort 000516 (the only cohort fixture with a lodged rooflight,
a Double-glazed 1.18 m² × g_L=0.80 × FF=0.70 × Z_L=1.0), the empty
sap_roof_windows on the test EPC undercut the daylight factor →
cascade lighting (67) Jan 33.78 W vs ws 32.68 W (+1.1 W/month) →
lighting_kwh_per_yr 238.65 vs ws 230.88 (+7.77 kWh/yr).
Fix: thread `fixture.build_epc().sap_roof_windows` through the
minimal EPC. Cohorts 000474/477/480/487/490 have no rooflights →
list is None → cascade unchanged for those certs.
Test movement: 000516 (67) Jan 33.78 → 32.68 ✓ EXACT. 000516
lighting_kwh_per_yr 238.65 → 230.88 ✓ EXACT.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two changes bundled (same file, same RdSAP 10 §15 spec citation):
1. Tighten cohort cert (000474 / 000490) heat_transmission LINE_xx
pins from abs=0.01 / 0.1 → abs=1e-4 (4 pins). Pre-slice the cohort
landed at 1e-4 of the U985 PDF but the test pins were holdovers
from when the cascade was less precise. Per [[feedback-e2e-
validation-philosophy]]:
"per-component tests pin against U985 worksheet line refs at
<1e-3 tolerance ... 1e-4 since PDF lodges 4 d.p."
Probe data at HEAD post-§15:
000474 LINE_33 cascade=209.108439 ws=209.1084 Δ=+4e-5
000474 LINE_37 cascade=232.116939 ws=232.1169 Δ=+4e-5
000490 LINE_33 cascade=211.893610 ws=211.8936 Δ=+1e-5
000490 LINE_37 cascade=236.621110 ws=236.6211 Δ=+1e-5
2. Update `test_room_in_roof_simplified_type_1` and `..._type_2`
expected-value formulas to round A_RR_shell to 2 d.p. per RdSAP
10 §15 (p.66) — matching the cascade behaviour now enforced by
Slice S0380.116. The unrounded expected was 100.9156 / 71.857;
spec-correct rounded is 100.919 (39.5285 → 39.53) and 71.846
(32.2749 → 32.27). Same abs=1e-4 pin enforces both arithmetic
and rounding correctness.
New import: `_round_half_up` from heat_transmission (the same
helper the cascade uses for §15 rounding).
Net pyright change: 71 → 71. Net test change: 4 newly-tight pins,
2 newly-passing RR synthetic tests, 670 → 670 passing.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Stage-2 entry point for the First Run use case. Adds the
`ara_first_run` Lambda package mirroring the `postcode_splitter`
template, its typed trigger contract, and a stub `FirstRunPipeline`.
- `AraFirstRunTriggerBody`: thin command of five fields — `task_id`,
`sub_task_id` (UUID, lifecycle), `portfolio_id`, `property_ids`,
`scenario_ids` (int business IDs). No `model_config` override, so
Pydantic's default `extra="ignore"` lets the FastAPI backend add
fields without breaking deployed lambdas. UPRNs / Scenario defs are
deliberately off the event — read from source-of-truth tables.
- Thin `handler.py`: validate-and-delegate only, via a named
`dispatch_first_run` seam (testable without the Lambda runtime).
Subtask status (in-progress/complete/failed) + CloudWatch log URL
come for free from the existing `@subtask_handler()` decorator.
- `FirstRunPipeline` (orchestration/) stub: `run(command)` receives the
validated command. Declares a structural `FirstRunCommand` Protocol
(the three business fields) that `AraFirstRunTriggerBody` satisfies,
so orchestration needs no application-layer import — rhymes with the
`EpcFetcher`/`SolarFetcher` Protocols on IngestionOrchestrator
(ADR-0011). Full Ingestion→Baseline→Modelling composition lands in
#1136.
- Dockerfile / requirements.txt / local_handler/ mirror postcode_splitter.
TDD: 7 new tests (trigger-body validation incl. forward-compat +
id-types, pipeline seam, handler delegation). pyright strict clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Slice S0380.116 rounded `A_RR_shell = 12.5 × √(A_RR_floor / 1.5)` to
2 d.p. per RdSAP 10 §15 (p.66). Two certs in the golden corpus have
RR-driven cascade paths that fire this rounding:
0240 (TFA 118, age J, RR on BP[0]): PE +12.4941 → +12.4933
6035 (TFA 128, age A, RR + gas combi): PE +46.0936 → +46.0952
CO2 deltas on both are sub-1e-4 (display-precision noise) so those
pins stay. All 51 cohort-2 certs are unchanged — their A_RR_shell
paths either bypass the Simplified branch (Detailed RR with
`slope`/`flat_ceiling` roof lodgements) or have no RR.
Per [[feedback-golden-residuals-near-zero]] re-pin to track new
cascade output rather than absorb the drift into the test tolerance.
The ±0.01 PE / ±0.001 CO2 absolute tolerances on the pin stay; what
changes is the expected residual value.
Test still passes at ±0.0000 drift on all 53 certs post-repin.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
RdSAP 10 Specification §15 "Rounding of data" (PDF p.66):
"For consistency of application, after expanding the RdSAP data into
SAP data using the rules in this Appendix, the data are rounded
before being passed to the SAP calculator. The rounding rules are:
U-values: 2 d.p.
All element areas (gross) including window areas and conservatory
wall area: 2 d.p."
The §3.9.1 / §3.10.1 shell formula A_RR_shell = 12.5 × √(A_RR_floor /
1.5) produces a gross element area for the room-in-roof. Pre-slice the
cascade kept the raw float (e.g. cert 000565 BP[0]: 12.5 × √30 =
68.46532...), then subtracted lodged wall surfaces to obtain the (30)
residual roof area. The worksheet rounds A_RR_shell to 2 d.p. (68.47)
BEFORE the subtraction — per §15 above.
Cert 000565 has three BPs that fire this path (Main, Ext1, Ext3 — all
have detailed wall surfaces with no `slope` / `flat_ceiling` /
`stud_wall` lodgement, so §3.10.1 residual fires). Each contributes a
sub-rounding residual that the unrounded cascade was missing:
BP[0] Main: 68.4653 → 68.47; residual 43.9653 → 43.97 (+0.0016 W/K)
BP[1] Ext1: 59.5119 → 59.51; residual 18.2519 → 18.25 (−0.0007 W/K)
BP[3] Ext3: 57.7350 → 57.74; residual 17.3450 → 17.35 (+0.0017 W/K)
Movement (HEAD `d0268a5b` → this slice) for cert 000565:
roof_w_per_k 51.3768 → 51.3795 ✓ EXACT (Δ −0.0027 → 0.0)
thermal_bridging 128.6448 → 128.6460 ✓ EXACT (Δ −0.0012 → 0.0)
total_external_a 857.6323 → 857.6400 ✓ EXACT (Δ −0.0077 → 0.0)
space_heating_kwh 59008.2363 → 59008.3499 ✓ EXACT (Δ −0.1136 → 0.0)
main_fuel_kwh 34710.7272 → 34710.7941 ✓ EXACT (Δ −0.0669 → 0.0)
total_fuel_cost 4680.2515 → 4680.2593 ✓ EXACT (Δ −0.0078 → 0.0)
co2_kg_per_yr 6447.6161 → 6447.6263 ✓ EXACT (Δ −0.0102 → 0.0)
sap_score_cont 28.5087 → 28.5087 ✓ EXACT (Δ +4.2e-5 → −4.7e-5)
sap_score (int) 29 ✓ EXACT (preserved)
ecf 5.38682 → 5.38683 (vs ws 5.3868, Δ +3.2e-5)
Cert 000565 truly closes — every SAP-result field within 1e-4 of the
worksheet PDF.
Cohort safety: 6 cohort certs (000474..000516) unchanged — cohort
000516's roof routes through the Detailed branch with `slope` /
`flat_ceiling` / `stud_wall` lodgements, so `has_roof_lodgement=True`
short-circuits the §3.10.1 residual block. Cohort certs 000474/477/
480/487/490 are pre-S0380.103 hand-built fixtures whose RR fields don't
exercise the simplified A_RR_shell path (rir.floor_area=0 or
detailed_surfaces only).
Test added: `test_summary_000565_a_rr_shell_rounded_2_dp_closes_roof_
w_per_k_per_rdsap_10_section_15` pins the cascade roof_w_per_k = 51.3795
exactly (Δ ≤ 1e-4 vs worksheet (30) Σ).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The cert 000565 ECF pin was a transcription typo. U985-0001-000565.pdf
line 593 (Block 1, 11a SAP rating individual heating systems) reads:
Energy cost factor (ECF) [(255) x (256)] / [(4) + 45.0] = 5.3868 (257)
The pin captured 5.3866 — likely a mis-copy from line 871 / 873 (Nov
MIT (92)m = 15.3866). The cascade output 5.386823 matches the worksheet
PDF at 4 d.p.; the pin was always 0.0002 wrong against the source.
Per [[feedback-verify-handover-claims]], handover narratives are verified
against the source PDF; the cascade is correct and the pin was wrong.
Test movement: `test_sap_result_pin[000565-ecf]` now passes (diff
0.000023 against the corrected pin 5.3868, within abs=1e-4). Four
expected fails remain (cost / CO2 / SH / main_fuel) — closed in the
next slice (A_RR_shell rounding per RdSAP 10 §15).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Stage 1 of the pipeline: per property, read its UPRN from the property row,
fetch its EPC, resolve coordinates from the Geospatial reference repo, thread
those into the Solar fetcher, and persist EPC + solar via repos. Fetchers never
call each other — the orchestrator threads the coordinate (ADR-0011). Coordinates
are reference data (deterministic from UPRN), resolved transiently to drive the
solar fetch rather than persisted per-property.
Depends on thin EpcFetcher/SolarFetcher Protocols (EpcClientService and
GoogleSolarApiClient satisfy them structurally). Unit-tested against fakes — no
DB, gov API, or network: persists EPC, threads coords into solar, skips
UPRN-less properties and skips solar when coordinates are absent. pyright clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add Coordinates value object + GeospatialRepository port + GeospatialS3Repository
adapter. Resolves a Property's lon/lat from the partitioned Ordnance Survey
Open-UPRN parquet (filename_meta -> partition -> UPRN row). A Repo, not a
Fetcher (ADR-0011): no live OS API call. The parquet reader is injected, so it's
unit-tested against fixture parquets with no S3/network; returns None when the
UPRN is uncovered or absent. pyright strict clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Five spec-cited slices closed cert 000565 from continuous SAP
Δ = -0.0059 → +0.000042 (within user 1e-4 tolerance):
- S0380.110: per-rooflight g_L via Appendix L §L2a
- S0380.111: roof-window inclination adj via Table 6e Note 2
- S0380.112: per-BP rooflight deduction via RdSAP §3.7
- S0380.113: H=0 gable retention via RdSAP §3.9.2 step (b)
- S0380.114: pump GAIN for HP+boiler via Table 5a Note a)
Handover documents the two parallel workstreams the next agent
should tackle:
1. Final sweep for TRULY exact continuous SAP on cert 000565
(close the remaining sub-1e-4 cost/CO2/SH/fuel/ECF residuals)
2. Tighten golden test residuals across the corpus per
[[feedback-golden-residuals-near-zero]]
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
SAP 10.2 Table 5a (PDF p.177) verbatim:
"Central heating pump in heated space, 2013 or later: 3 W"
Note a): "Where there are two main heating systems serving
different parts of the dwelling, assume each has its own
circulation pump and therefore include two figures from this
table. ... Set to zero in summer months. **Not applicable for
electric heat pumps from database.** Where two main systems serve
the same space a single pump is assumed."
The Note a) "not applicable for electric heat pumps" rule zeros the
pump GAIN only for HP-category systems themselves. Where a cert
lodges a non-HP main system alongside an HP, the non-HP system's
circulation pump still operates and dissipates 3/7/10 W into the
dwelling as an internal gain.
Pre-slice the cascade conflated TWO different spec rules:
Table 4f (ELECTRICITY) — HP pump electricity is in the COP, so
worksheet line 230b = 0 for HP certs.
Table 5a (GAIN) — HP-from-database pump gain is omitted
ONLY for that HP system, not for any
non-HP system in the same cert.
`_main_heating_category_from_cert(epc)` returned `details[0].
main_heating_category` and the caller zeroed pump_w whenever that
was category 4. This dropped the 3 W gain for any cert whose first
main system was an HP — even when system 2 was a non-HP boiler with
its own pump.
Cert 000565 lodges TWO main systems:
[0] HP (category 4) pump_age "2013 or later"
[1] Gas boiler (category 2) pump_age None
Per spec the system [1] gas boiler's pump contributes 3 W (post-2013
date from [0]'s lodgement). Worksheet (70) confirms:
Pumps, fans 3.0 3.0 3.0 3.0 3.0 0.0 0.0 0.0 0.0 3.0 3.0 3.0 (70)
Pre-slice cascade returned 0 every month, missing 24 W·months of
winter internal gains. Downstream: +10 kWh space heating, +£0.71
fuel cost, +0.90 kg CO2, -0.008 continuous SAP.
Cert 0380 (cohort-1 ASHP, HP-only):
[0] HP (category 4) pump_age unknown
(no [1])
Worksheet (70) = 0 every month. Cascade post-slice: every main
system is HP → pump_w = 0 ✓ unchanged.
Fix:
`domain/sap10_calculator/worksheet/internal_gains.py`:
- Replace `_main_heating_category_from_cert` + the {4} set-membership
check with `_all_main_systems_are_heat_pumps(epc)`. Returns True
iff every lodged `main_heating_details[i].main_heating_category`
equals 4. Pump gain is zeroed only in that case.
- Existing `_pump_date_category_from_cert` (reads [0]'s pump_age)
unchanged — Elmhurst lodges the dwelling's pump_age on detail[0]
regardless of which system the pump serves.
Cohort safety: all 6 cohort certs have a single main system (gas
boiler, category 2) → `all_main_systems_are_heat_pumps` returns
False → pump_w applies, same as the prior `else` branch. Cert 0380
(ASHP) has a single HP main → True → pump_w = 0, unchanged.
Cert 000565 cascade snapshot (HEAD 59de805e → this):
(70)m pumps_fans gain [0]*12 → [3,3,3,3,3,0,0,0,0,3,3,3] ✓ EXACT
sap_score (int) 29 ✓ EXACT (preserved)
sap_score_continuous 28.5007 → 28.508742 (Δ -0.0080 → +0.000042)
**← essentially exact at 4.2e-5**
ecf 5.3876 → 5.386823 (Δ +0.0010 → +0.0002)
total_fuel_cost_gbp 4680.97 → 4680.2515 (Δ +0.71 → -0.008)
co2_kg_per_yr 6448.53 → 6447.6161 (Δ +0.90 → -0.010)
space_heating_kwh 59018.52 → 59008.2363 (Δ +10.17 → -0.114)
main_heating_fuel 34716.78 → 34710.7272 (Δ +5.98 → -0.067)
**Cert 000565 continuous SAP now exact at 1e-4 tolerance.** Every
intermediate (66-73, 83-84, 93-98, fuel/cost/CO2) closes the
worksheet at ≤1e-3 relative error.
Pyright net-zero (17 → 17 errors across touched files).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Move the EpcClientService package (client + _retry + exceptions + tests) from
the dying backend/ tree to infrastructure/epc_client/ as the New-EPC-API Fetcher;
update the two callers (address2UPRN, a script). All 14 client tests pass.
Add SolarRepository port + SolarPostgresRepository persisting Google Solar
building insights as JSONB (solar_building_insights table), one row per Property.
The EPC repo half of this slice already landed in #1129. pyright strict clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add the Ara modelling aggregate root (ADR-0002): domain/property/ with
PropertyIdentity, SiteNotes, Property, Properties. Property.source_path
implements the two disjoint source paths + Recency Tie-Break (ADR-0001;
survey wins on an equal date); effective_epc resolves to the surveyed data
(Site Notes path) or the public EPC (epc_with_overlay path — Landlord
Overrides overlay is a later slice). Pure dataclasses, no infrastructure imports.
PropertyRepository port + PropertyPostgresRepository hydrate the aggregate
whole from a defensive view of the FE-owned 'property' table (identity columns)
plus the EPC slice via EpcRepository.get_for_property. Reads only from repos
(ADR-0003). 8 domain + 1 hydration test; pyright strict clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add epc_renewable_heat_incentive table (space_heating_kwh, water_heating_kwh +
the three insulation-impact kWh fields), wired into EpcPostgresRepository
save/get. This is the P0 gap: RenewableHeatIncentive carries the baseline
space-heating/hot-water kWh that EPC Energy Derivation consumes.
The round-trip test now asserts full deep-equality (dropped the
renewable_heat_incentive exclusion) and passes for RdSAP 21.0.0 + 21.0.1.
DB migration for the new table documented in
docs/migrations/epc-property-round-trip-fidelity.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Relocate EpcPropertyModel + child tables from the dying backend/ tree to
infrastructure/postgres/epc_property_table.py (re-export shim keeps
documents_parser working). Add EpcRepository port + EpcPostgresRepository with
a full reverse mapper (epc_property tables -> EpcPropertyData).
Round-trip test surfaced two fidelity gaps:
1. Union[int,str] SAP code fields were str()-coerced on save, losing the int
(API) vs str (Site Notes) distinction. Now stored as JSONB (type-preserving).
2. The schema was a partial projection. Closed the cheap gaps on the model
(heating shower/bath counts, roof_construction_type, curtain_wall_age,
addendum, mechanical_vent_duct_insulation_level, SAP 10.2 §2 ventilation
fields + a ventilation_present flag). Structural gaps tracked as follow-ups;
renewable_heat_incentive (P0, #1137) excluded from the assertion until landed.
Round-trip passes for RdSAP-Schema-21.0.0 and 21.0.1; pyright strict clean.
Migration inventory for the DB: docs/migrations/epc-property-round-trip-fidelity.md
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Records the grill-with-docs outcomes for the ara_first_run rebuild: three
composable stage orchestrators (Ingestion/Baseline/Modelling), one lambda per
use case chaining them through repos (not in-memory), and the Fetcher-vs-Repo
data-source taxonomy. Amends ADR-0003's chaining rule to generalise beyond
RefreshOrchestrator. Adds the pipeline-composition + First Run vocabulary to
CONTEXT.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
RdSAP 10 §3.9.2 step (b) (PDF p.23) verbatim:
"Software calculates the area of each gable or adjacent wall by
using the equation:
A_RR_gable = L_gable × (0.25 + H_gable) − [(H_gable − H_common_1)² / 2
+ (H_gable − H_common_2)² / 2]"
Step (d):
A_RR_final = A_RR_wall − (Σ A_common + Σ A_gable + Σ A_party
+ Σ A_sheltered + Σ A_connected)
The spec equation is signed and applies for all L > 0 — including
H_gable = 0. When the gable is shorter than the common walls the
correction term `(H_gable − H_common)² / 2` exceeds the
L × (0.25 + H_gable) term, producing a negative A_RR_gable.
Elmhurst's worksheet evaluates the equation literally; the negative
value adjusts A_RR_final upward via step (d) without billing a
physical wall area.
Cert 000565 §8.1 lodges Ext3's RR (Simplified Type 2) with an
absent Gable Wall 2:
Gable Wall 1 L=9.00 H=7.00 Exposed U=0.45
Gable Wall 2 L=4.00 H=0.00 U=0.00 ← lodged but H=0
Common Wall 1 L=5.00 H=1.50 U=0.45
Common Wall 2 L=7.50 H=0.30 U=0.45
Spec equation for Gable Wall 2:
A_gable_2 = 4 × (0.25 + 0) − (0 − 1.5)²/2 − (0 − 0.30)²/2
= 1.0 − 1.125 − 0.045 = −0.17 m²
Worksheet (30) Ext3 residual = 17.35 m² back-solves exactly:
A_RR_shell = 12.5 × √(32.0 / 1.5) = 57.7350
Σ walls (incl. -0.17 absent gable) = 40.3850
residual = shell − walls = 17.3500 ✓ 4 d.p.
Pre-slice the mapper had two clamps that together dropped the
spec-computed −0.17 m² adjustment:
mapper.py:3350 `if length_m <= 0 or height_m <= 0: return None`
→ filtered out any H=0 surface
mapper.py:3443 `area_m2 = max(0.0, length_m * (0.25 + H) − correction)`
→ clamped negative gable areas at 0
Combined the cascade computed residual = 17.18 m² (cascade UNDER
by 0.17). Plus a related secondary `if height_m > h` filter on the
correction sum that masked the all-common-walls-taller case.
3-layer fix:
1. `datatypes/epc/domain/mapper.py` `_map_elmhurst_rir_surface`:
- Split the early-return filter: drop only when L<=0 (no wall),
OR when H<=0 AND not (Simplified Type 2 with common walls).
- Apply the spec gable-area formula to BOTH `gable_wall` (party
default) and `gable_wall_external` kinds in Simplified Type 2
(the U-value routing differs by kind, but the area equation
is the same).
- Remove `max(0.0, ...)` clamp so the signed result reaches the
cascade.
- Remove `if height_m > h` correction-sum filter (spec applies
the full square unconditionally).
2. `domain/sap10_calculator/worksheet/heat_transmission.py` per-
surface loop:
- `gable_wall` branch: skip `party += 0.25 × area` when area < 0
(wall doesn't exist physically) but still add the signed area
to `rr_walls_in_a_rr_area` so the residual deduction in step (d)
grows by |area|.
- `gable_wall_external` branch: same skip pattern for `walls +=
u × area` and `rr_detailed_area += area`.
Cohort safety: only cert 000565 Ext3 hits this in the corpus. All
other cohort certs are Type 1 RR (no common walls, formula gives
the same answer) or have all gables H > 0. The cascade's per-element
test pins (Ext1's Connected gable + Exposed gable, Ext4's Detailed
RR) unchanged.
Cert 000565 cascade snapshot (HEAD a461b70d → this):
roof_w_per_k 51.3185 → 51.3768 ✓ EXACT (Δ -0.06 → -0.003)
total_external_area 857.46 → 857.6323 ✓ EXACT (Δ -0.18 → -0.008)
thermal_bridging 128.62 → 128.6448 ✓ EXACT (Δ -0.03 → -0.005)
total_w_per_k 936.97 → 937.0563 ✓ EXACT (Δ -0.09 → -0.004)
sap_score (int) 29 ✓ EXACT (preserved)
sap_score_continuous 28.5027 → 28.5007 (Δ -0.0060 → -0.0080)
ecf 5.3877 → 5.3876
total_fuel_cost_gbp 4681.01 → 4680.97
co2_kg_per_yr 6448.59 → 6448.53
space_heating_kwh 59019.21 → 59018.52
main_heating_fuel 34715.31 → 34716.78
**Cert 000565 fabric cascade now essentially exact** (HTC −0.004 W/K
total residual across all 8 fabric components). The remaining
continuous SAP -0.0080 / cost +£0.71 / SH +10 kWh residuals come
from non-fabric upstream (likely ventilation or appliances) —
candidates for a future audit.
Pyright net-zero (57 → 57 errors across touched files).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
