Three more pre-existing failures (present at 9ee38211, before this branch's
recent commits; same family as the orchestration multi-measure re-pin) —
golden-cert plan expectations that predate the ASHP generator (ADR-0025)
and the optimiser folding forced dependencies into candidate gain (ADR-0016):
- test_console: a multi-measure plan now leads with air_source_heat_pump,
not cavity_wall_insulation (which is dropped — its forced ventilation makes
the pair net-negative). Assert a measure actually in the package.
- test_report 0330: package is now {solid_floor_insulation, air_source_heat_
pump}; cavity_wall + forced mechanical_ventilation correctly excluded.
- test_report 0036: gain-maximising package is now {solid_floor_insulation,
low_energy_lighting}.
Same verified-correct optimiser evolution as 077e3a39 (cavity_wall +2.9 SAP
alone but its forced fabric→ventilation dep drags the pair net-negative).
Re-pin to the actual packages + their trigger fields; the forced wall→vent
edge stays covered by test_measure_dependency / test_optimiser.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The optimiser-package expectation was stale: it predated the optimiser
folding a triggered measure's forced dependency into its candidate gain
(ADR-0016). The run considers ALL measures (considered_measures defaults
to None — no restriction), so once the ASHP bundle became SAP-beneficial
(ADR-0025) the gain-maximising package shifted.
Verified the new package is CORRECT, not a regression: on the test EPC,
cavity-wall insulation earns +2.9 SAP alone but its forced fabric→
ventilation dependency (ADR-0016) drags the wall+ventilation pair to a
NET −1.8 SAP (−0.9 on top of the ASHP package), so the gain-maximising
Optimiser correctly excludes the wall and its forced ventilation. Update
the expected set to {air_source_heat_pump, suspended_floor_insulation,
low_energy_lighting, secondary_heating_removal} and drop the wall/vent-
specific assertions — the forced wall→ventilation edge is covered by
test_measure_dependency / test_optimiser; this integration test keeps its
end-to-end optimise→persist→telescope coverage on the chosen package.
Pre-existing failure (present before this branch's recent commits), outside
the handover regression gate.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The realistic re-generation of case 37 (code-117 gas boiler, control 2102,
+ a MAINS-GAS condensing gas-fire secondary code 611, vs case 37's biogas
605). The full extractor -> mapper -> calculator pipeline reproduces the
worksheet's SAP-rating block EXACTLY: continuous SAP 60.9152 (Δ 2e-5) and
(272) CO2 5801.0770 (Δ ~0). This confirms the boiler-efficiency /
control-2102 −5pp interlock / secondary-fuel handling are all correct, and
that case 37's +7 gap was purely the biogas sub-fuel the Summary export
cannot carry.
Summary mirrored into backend/documents_parser/tests/fixtures so the pin
runs without the unstaged workspace. PE not pinned — it is a separate
DPER block (different scope) already guarded by the corpus PE gauge.
Worksheet harness 47/47 unchanged; pyright net-zero.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The HHR-storage HeatingOverlay (ADR-0024) added an off-peak electric
immersion cylinder but never set `immersion_heating_type`, so the overlaid
cert left it None. The calculator then could not resolve `immersion_single`
for the SAP 10.2 Table 13 HW high-rate split and billed hot water 100% at
the off-peak low rate — £127.41 vs the relodged after-cert's £169.39,
overstating the overlay's SAP by +1.26 (CO2/PE matched, isolating it to the
HW cost path).
Add `immersion_heating_type` to HeatingOverlay, route it through
`_fold_heating` (it lives on `sap_heating`), and set it to 1 (single
off-peak immersion) on the HHR overlay to match the relodged reference.
Closes both `test_hhr_storage_overlay_reproduces_the_relodged_after_*`
cascade pins (electric-storage and no-system befores share the after).
Pre-existing failure (present before this branch's recent commits), outside
the handover regression gate. Full modelling suite 220 pass, pyright net-
zero.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`_synthesise_20_0_0_sap_windows` passed `schema.multiple_glazing_type`
straight into `_api_cascade_glazing_type`, which raised UnmappedApiCode on
the "ND" (Not Defined) string that the 20.0.0 corpus lodges alongside the
1-8 integer codes — failing the mapper-coverage guard on every ND-glazed
20.0.0 cert. Mirror the existing 18.0/19.0/17.x seams: route integer codes
through the cascade, fall the "ND" string back to the DG-modal default
(cascade code 2 → daylight g_L 0.80). Also corrects the 20.0.0 schema
field type `int` → `Union[int, str]` to match the data (as 18.0 already
does), which keeps the isinstance guard pyright-clean.
Pre-existing failure (present before this branch's recent commits), not in
the handover regression gate. Fixes all 15 RdSAP-Schema-20.0.0 ND certs;
test_mapper_corpus 6002/6002 pass. pyright net-zero.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Completes `_elmhurst_secondary_fuel_from_sap_code` per SAP 10.2 §12
(PDF p.34: "Secondary heating systems and applicable fuel types are taken
from the room heaters section of Table 4a") + RdSAP 10 §10.4.1. Each
Table 4a room-heater code now resolves to its fuel CATEGORY's modal fuel:
- gas room heaters 601-613 → mains gas (26 → Table 32 1, 3.48 p/kWh)
- liquid room heaters 621-625 → heating oil (28 → Table 32 4, 5.44 p/kWh)
- solid room heaters 631-636 → house coal (11 → Table 32 11, 3.67 p/kWh)
- electric room htrs 691-694/699/701 → None (cascade electricity default)
Previously only the gas (601-613→26) and solid (631-634→11) blocks were
mapped; liquid heaters (621-625) and 635-636 fell through to None →
silently billed as electricity (13.19 p/kWh), a large mis-price for an
oil/solid heater. The prior slice raised on those; this maps them to the
correct category fuel instead, and keeps the raise ONLY for codes inside
the room-heater range (601-701) that are not a recognised Table 4a row.
The specific sub-fuel within a category (mains gas vs LPG vs biogas) is a
SEPARATE lodgement per §10.4.1 and is NOT exported in the Summary, so the
gas block stays the modal mains gas — worksheet "simulated case 37" lodged
its 605 live-effect fire on biogas (7.60 p/kWh), unrecoverable from the
Summary code alone (this is the entire +7 SAP case-37 gap: secondary
energy £131 + a separate biogas standing charge £70; every other line
matches the worksheet exactly, incl. (206) main efficiency 61%).
5 AAA tests, harness 47/47 (0 raised), pyright net-zero, regression clean,
corpus gauge unchanged (Elmhurst-path only).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Elmhurst Summary lodges only the secondary heating SAP code (Table 4a
Category 10), never its fuel. `_elmhurst_secondary_fuel_from_sap_code`
mapped the gas block (601-613 → mains gas) and solid block (631-634 →
house coal) to their modal defaults, but returned None for any OTHER
Category-10 code — and None makes the cascade SILENTLY bill the secondary
as electricity (13.19 p/kWh). For a fuel-fired heater (e.g. 621-625
liquid-fuel oil/bioethanol) that is a large, invisible mis-price.
Per the UnmappedElmhurstLabel strict-raise pattern (mirrors the wall_type
/ glazing label raises), a fuel-fired Category-10 code (601-699) outside
the mapped gas/solid blocks now RAISES instead of guessing. Electric room
heaters (691-699) keep returning None — electricity IS their fuel.
The gas block 601-613 still resolves to the modal default mains gas: the
Summary cannot distinguish mains gas from LPG/biogas, so an LPG or biogas
live-effect fire (worksheet "simulated case 37" used biogas at 7.60 p/kWh
vs our 3.48 p/kWh mains-gas default, a +7 SAP gap) is not recoverable from
the Summary export — that is a data-availability limit, not a guess we can
fix here. This commit closes the genuinely-silent-wrong path; the gas
sub-fuel remains the documented modal default.
Worksheet harness 47/47, 0 raised. 3 AAA tests, pyright net-zero,
regression clean, corpus gauge unchanged (Elmhurst-path only; the API path
lodges the secondary fuel explicitly).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
End-to-end API-path regression pin for the battery behaviour validated by
the user-simulated Elmhurst worksheet pair (cert 001431 "simulated case
35/36", 5 kWh, export-capable, mains-gas, standard tariff). The official
SAP rating ("10a. Fuel costs - using Table 12 prices") values PV used-in-
dwelling and PV exported identically at 13.19 p/kWh (export code 60 ==
import code 30, ADR-0010), so a battery only redistributes PV between two
equally-priced lines: worksheet PV credit (252) = -455.6458 and SAP (258)
= 88.0859 are IDENTICAL with/without the battery (ΔSAP = 0).
Two tests over the committed RdSAP-21.0.1 corpus:
- standard tariff (meter 2): toggling the battery holds continuous SAP
EXACTLY constant, while at least one cert's primary energy DOES respond
(proving the App-M1 §3c β-split is wired, not a dropped battery).
- off-peak tariff (meter != 2): the battery STRICTLY raises SAP, because
self-consumed PV displaces high-rate import (15.29) above the 13.19
export credit — confirming the standard-tariff neutrality is a price
coincidence, not a no-op.
Guards table_32 export price (code 60) and the battery β-split against
silent regression. Complements the unit-level β tests in
test_photovoltaic.py.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A "No system present: electric heaters assumed" lodging carries SAP
Table 4a code 699 (electric room heaters) but RdSAP main_heating_category
1, NOT 10. `_table_12a_system_for_main` keyed the direct-acting-electric
routing on category==10 only, so the category-1 form fell through to None
and `_space_heating_fuel_cost_gbp_per_kwh` billed space heating 100% at
the off-peak LOW rate — as if direct-acting room heaters charged overnight
like storage.
Per RdSAP 10 §12 Rule 3 (PDF p.62) electric room heaters (691-694, 699)
route to the 10-hour tariff, and SAP 10.2 Table 12a Grid 1 (PDF p.191)
gives the "other direct-acting electric" row a 0.50 high-rate fraction at
10-hour (1.00 at 7-hour). Route those SAP codes — the same set §12 Rule 3
already uses — to OTHER_DIRECT_ACTING_ELECTRIC alongside the category-10
gate.
Found via the PE/CO2-vs-cost split on the worst over-rater in the /tmp
sample: cert 2958 PE +0% / CO2 -1% (energy correct) but SAP +32.2 — a
pure cost-side bug. Space rate 7.50 -> 11.09 p/kWh; cert 2958 +32.2 ->
+14.7. The committed corpus gauge is unchanged (its 3 non-category-10
code-699 certs are all on Single meters -> STANDARD tariff, so this split
never applies to them); the win is on the unbiased /tmp population's
single worst cert.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SAP 10.2 Appendix M1 (PDF p.94): "EPV,ex,m = 0 if the PV system is not
connected to an export-capable meter." The cascade computed the β-split
export stream regardless of `is_dwelling_export_capable`, so a non-export-
capable dwelling was credited the full PV export — in the §10a COST it
credits at the Table 32 import rate (13.19 p/kWh), which dominates the rating.
On 7 Wybourn Terrace S2 5BJ the PE (144 vs lodged 151) and CO2 (27 vs 29)
already matched, yet the phantom export cost credit pulled SAP from ~73 to
92.1 (+19). Zero `epv_exported_monthly_kwh` after the Appendix-G4 diverter
adjustment when not export-capable; the onsite (EPV,dw) consumption and the
diverter HW reduction are unchanged.
Not-export-capable PV cohort (corpus, 4 certs): 7 Wybourn +19.1 -> +6.5,
4 Lime Ave +11.1 -> +0.4, 8 Hatherleigh +7.6 -> -0.2, Flat 5 ~-0.4. Gauge
66.1% -> 66.9%, MAE 1.124 -> 1.039. Floor 0.64 -> 0.65 / ceiling 1.18 -> 1.08.
Worksheet harness 47/47 0 diverge (Summary certs carry export-capable meters).
1 AAA test, pyright net-zero. Found by auditing the worst over-rater without a
worksheet: PE/CO2-match + cost-miss localised it to the PV export credit.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
RdSAP 10 §10.5 (PDF p.55): "If the actual size is not determined, the size of
a hot-water cylinder is taken as according to Table 28." When a cylinder is
present (has_hot_water_cylinder) but no size descriptor resolves — the gov API
lodges cylinder_size=0, or Exact with no measured volume — `_hot_water_
cylinder_volume_l` returned None, silently dropping BOTH the cylinder's
storage loss and the Table 13 electric-DHW high-rate fraction, under-costing
and over-rating the dwelling. Default such cylinders to the Table 28 baseline
"Normal" 110 L (the value §10.7 also instantiates as the first-row default).
The context-dependent Inaccessible 210/160 values are deliberately NOT applied
here — they are tied to the explicit "Inaccessible" descriptor (code 5) the
assessor lodges, not to an unpopulated size field.
Scope: 7 of 301 cylinder certs in the corpus (2%). Correctness fix — closes a
real spec gap; marginal on the headline (within-0.5 66.1% unchanged, MAE
1.128 -> 1.124) because these certs' residual is dominated by a separate HW-
demand gap, not the cylinder. Worksheet harness 47/47 0 diverge (Summary certs
lodge a real size, so the fallback never fires). 1 AAA test, pyright net-zero.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SAP 10.2 Table 12a Grid 1 (PDF p.191): electric storage heater SAP code 408
is an "Integrated (storage + direct-acting) system" with a 0.20 space-heating
high-rate fraction on a 7-hour tariff — NOT the 0.00 of "other storage
heaters". `_table_12a_system_for_main` returned None for all storage codes (an
explicit TODO), so code 408 fell back to the 100%-low-rate path and billed
space heating at the bare 7-hour low rate (5.50 p/kWh) — under-costing →
over-rating. Mapped cat-7 storage: 408 -> INTEGRATED_STORAGE_DIRECT (0.20),
others -> OTHER_STORAGE_HEATERS (0.00, unchanged behaviour). The enum +
fraction rows already existed; this only wires the dispatch, so the split
flows self-consistently to both the §10a cost and the Appendix-M1 D_PV
high-rate fraction.
Corpus: sap408 over-raters +14.6/+12.9/+12.7 -> +7.1/+5.1/+3.4 (two crossed
into within-0.5). Gauge 65.9% -> 66.1%, MAE 1.160 -> 1.128. Floor 0.63 -> 0.64
/ MAE ceiling 1.22 -> 1.18. Worksheet harness 47/47 0 diverge. The residual
+3..+7 is the "all other uses" 0.90 high-rate fraction (lighting/pumps/HW
still billed 100%-low on the off-peak legacy path) — the next slice.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SAP 10.2 Table 4c(3) (PDF p.169) "Factor for controls and charging method"
multiplies a heat network's heat requirement by 1.05-1.10 for FLAT-RATE
charging (note d: household pays a fixed amount regardless of heat used, so
no incentive to economise), and by 1.0 for charging linked to use. The
worksheet folds it into the heat-network requirement alongside the Table 12c
distribution loss factor:
(307) space = (98c) x (302) x (305) x (306)
(310) DHW = (64) x (305a) x (306)
Our cascade applied (306) DLF but never (305)/(305a), so every flat-rate
community-heating cert under-counted demand -> over-rated SAP.
Folded the factor into the 1/DLF efficiency override at the space-heating
(206) and DHW (water-inherits-from-main) sites. Space column adds +0.05 for
no thermostatic control (2301/2302); DHW column is 1.05 flat-rate / 1.0
linked-to-use.
Corpus (RdSAP-21.0.1, 1000 certs): community cluster median +0.32 -> -0.19,
within-0.5 38% -> 62% (control 2307 +0.83 -> -0.19; 2306 unchanged at factor
1.0 as spec requires). Overall gauge 65.0% -> 65.9%, MAE 1.174 -> 1.160.
Ratcheted the corpus-test floor 0.62 -> 0.63 / MAE ceiling 1.25 -> 1.22.
Also records (corpus-test comment + scripts/decompose_co2_pe_error.py) the
disproof of the prior "CO2/PE +5% is a factor/scope bug" lead: factors are
spec-exact, scope identical, and the bias is per-cert demand fidelity
(corr(SAP-err, PE-diff) = -0.54), not a one-slice factor fix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a committed integration test driving the full API path — raw gov-EPC
response → from_api_response → cert_to_inputs → calculate_sap_from_inputs —
across all 1000 certs in the in-repo RdSAP-21.0.1 corpus, and pins the
aggregate accuracy of our continuous SAP (plus CO2 and primary energy)
against each cert's lodged figures. Mirrors scripts/eval_api_sap_accuracy.py
but runs in CI off the committed corpus (~2s, no /tmp sample needed).
Scoped to RdSAP-21.0.1 — the SAP 10.2-era schema whose lodged rating uses the
same methodology we compute (a fair target). Pre-SAP10 schemas (17.x-20.0.0)
lodge SAP 2012 ratings and are out of scope (guarded for mapping only by
test_mapper_corpus.py).
Current: SAP within-0.5 = 65.0%, MAE = 1.174 (tight floor/ceiling — the
optimised gauge). CO2 MAE = 0.27 t/yr (bias +0.17) and PE MAE = 14.6
kWh/m2/yr (bias +8.9) are reported + loosely guarded: cost is well-calibrated
but CO2/PE both run ~+5-10% high (uniform across fuels — a systematic
CO2/PE-factor or scope gap, not yet investigated). Thresholds ratchet as
slices tighten each metric.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The mixer-shower hot-water demand (worksheet 42a) divided N_shower by the
count of MIXER outlets only. But SAP 10.2 Appendix J step 1a is explicit:
"Establish how many shower outlets are present in the dwelling, Noutlets
(including in the count any instantaneous electric showers)" — and the
electric-shower step (64a) uses that same Noutlets from step 1a. So a
dwelling with both a mixer and an electric shower assigned the FULL N_shower
to the mixer system AND billed the electric shower on top of it, double-
counting shower demand → over-counted main HW → under-rated the dwelling.
Fix: thread the electric-shower count into the mixer demand so the
denominator is the total outlet count (mixer + electric), iterating the
warm-water draw over the mixer outlets only (per step 1e).
shower_types=1,2 cohort: -0.37 median -> +0.28 (crossed zero); API gauge
68.4% -> 69.0% within-0.5. Golden cert 0300-2747 (1 mixer + 1 electric)
re-pinned: PE +0.93 -> -0.10, CO2 +0.25 -> +0.15 (both toward zero,
confirming the double-count). Worksheet harness 47/47, 0 divergers (the
Elmhurst fixtures have no electric showers).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The no-PCDB MEV fan-electricity path fed the SAP 10.2 Table 4g default SFP
(0.8 W/(l/s)) directly as SFPav. But Table 4g note 3 (PDF p.176) is explicit:
the default SFP values "are to be multiplied by the appropriate in-use factor
for default data from the PCDB" — PCDB Table 329 system_type 10 ("default
data, used when SFP is taken from Table 4g rather than the PCDB"), IUF 2.5
(duct-agnostic per note 2). Table 4h, which previously held these factors, is
retired ("no longer used – data now stored in the PCDB").
Omitting the IUF under-billed the index-less MEV fan electricity by 2.5x
(SFPav 0.8 instead of 0.8 x 2.5 = 2.0), so cost was too low and the cohort
over-rated. This is distinct from the with-index path, which already applies
the tested-product system_type-2 "no scheme" IUF (~1.45) per fan.
Index-less gas-house MEV cohort: +1.37 median -> -0.18 (12% -> 92% within 0.5),
no overshoot — the missing IUF was exactly the over-rate. API gauge 67.7% ->
68.4% within-0.5 (mean|err| 0.992 -> 0.986, signed +0.031 -> +0.006).
Worksheet harness 47/47, 0 divergers (Summary-path MEV certs carry a PCDB
index or are natural, so unaffected).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
