The gov-EPC API mapper sets BOTH roof_construction (int) and
roof_construction_type (str, derived via _API_ROOF_CONSTRUCTION_TO_STR),
but the Elmhurst mapper set only the string — leaving roof_construction
None on every site-notes cert. The SAP cascade reads the STRING (so SAP
cross-mapper parity always held), but consumers of the int (e.g.
domain/sap10_ml/transform.py ML aggregates `main_dwelling_roof_
construction`) silently saw None on the Elmhurst path.
New `_elmhurst_roof_construction_int` maps the Elmhurst roof-type code to
the same SAP10 int the API lodges (F→1, PN→3, PA→4, PS→8, S/A→7),
harvested from the committed Summary fixtures. Unlike the wall map it
returns None (not a strict-raise) for unmapped codes: the int is not
cascade-load-bearing, so an unknown roof must not block the cert (vaulted
5 / thatched 6 / NR omitted until a fixture surfaces them).
The 6 hand-built U985 reference fixtures gain the matching
roof_construction int (4/4/3 etc.) so test_from_elmhurst_site_notes_
matches_hand_built_* still asserts structural parity. SAP output is
unchanged (cascade reads the string). §4 suite green (2407 passed); the
two pre-existing stone-§5.6 sap10_ml failures are unrelated/out of scope.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
RdSAP 10 §12 (PDF p.62) Dual-meter dispatch: "the choice between 7-hour
and 10-hour is made by the main heating type ... if the main system is a
direct-acting electric boiler (191), or electric room heaters ... it is
10-hour tariff." The electric room-heater codes — Table 4a 691 (panel/
convector/radiant), 692 (fan), 693 (portable), 694 (water-/oil-filled),
699 (assumed) — were missing from `_RULE_3_TEN_HOUR_CODES` (the long-
standing TODO there), so a Dual-meter room-heater cert fell through to
Rule 4 (7-hour default).
Compounded with S0380.230 (which routes room heaters to Table 12a
OTHER_DIRECT_ACTING_ELECTRIC): at 7-hour the high-rate fraction is 1.00
(all at 15.29 p), but at the correct 10-hour it is 0.50 split over the
10-hour rates (14.68 / 7.50 p) → blended ~11 p. Without this fix .230
over-charged and flipped the cluster from over- to under-rating.
1,000-cert 2026 API sample: cat-10 mean |err| 7.11 → 5.26, signed mean
+5.08 → -0.86 (now balanced, 22 over / 26 under — the systematic
directional bias is gone). Overall mean |err| 2.16 → 2.04. Full §4 suite
green (2406 passed).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SAP 10.2 Table 12a Grid 1 (PDF p.191): an electric room heater (RdSAP
main_heating_category 10, e.g. SAP code 691) is direct-acting electric,
so it sits on the "Other systems including direct-acting electric" row —
7-hour high-rate fraction 1.00, 10-hour 0.50. It runs on demand, mostly
at the HIGH rate; it does NOT earn the 100%-low-rate of overnight storage
charging (which is category 7).
`_table_12a_system_for_main` only mapped ASHP, so an electric room heater
fell through to the "100% low-rate" fallback (5.50 p, £0.0550), under-
charging space heating by ~9.79 p/kWh and systematically OVER-rating the
cluster. Now maps electric cat-10 mains to OTHER_DIRECT_ACTING_ELECTRIC
(gated on `_is_electric_main`, so gas/solid-fuel cat-10 room heaters are
excluded). The same Table 12a fraction flows through cost, CO2 (Table
12d) and PE (Table 12e) — all three callers already pre-gate on electric.
Mirror of S0380.228 (same fallback bug for electric SECONDARY heating).
1,000-cert 2026 API sample (no worksheet for this cluster — ±0.5-vs-lodged
fallback bar): cat-10 mean |err| 9.49 → 7.11, %<0.5 10.4% → 16.7%;
headline %<0.5 42.5% → 42.9%, overall mean |err| 2.29 → 2.16. cat-7
(storage) and cat-2 (gas) unchanged. Full §4 suite green (2405 passed).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Flags the SY/B disambiguation change and the field-vs-property merge
landmine (raises AttributeError at first EpcPropertyData instantiation,
not at import; git merges silently) for the feature/bill-derivation
reviewer, with the recommended reconciliation and the strict-xfail
tripwire they own.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
RdSAP10 `wall_construction == 6` is canonically WALL_SYSTEM_BUILT — a
WALL TYPE — but the gov-EPC basement heuristic hijacked it: Elmhurst
lodges both "SY System build" and "B Basement wall" as code 6, and the
API lodges basements as code 6 too, so a system-built wall was
mis-flagged `main_wall_is_basement` → wrong RdSAP §5.17 / Table 23
u_basement_wall/u_basement_floor overrides, and downstream the solid-wall
Recommendation Generator couldn't offer EWI/IWI on system-built walls.
System-built stays the wall type on its canonical code 6; the basement
signal moves OFF code 6 to a dedicated `is_basement` (SapAlternativeWall)
/ `wall_is_basement` (SapBuildingPart) Optional[bool] flag:
- Elmhurst: `_elmhurst_wall_is_basement` sets it from the distinct
"SY"/"B" labels (False for SY, True for B, None otherwise).
- gov-EPC API: per-wall code 6 can't be told apart at lodging time, so
`from_api_response` post-processes via `_clear_basement_flag_when_
system_built` — when the cert addendum marks the dwelling system-built,
the code-6 basement heuristic is cleared. A genuine basement (no
addendum signal) keeps the code-6 fallback.
- `main_wall_is_basement` / `is_basement_wall` honour the flag when set,
else fall back to the code-6 heuristic — so untouched API basements and
the cert 000565 "B" cohort are unchanged.
`EpcPropertyData.system_build` is a derived property over the wall type:
the MAIN wall is system-built iff `wall_construction == 6` and it is not
flagged basement. System-built lives on `wall_construction`; the basement
attribute is separate.
Acceptance: a system-built main wall (Elmhurst SY, or API addendum
system_build) → wall_construction == 6, main_wall_is_basement is False,
system_build is True; a genuine basement main wall → main_wall_is_basement
is True, system_build is False. Full §4 suite green (2404 passed).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
RdSAP10 `wall_construction == 6` is canonically WALL_SYSTEM_BUILT, but
the gov-EPC basement heuristic hijacked it: Elmhurst lodges both "SY
System build" and "B Basement wall" as code 6, so a system-built wall
was mis-flagged `main_wall_is_basement` and routed to the RdSAP §5.17
`u_basement_wall` override instead of the system-built U-value table.
System-built stays on its canonical code 6; the basement signal moves
to an explicit `is_basement` (SapAlternativeWall) / `wall_is_basement`
(SapBuildingPart) Optional[bool] flag, set by the Elmhurst mapper from
the distinct "SY"/"B" codes via `_elmhurst_wall_is_basement` (True for
B, False for SY, None otherwise). The `main_wall_is_basement` /
`is_basement_wall` properties honour the flag when set and fall back to
the gov-EPC API code-6 heuristic when None — so the API path (basement
lodged as integer 6, no flag) and the cert 000565 "B" cohort are
unchanged.
Acceptance (a recommendation-summary generator depends on it): a
system-built MAIN wall reports wall_construction == 6 AND
main_wall_is_basement is False; a genuine basement main wall still
reports main_wall_is_basement is True.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SAP 10.2 Table 3 (PDF p.160) row 1: primary circuit loss applies when
"hot water is heated by a heat generator (e.g. boiler) connected to a
hot water storage vessel via insulated or uninsulated pipes". The Table
4a hot-water-only codes (PDF p.166) 911 gas / 912 liquid / 913 solid
boiler-circulator + 921-931 range cooker with boiler are each a heat
generator feeding the cylinder through a primary loop.
`_primary_loss_applies` keyed only off the resolved DHW `main` — but for
these certs `_water_heating_main` returns the SPACE main (e.g. electric
storage heaters, SAP code 402, which has no primary loop), so every
boiler branch missed the gas water-boiler's primary circuit and (59)m
went to zero. New branch keys off `water_heating_code` ∈
`_WATER_HEATING_BOILER_CIRCULATOR_CODES`. 941 (electric HP for water
only) is excluded — HP DHW vessels follow the Table 3 integral-vessel
rules.
Simulated case 19 (electric storage main + WHS 911 + 210 L cylinder):
(62)m total HW demand 2493.30 → 3169.98 kWh/yr, matching the worksheet
(the missing 676.68 kWh/yr = the worksheet's (59) primary-loss annual
sum, h=5/p=0). The remaining (64)/(219) gap is the PV diverter (63b),
deferred to its own slice.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SAP 10.2 Table 12a Grid 1 (PDF p.191): secondary heating is a direct-
acting electric room heater (RdSAP 10 §A.2.2 default), on the "Other
systems including direct-acting electric" row — 7-hour high-rate fraction
1.00, 10-hour 0.50. A room heater runs on demand, mostly at the high
rate; it does NOT earn the 100%-low-rate of overnight storage charging.
`_secondary_fuel_cost_gbp_per_kwh` previously returned the flat off-peak
LOW rate (5.50 p, £0.0550) for every off-peak electric secondary, under-
charging by 9.79 p/kWh. New `_secondary_off_peak_rate_gbp_per_kwh` mirrors
`_space_heating_fuel_cost_gbp_per_kwh`: it blends the Table 12a high-rate
fraction (OTHER_DIRECT_ACTING_ELECTRIC) against the Table 32 high/low
rates, with the 18-/24-hour fallback to the low rate.
Simulated case 19 (electric storage main + electric secondary, Dual/7-hour
meter) is the worksheet case (242): "Space heating - secondary
(1.00*15.29 + 0.00*5.50)" → 15.29 p/kWh = £0.1529. This was the primary
cat-7-cluster cost driver: total cost 1485.68 → 1835.53 (worksheet
1816.58), SAP cont 60.11 → 50.67 (worksheet ~51.22). Remaining +19 cost
is HW/space-heating kWh (next slices).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SAP 10.2 Table 2b note b (PDF p.159) applies the ×0.9 temperature-factor
reduction only when DHW is "separately timed" relative to space heating
on a SHARED heat generator ("boiler systems, warm air systems and heat
pump systems"). Per RdSAP 10 §10.5.1 (PDF p.55) a separate boiler/
circulator providing DHW only (water-heating code 911 = "Gas boiler/
circulator for water heating only") is NOT the main space-heating system
— so there is no shared timer to apply the ×0.9 against. `_separately_
timed_dhw` now returns False when water_heating_code is not "from main /
2nd-main system" ({901,902,914}), mirroring the existing WHC 903 electric-
immersion carve-out.
Simulated case 19 (electric storage main SAP 402 + WHS 911 + 210 L
loose-jacket cylinder) is the worksheet case. The single flag drives both:
- (53) Temperature factor: 0.54 → 0.6000 (worksheet base, no ×0.9)
- (55) storage loss/day: → 3.4531; (56)/(57)m Jan → 107.0456 (1e-4)
- (59)m primary loss: h=3 (43.31) → h=5 (Jan 64.5792), worksheet-exact
This also worksheet-pins S0380.224's loose-jacket storage loss magnitude
at 1e-4, previously only direction-validated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bump HEAD/next-slice/baseline, note the committed scripts toolkit, and add
the active "simulated case 19" section: the electric-storage-heater +
loose-jacket worksheet the user generated, what S0380.226 unblocked, and
the prioritised cluster bugs it exposed (cost (255) -334 = the +9 SAP
driver; Table 2b TF x0.9; WHS-911 storage-vs-combi routing; fabric +1.0).
Updated the "what to generate" ask to the two highest-value follow-ups
(electric room heaters; Sheltered/Adjacent RR gables).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Summary-path mapper raised UnmappedElmhurstLabel for a §15.1
"Cylinder Insulation Type: Jacket" lodging — only "Foam" (→1, factory)
was mapped. SAP10 cylinder_insulation_type uses 2 for loose jacket
(matching the GOV.UK API codes), and SAP 10.2 Table 2 Note 1 gives it a
separate ~2× storage-loss factor that the cascade now handles
(S0380.224). Add "Jacket" → 2 for cross-mapper parity with the API path
and so the loose-jacket storage-loss branch fires on the Summary path.
Surfaced by simulated case 19 (a 210 L jacket cylinder + electric storage
heaters), which previously couldn't extract at all. §4 suite 2397 passed;
mapper.py pyright unchanged at 32.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three reusable scripts (each with a purpose/usage docstring) for wide-scale
testing of the calculator's API front-end against the GOV.UK EPB register —
the toolkit behind the 1000-cert study (docs/HANDOVER_API_SAMPLE_ACCURACY.md):
fetch_2026_epc_sample.py — sample cert numbers across a date window
(random pages) + download full schema-21 JSON
to a cache; resumable, 429/5xx backoff.
eval_api_sap_accuracy.py — % within 0.5 SAP, error histogram, worst-40,
and the mapper/calculator raise breakdown.
analyse_api_sap_clusters.py — error grouped by property + heating type to
locate clusters (electric heating, flats, PV).
Cache dir defaults to /tmp/epc_2026_sample, overridable via EPC_SAMPLE_CACHE.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Captures the wide-scale 2026-register study (41.8% <0.5, heating-driven
cluster table), the 7 slices shipped (S0380.219-225), the prioritised
remaining work (electric-heating clusters + worksheet-backed raises), and
the single highest-ROI worksheet to generate: an electric-storage-heater
house with a loose-jacket cylinder + a room-in-roof with Sheltered/
Adjacent gables + an extension — one document that validates the #1
accuracy cluster, pins the S0380.224 loose-jacket fix at 1e-4, closes the
gable_wall_type Table 4 raise, and exercises multi-bp fabric.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The §10.7 no-water-heating default cylinder raised UnmappedSapCode for
age bands A-F (2 certs in a 2026 sample, bands B + C) because Table 29's
"A to F: 12 mm loose jacket" row wasn't plumbed — the loose-jacket
storage-loss branch didn't exist. S0380.224 added it, so this slice
completes the Table 29 lookup.
Restructure _TABLE_29_DEFAULT_CYLINDER_INSULATION_BY_AGE to carry
(cylinder_insulation_type, thickness_mm) per band — A-F → (loose jacket,
12), G/H → (factory, 25), I-M → (factory, 38) per RdSAP 10 Table 29
(PDF p.56) — and have the default read both, setting the loose-jacket
type for A-F instead of hardcoding factory. The strict-raise is retained
only for an absent / out-of-A-M age band (no Table 29 row).
Validated: certs 2211 (band B, SAP 49.8 vs lodged 52) and 3420 (band C,
11.2 vs 11) now compute. §4 + golden suite 2395 passed — the corpus
"no system" cert (age G, 25 mm factory) is unchanged. cert_to_inputs.py
pyright unchanged at 32; new test suppresses reportPrivateUsage.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`_cylinder_storage_loss_override` returned None for any cylinder whose
cylinder_insulation_type wasn't 1 (factory), so a loose-jacket cylinder
(code 2, RdSAP 10 field 7-11) fell to the cascade's zero-storage-loss
combi/instantaneous default — its real storage loss vanished. SAP 10.2
Table 2 Note 1 gives loose jacket a SEPARATE, ~2× higher loss factor
(L = 0.005 + 1.76/(t+12.8) vs factory 0.005 + 0.55/(t+4)); the
cylinder_storage_loss_factor_table_2 helper already implements it — only
the dispatch was missing.
Fix: a `_cylinder_storage_loss_insulation_label` resolver maps the lodged
code to the Table 2 branch (1 → factory_insulated, 2 → loose_jacket;
None/0/unknown → None, keeping the conservative no-loss default). The
override and the HW storage call now route through it instead of
hardcoding "factory_insulated".
Evidence + validation: a random 2026 register sample has 22 loose-jacket
certs that over-predicted SAP by +2.29 mean (18/22 too high, 1/22 within
0.5) — the exact signature of under-counted HW storage loss. After the
fix their mean error collapses to +0.45 and 11/22 land within 0.5, with
ZERO regression across the worksheet-validated cohort (§4 + golden suite
2394 passed — no validated cert lodges loose jacket, so none shifts).
Also unblocks the §10.7 A-F no-water-heating default (next slice) which
needs the loose-jacket branch. cert_to_inputs.py pyright unchanged at 32.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Summary PDFs preprocessed from `pdftotext -layout` wrap the windows-table
header across several lines. The third header line's tail ("U value / g
value / Draught Proofed / Permanent Shutters") tokenises to "value value
Proofed Shutters" and lands directly above the FIRST window's data row.
Because the first window in a building part has `before_start = 0`, its
prefix block reaches back into that header remnant. The remnant is
neither an orientation nor a building-part fragment, so it survived the
pops in `_compose_window_descriptors` and leaked into glazing_type as
"value value Proofed Shutters Double between 2002 and 2021" (windows 2-3,
whose prefix starts after the previous window's manufacturer line, were
clean).
Fix: the glazing-type phrase always starts with a glazing-start word
(Single/Double/Triple/Secondary), so trim any prefix fragments preceding
that word before joining the glazing type. Orientation/bp pops still run
on the full prefix, so they are unaffected.
Reproduced from `sap worksheets/Recommendations Elmhurst Files/
cavity_wall_insulation - main wall/before/Summary_001431.pdf`. Added a
regression test driving the real `_extract_windows_from_layout` path with
the verbatim tokenised header+rows. 2306 passed (+4), pyright net-zero.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds the user-simulated 001431 case (the cert that drove S0380.189/.190)
as an Elmhurst-only e2e fixture: Summary PDF → extractor → mapper →
calculator, every Block-1 SapResult field pinned against the
P960-0001-001431 worksheet at abs=1e-4. All 11 pins pass with zero
residual — the case is clean, confirming the S0380.190 gas-combi fuel
derivation closes the Summary path natively.
Verified the handover's flagged "+0.0007 SAP" was a target artifact, not
a cascade gap: the worksheet displays ECF (257) rounded to 1.6047 and
integer SAP (258)=78; the cascade's continuous SAP is computed from the
UNROUNDED ECF = (255)*(256)/((4)+45) = 660.9750*0.4200/173.0, giving
77.6147 — which matches the worksheet's own unrounded value. Pinning the
continuous SAP from the display-rounded ECF (→ 77.6144) was the wrong
target. Block-1 line refs all match exactly: (211) 10699.7225, (219)
3327.1592, (231) 86.0, (232) 283.2229, (255) 660.9750, (272) 3000.1664,
Σ(98) 8987.7669.
Summary mirrored into the tracked fixtures dir as
Summary_001431_gas_combi.pdf (distinct name — the corpus reuses cert
001431 across every heating variant); source Summary + worksheet tracked
under sap worksheets/golden fixture debugging/ as the pin ground truth.
2302 passed (+11), 0 failed; pyright net-zero on new/changed files.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The newer Elmhurst Summary export lodges a gas combi as §14.0 "Fuel Type"
empty + "Main Heating SAP Code" 104 (EES "BGW"), with no fuel string. The
site-notes mapper left `main_fuel_type=''`, so `cert_to_inputs` raised
`MissingMainFuelType` — blocking the whole gas-combi Summary path
(reproduced on the simulated 001431 case).
SAP 10.2 Table 4b (PDF p.168) rows 101-119 are "Gas boilers (including
mains gas, LPG and biogas)": the code fixes the boiler type/efficiency but
NOT the carrier, so 104 alone can't distinguish mains gas from LPG. The
disambiguator is §15.0 "Water Heating Fuel Type" — a combi/boiler heats
space + water from one appliance — exactly mirroring the existing
liquid-fuel (codes 120-141) fallback. `_elmhurst_gas_boiler_main_fuel`
adopts the §15.0 carrier only when the SAP code is in 101-119 AND §15.0
resolves to a gas/LPG fuel, so a regular boiler + electric immersion
(§15.0 = "Electricity") still strict-raises rather than mis-billing gas
as electric.
2291 passed (+1), 0 failed; pyright net-zero on both files.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Point-in-time note for the next agent: what S0380.185-189 shipped (worksheet
PE/CO2 pins, the two D_PV electricity-vs-gain fixes, and the thermal-mass-
parameter Table 22 fix), the per-line diagnosis template, the two worksheet-
block / gains-vs-solar traps, and the ranked open slices (Summary-path fuel
derivation first, then pin the simulated 001431 case, then cert 6035).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The §7 mean-internal-temperature cascade hardcoded the thermal mass parameter
(TMP) to 250 kJ/m²K at all 5 call sites, ignoring construction. RdSAP 10
§5.16 Table 22 (PDF p.48) makes TMP construction-dependent:
100 kJ/m²K — timber frame, cob, park home (regardless of internal
insulation); OR masonry (stone/solid brick/cavity/system
built) WITH internal insulation.
250 kJ/m²K — masonry WITHOUT internal insulation.
A too-high TMP inflates the §7 time constant τ = Cm/(3.6·H) (e.g. 40 h vs
16 h), under-cuts the temperature reduction between heating periods, and
over-states mean internal temperature → over-states space heating.
`_thermal_mass_parameter_kj_per_m2_k(epc)` classifies the MAIN building's
wall via the RdSAP `wall_construction` codes (5/7/8 = timber/cob/park) and
`wall_insulation_type` codes (3/7 = internal); unknown/curtain fall back to
the masonry 250 (no regression on unlisted classes). 17-case parametrised
test covers every Table 22 branch.
Diagnosis (per-line walk vs the user-simulated 001431 worksheet, same
archetype as golden cert 6035): fabric (26-37), internal gains (73), climate
(96)m and HTC (39) all EXACT; the entire +8.78 PE / -1.76 SAP gap was §7 MIT
(92) +0.71 °C, traced to TMP 250 vs Table 22's 100 (solid brick WITH internal
insulation). Fix closes the simulated case to 1e-4 on PE and CO2.
Blast radius: only golden cert 6035 re-pins (solid brick + internal
insulation) — SAP resid -6 → -2, PE +46.42 → +19.16, CO2 +1.07 → +0.42. The
47 dr87 cohort, 6 U985 fixtures and 41-variant heating corpus are all
masonry-no-internal → TMP unchanged at 250, all still pass. 2290 pass
(+17 new), 0 fail; pyright net-zero.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A single durable doc so agents can pick up the calculator without reading
historical handovers: (1) the accuracy bar for the two input paths
(site-notes 1e-4 vs worksheet; API 1e-4 when a worksheet exists, ±0.5
register fallback otherwise; cross-mapper parity); (2) the per-line-walk
debugging loop incl. comparing site-notes vs API; (3) the tools &
pipeline (Summary PDF → extractor → from_elmhurst_site_notes →
cert_to_inputs → calculate_sap_from_inputs → SapResult, plus the API
from_api_response front-end, section helpers, and where the test vectors
live). Pointer added from SAP_CALCULATOR.md; HANDOVER_* flagged as
point-in-time notes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SAP 10.2 Appendix M1 §3a (p.93) defines PV-eligible demand as
D_PV,m = E_L,m + E_A,m + E_cook,m + E_ES,m + (231)·n_m/365 + E_space,m + E_water,m
where E_L,m is the lighting ELECTRICITY (Appendix L eq L10, = line (232)).
The cascade fed `internal_gains_result.lighting_monthly_w` — the L12 internal
heat GAIN G_L,m = E_L,m × 0.85 ("assuming 15%" of lighting energy does not
become internal heat) — into D_PV, understating it by 15% of lighting on
every PV cert. That depressed the monthly β onsite/export split and
under-credited PV primary energy uniformly across the year.
Same gain-vs-electricity class as the cooking fix S0380.73 (L18 gain vs L20
electricity). Fix: scale the (shape-identical) lighting gain profile to the
annual E_L `lighting_kwh_per_yr` (= (232)), mirroring the (219)m hot-water
scale-to-annual. Magnitude-only, so the shape-weighted lighting CO2/PE
effective factor (Σkwh×f/Σkwh, magnitude-invariant) is unchanged; appliances
need no scaling (G_A = E_A, no 0.85). Diagnosis was empirical first (calc
lighting D_PV 95.1 vs worksheet (232) 111.88, ratio exactly 0.85) then
confirmed against the spec text (L9d/L10/L12, M1 §3a).
Impact (calc − full-precision dr87 worksheet): ALL 47 worksheet certs now
match at <1e-4 on BOTH PE (max |Δ| 0.0000 kWh/m²) and CO2 (max |Δ| 0.0000 kg)
— the convergence target, met cohort-wide. Combined with S0380.187 this
closes the entire gas+PV + ASHP PV residual. Re-pinned 47 worksheet residuals
to 0.0000 and 31 drifted lodged residuals (PV certs). SAP integers unchanged;
chain SAP 1e-4 intact (164 pass). 2273 pass, 0 regressions; pyright net-zero.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The PV onsite/export β-split (SAP 10.2 Appendix M1 §3a, p.93) divides PV
generation by the monthly PV-eligible electricity demand D_PV,m. The cascade
included main and water electricity (when those fuels are electric) but had
no term for SECONDARY space heating. For the 10 cohort-2 gas-main +
electric-secondary + PV certs, the (215)m secondary electric fuel was dropped
from D_PV,m — understating demand in the heating months only, depressing the
monthly β, and under-crediting onsite PV primary energy.
Spec: Appendix M1 §3a counts E_space,m as the dwelling's TOTAL electric
space-heating demand; for a gas-main/electric-secondary dwelling that is the
secondary fuel. Diagnosis was decisive: E_PV (generation) matched the
worksheet exactly every month, the onsite (233a) split diverged ONLY in
heating months (Jun-Sep near-exact), and all 10 affected certs have PV while
all clean gas certs have none. Empirically adding (215)m to D_PV closed cert
3136 onsite 726.9 → 790.3 (worksheet 792.1).
Impact (calc − full-precision dr87 worksheet), the 10 certs:
PE +0.5..+1.5 → +0.02..+0.046 kWh/m²; CO2 −0.5..−1.1 → +0.002..+0.0095 kg.
The whole 47-cert cohort now matches at PE <0.05 / CO2 <0.025. SAP integers
unchanged; chain SAP 1e-4 pins intact (164 pass). The uniform ~0.03 PE remnant
on PV certs is the separate (233a)/(233b) summer-month D_PV discrepancy.
Re-pinned the 10 worksheet + 9 lodged golden residuals (improvements).
2273 pass, 0 regressions; pyright net-zero (file's 32 errors pre-existing).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The existing golden test compares calc PE/CO2 against the integer-rounded
lodged register values (energy_consumption_current / co2_emissions_current),
which conflates real calculator gaps with register rounding. This adds a
parallel pin against each cert's Elmhurst dr87 worksheet (286)/(272) at full
precision — a clean calculator-vs-Elmhurst signal for the 47 worksheet-backed
certs (9 ASHP + 38 cohort-2).
Findings at capture (calc − worksheet, on the worksheet's own decimal TFA):
- 37/47 exact on both PE (<0.05 kWh/m²) and CO2 (<0.02 kg).
- 10 higher-consumption gas certs carry PE +0.5..+1.5 kWh/m² AND
CO2 -0.5..-1.1 kg simultaneously. PE-over + CO2-under on the same
certs is the fingerprint of a small gas→electricity fuel-split
difference (elec PE 1.51 > gas 1.13, but elec CO2 0.136 < gas 0.21),
not a factor-value error — next slice candidate.
An earlier "41/47 PE gaps" reading was a JSON-integer-TFA division artifact;
comparing on the worksheet's decimal TFA (which the calculator also uses)
collapses it to the real 10. Worksheet values frozen as literals (the dr87
PDFs are untracked, so not parsed at test time) per the worksheet_unrounded_sap
convention. Also replaced a pre-existing pytest.approx with abs-diff to keep
the file at zero pyright errors (feedback_abs_diff_over_pytest_approx).
106 passed (was 59); pyright 0 errors.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CH6's P960 worksheet input lodges Distribution Loss = "Two adjoining
dwellings sharing a single heating system" → (306) DLF = 1.0000, vs CH4's
"Calculated" → 1.5 → (306) = 1.4500. That DLF choice swings SAP/cost/CO2/PE
materially, but it is NOT present in the Summary PDF that the corpus pipeline
consumes (Summary → ElmhurstSiteNotesExtractor → mapper → calculator).
Proven empirically with a user-supplied controlled pair (CH adjoined
dwellings/Summary_001431 (1) vs (2)): the two Summaries are byte-identical
across every RdSAP INPUT field, differing only in the derived header
(SAP 80 vs 75, bill £954 vs £1237, emissions 5.407 vs 7.394 t). A
case-insensitive scan of the CH6 Summary for "distribution"/"adjoin" returns
0 hits. Since CH4/CH6 Summaries are themselves identical bar fuel type, no
Summary-derivable rule can yield CH4=1.45 AND CH6=1.0.
Doc-only change (comment in _EXPECTATIONS); 20/20 community-heating corpus
tests pass. Closes the CH6 re-litigation: pin held.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SAP 10.2 worksheet block 12b/13b (367)/(467) for a community heating
electric heat pump (Table 4a code 304 → Table 12 fuel 41 "heat from
electric heat pump"). The HP meters grid electricity, so per Table 12
note (s)/(t) + block 12b/13b footnote (a) its emission/PE factor is the
MONTHLY Table 12d/12e cascade (fuel 41 = standard-electricity profile),
weighted by the network heat profile, then × 1/heat-source-eff (1/COP):
(367)/(467) = [(307)+(310)] / COP × Σ((307+310)_m × factor_m)/Σ(...)
Per-line walk of CH3 (the displayed (367) 0.1535 / (467) 1.5717 are PDF
artifacts; the (373)/(473) totals reconcile only with):
CO2 factor = 0.15040 (monthly Table 12d wtd) vs cascade annual 0.136
PE factor = 1.55692 (monthly Table 12e wtd) vs cascade annual 1.501
Pre-slice the cascade routed code 304 through the non-electric branch
(`_co2_factor_kg_per_kwh(main) × 1/COP` = annual × scaling). New
`_is_heat_network_electric_main` (heat-network main whose fuel has a
Table 12d monthly set — i.e. fuel 41) routes all four factor helpers
(main + HW, CO2 + PE) through the monthly cascade × 1/COP. Non-electric
heat networks (gas 51 / oil 53 / coal 54) have no monthly set → annual
path unchanged (CH1, CH6 untouched).
Closure (CH3 was already SAP+cost EXACT):
CH3 (HP/Elec) CO2 −75.32→+0.0000 (= [(307+310)/3]×(0.1504−0.136)),
PE −249.32→−0.0000 (× (1.5569−1.501)) — FULLY EXACT
Corpus now 40/41 EXACT on all four metrics. Only CH6 remains: its
worksheet lodges a manual DLF=1.0 ("two adjoining dwellings") absent
from the Summary PDF (byte-identical to CH4 bar fuel type) — an
architectural limit, not a cascade gap. 2226 pass + 1 skip + 0 fail
(tolerances 1e-4 all metrics); pyright net-zero 43→43.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SAP 10.2 §10b: hot water for a community-heating dwelling bills at the
heat-network rate, not the cert-lodged fuel. Elmhurst §15.0 lodges
`water_heating_fuel_type = "Mains gas"` (3.48 p/kWh) as a placeholder on
community certs; the worksheet (342) Water-heating cost = (310) × the
S0380.171 CHP heat-fraction blend — the SAME rate as space heating (340).
Per-line walk of the CH2 block 10b:
(340) space = 11837.83 × 0.037955 = 449.3047 (cascade EXACT)
(342) water = 3854.12 × 0.037955 = 146.2830 (cascade billed
3854.12 × 0.0348 = 134.12 → −£12.16, the whole residual)
(350) lighting + (351) standing → (355) 754.1502.
`_hot_water_fuel_cost_gbp_per_kwh`'s `inherit_main_for_community_heating`
path already routes HW cost through `_fuel_cost_gbp_per_kwh(main)` (the
CHP blend), but its gate `_is_community_heating_hw_from_main` excluded
code 302. S0380.182 wired the 302 CO2/PE credit via
`_heat_network_code_302_effective_factor`, which intercepts the HW
CO2/PE helpers ABOVE this predicate's branch — so extending the
predicate to include 302 now affects ONLY the cost path.
Closures:
CH2 (CHP/Gas) SAP +0.5277→−0.0000, cost −£12.16→−£0.00 — FULLY EXACT
CH4 (CHP/Oil) SAP +0.5277→−0.0000, cost −£12.16→−£0.00 — FULLY EXACT
CH6 (CHP/Coal) SAP −7.49→−8.02, cost +£172.68→+£184.84 — its HW now
also bills the blend, compounding the DLF=1.0 quirk
(cascade DLF=1.45); same separate CH6 DLF front.
Corpus now 39 variants EXACT on all four metrics (CH2/CH4 join). Open:
CH3 CO2/PE (code-304 community-HP COP), CH6 all-metric (DLF=1.0 manual
override the Summary doesn't carry). 2225 pass + 1 skip + 0 fail
(tolerances 1e-4 all metrics); pyright net-zero 32→32.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SAP 10.2 worksheet block 12b (CO2) / 13b (PE) for community heating
"CHP and boilers" (SAP code 302). Per unit of network heat fuel
H = (307)+(310) the effective generation factor is:
chp×100/(362)×f_fuel − chp×(361)/(362)×f_disp + (1−chp)×100/(367)×f_fuel
(363)/(463) CHP fuel = chp_frac × 100/heat_eff × f_fuel
(364)/(464) less credit = −chp_frac × elec_eff/heat_eff × f_disp
(368)/(468) boiler fuel = (1−chp_frac) × 100/boiler_eff × f_fuel
f_fuel = Table 12 heat-network fuel factor (the CHP unit and the back-up
boilers burn the same community fuel — verified vs CH2 gas / CH4 oil /
CH6 coal worksheets (363)/(368)); f_disp = Table 12f (PDF p.196) credit
for the CHP-generated electricity. RdSAP 10 §C (p.58) defaults: heat eff
50% (362), electrical eff 25% (361), boiler eff 80% (367); CHP heat frac
0.35 per-cert via community_heating_chp_fraction.
New `_heat_network_code_302_effective_factor` + Table 12f flexible
constants (0.420 CO2 / 2.369 PE) + RdSAP §C efficiency constants, wired
into all four factor helpers (main + HW, CO2 + PE) ahead of the existing
single-fuel / 1-over-heat-source-eff path. The worksheet (368)/(468)
boiler emissions DISPLAY rounded/mis-aligned in the PDF, but the
(373)/(473)/(386)/(486) totals reconcile only with the boiler at the
full Table 12 factor — verified EXACT.
Two spec citations applied:
- Table 12f flexible-operation default for RdSAP community CHP is an
Elmhurst engine choice (Table 12f notes make "standard" the default);
mirrored per [[feedback-software-no-special-handling]] and documented
in SAP_CALCULATOR.md §8.3.
- Table 12 heat-network oil/biodiesel CO2 (codes 53/56) corrected
0.298 → 0.335 per Table 12 (p.189) "assumes 'gas oil'"; the code-302
oil cascade (CH4) was the first to exercise it. PE 1.180 was already
correct. No other variant uses these codes (no regression).
Closures (CO2 + PE only — the CHP credit does not touch cost/SAP):
CH2 (CHP/Gas) CO2 −1411.49→+0.0000, PE +1331.23→+0.0000 EXACT
CH4 (CHP/Oil) CO2 −4378.24→−0.0000, PE +319.81→−0.0000 EXACT
CH6 (CHP/Coal) CO2/PE re-pinned (+2411.54 / +5023.48) — its worksheet
lodges a manual DLF=1.0 the Summary doesn't carry, so
cascade DLF=1.45 over-scales H; same root as the CH6
SAP −7.49 / cost +£172 (separate DLF front).
CH2/CH4 are now CO2+PE-exact but still carry the heat-network cost/SAP
residual (+0.5277 SAP / −£12.16 cost, exposed by S0380.175 — cost-side,
untouched here). CH3 unchanged (code 304 community-HP COP front).
Corpus state: 37 variants EXACT on all four metrics (incl. CH1);
remaining residuals are CH2/CH4 cost+SAP, CH3 CO2+PE (HP COP), CH6
all-metric (DLF quirk). 2223 pass + 1 skip + 0 fail (tolerances 1e-4 all
metrics per S0380.181); pyright net-zero 43→43.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The corpus residual-pin tolerances had drifted looser than the comment
above them claimed ("pin at 1e-4 relative to lodged precision"): SAP was
1e-3, cost ±£0.01, CO2 ±0.1 kg, PE ±0.1 kWh. A ±0.1 kg CO2 band could
silently mask a ~0.09 kg drift on a variant we report as EXACT.
The worksheet pins are extracted from the P960 PDF text, which prints
4 d.p., so the hard residual floor is ~5e-5 (half a unit in the last
printed digit) regardless of cascade precision. 1e-4 sits just above
that floor. All 41 variants hold at uniform 1e-4 on continuous SAP,
cost, CO2 AND PE — confirming the 37 EXACT variants are genuinely exact
to PDF print-rounding and the looser bands were masking nothing.
Aligns the guard with [[feedback-zero-error-strict]] /
[[feedback-continuous-sap-tolerance]] (basically zero error across all
four metrics). Test-only change; no cascade behaviour touched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SAP 10.2 Appendix C §C3.2 (PDF p.51), verbatim: "CO2 emissions and
Primary Energy associated with the electricity used for pumping water
through the distribution system are allowed for by adding electrical
energy equal to 1% of the energy required for space and water heating."
Worksheet line (313) = 0.01 × [(307)+(310)]; its CO2 (372) and PE (472)
bill on the Table 12d/12e monthly factors for fuel code 50 ("electricity
for pumping in distribution network"), weighted by the monthly heat
profile per worksheet footnote (a). (307)m/(310)m = (space_demand +
hw_output) / efficiency (the cascade models a heat network's generator
efficiency as 1/DLF).
This un-defers the (372)/(472) front the post-S0380.179 handover flagged
"don't guess until the factor source is identified": the source is
§C3.2 + Table 12d/12e code 50, NOT an empirical constant. The apparent
0.1994/0.2114 "factor" is an Elmhurst DISPLAY artifact — the worksheet
shows the (372) energy column as 0.01×(307) (space only) while computing
emissions on 0.01×(307+310) per the §C3.2 text. Verified EXACT line-by-
line against the CH2 corpus worksheet: (372)=23.6007 CO2 (rating),
(472)=208.2267 PE (demand).
New `_heat_network_distribution_electricity` helper (gated on
`_is_heat_network_main`) precomputes the energy + effective CO2/PE
factors; three new CalculatorInputs fields + calculator.py CO2/PE
summation terms (0.0/None → no-op for individually-heated certs).
Closures:
CH1 (Boilers/Gas) CO2 −23.60→−0.00, PE −208.23→+0.00 — FULLY EXACT
CH3 (HP/Elec) CO2 −98.92→−75.32, PE −457.54→−249.32 (distribution
component closed; code-304 community-HP COP remains)
CH2/CH4/CH6 gain their (372)/(472) component (CO2 +23.6, PE
+208.2); dominant CHP displaced-electricity credit
residual (Table 12f + block 12b/13b) is next slice.
No regression on the other 36 corpus variants (helper returns None off
heat-network mains) + golden + U985 fixtures. 2223 pass + 1 skip + 0
fail; pyright net-zero 43→43.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Captures the corpus state (36 EXACT + 5 pinned community-heating
variants), the SAP 302 CHP credit cluster as the highest-leverage
remaining front, the unresolved 0.8523 / 0.1994 worksheet-factor
mysteries to per-line-walk before hypothesising, and — importantly —
the new test layout (tests/domain/sap10_calculator/) that changes every
verification command.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The calculator tests lived under domain/sap10_calculator/{tests,worksheet/
tests,rdsap/tests,climate/tests,validation/tests}, none of which are in
pytest.ini testpaths — so CI (which collects tests/) never ran them. Relocate
all five dirs to tests/domain/sap10_calculator/{,worksheet,rdsap,climate,
validation}, mirroring the tests/domain/property_baseline/ convention, so the
cascade-pin / golden / e2e conformance suites run in CI.
Mechanics:
- git mv preserves history (110 files).
- Flattening the trailing /tests keeps each file's depth-to-repo-root
identical, so all 16 repo-root parents[4] fixture refs stay valid. Only
test_pcdb_etl.py's parents[1] (→ pcdb data) and one hardcoded absolute
golden-fixture path in test_cert_to_inputs.py needed rebasing.
- Cross-imports rewritten domain.sap10_calculator.worksheet.tests →
tests.domain.sap10_calculator.worksheet (21 files incl. the external
importer backend/documents_parser/tests/test_summary_pdf_mapper_chain.py).
- Golden-fixture path strings in test_summary_pdf_mapper_chain.py +
scripts/fetch_cohort2_api_jsons.py updated to the new location (the JSONs
moved with the rdsap tests).
load_cells / gitignored worksheet xlsx: the xlsx-pinned tests (test_dimensions
/ ventilation / water_heating) read 2026-05-19-17-18 RdSap10Worksheet.xlsx,
which is gitignored (.gitignore `*.xlsx`) and so absent in CI. _xlsx_loader.
load_cells now pytest.skip()s when the file is absent, so those tests run
locally and skip cleanly in CI instead of erroring — no new CI failures from
the move, and the gitignore policy is respected.
Verified: tests/domain/sap10_calculator + backend/documents_parser +
tests/domain/property_baseline = 2248 pass, 1 skipped; pyright resolves the
new import paths with zero import-resolution errors.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
test_heating_systems_corpus.py (and one pcdb-1 cross-check in
test_cert_to_inputs.py) read the 001431 controlled-variable corpus PDFs
directly at runtime from `sap worksheets/heating systems examples/`, but
that directory was never committed — it was supplied locally on
2026-05-30 and only ever existed on dev machines. CI therefore errored
with "no Summary PDF in …" for all 57 corpus variants.
Commit the 82 corpus PDFs (41 populated variant folders × Summary +
P960, 4.7 MB) in place so the cascade-vs-worksheet residual pins run in
CI, matching the existing convention where the U985 / 000565
conformance fixtures are committed under
backend/documents_parser/tests/fixtures/ (31 PDFs already tracked).
Only the .pdf fixtures are added; the stray .DS_Store and a P960 .txt
dump in pcdb 1/ are left untracked.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two unrelated breakages surfaced after merging the PR into this branch;
neither was caused by the appliances/cooking work.
test_appendix_u.py (9 failures) — signature drift + wrong methodology
label. The climate lookups were renamed `external_temperature_c(region=…)`
→ `(region_or_climate, month)` when PostcodeClimate support landed for
the demand cascade, but the tests still passed `region=`. The expected
values match our SAP 10.2 _TABLE_U1/U2/U3 exactly (UK-avg Jan 4.3 °C,
Thames Jul 17.9 °C, solar Jul 189 W/m², Shetland Jan wind 9.5 m/s), so
these are valid 10.2 coverage — fixed the call signature to positional
and corrected the mislabelled "SAP 10.3" docstrings to SAP 10.2 (we
track 10.2 deliberately). Also converted pytest.approx → abs(x-y)<=tol
per the repo convention; pyright on the file drops 48 → 0.
test_table_32.py (2 failures) — the parametrised "match PDF p.95" test
pinned heating oil (code 4) = 7.64 and FAME (code 73) = 5.44, but the
table deliberately diverges from the PDF for these two carriers: oil =
5.44 (Slice S0380.131, two independent lodging engines agree the PDF
7.64 is the outlier) and FAME = 7.64 (Slice S0380.168). Updated the two
expected values to the worksheet-canonical figures the table actually
uses, with inline citations + a docstring note on the divergence.
Full calculator + property_baseline + heating-corpus suites: 1748 pass,
0 fail. pyright net-improving on both files.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
