Adds the mapper-driven e2e cascade pin for "simulated case 50" (000565 semi,
electric storage main SAP 402 + portable electric secondary + MVHR + whc-903
DUAL electric immersion + 160 L cylinder, Economy-7). Routes the Summary PDF
through extractor + mapper + calculator like the other 000565 fixtures.
Locks in two off-peak fixes this case ground-truthed:
- the Table 13 HW high/low split applied to CO2/PE (commit 39ae2cf0), and
- the Table 12a Grid 2 MVHR fan fraction 0.71/0.58 (commit cd5113ab).
All 11 SAP-result fields reconcile to the U985 worksheet EXACTLY, including
the (272) rating CO2 2397.1237 — SAP 38.8426 (=39), cost £1317.0116, water
1668.0788 kWh, fans 315.6384 kWh.
Summary mirrored to the tracked fixtures dir so the test doesn't depend on
the unstaged `sap worksheets/` workspace.
pyright strict gate not run locally (pyright not installed in this container).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the mapper-driven e2e cascade pin for "simulated case 52" (000565
semi + regular non-combi mains-gas boiler SAP 102 + 160 L foam cylinder
heated from the main, no cylinder stat, uninsulated primary pipework,
standard tariff). Routes the Summary PDF through extractor + mapper +
calculator like the other 000565 / 001431_case* fixtures.
This closes the last untested branch of the cylinder/water chain: the
SAP 10.2 §4 cylinder storage loss (Table 2/2a/2b lines 51-55) + the
Table 3 PRIMARY circuit loss (59, uninsulated pipework + no stat) that
combi/immersion fixtures don't reach. All 11 SAP-result fields reconcile
to the U985 worksheet EXACTLY with no calculator change — SAP 57.2904
(=57), cost £911.1973, water 3929.7635 kWh — confirming the cylinder-loss
derivation is correct.
Summary mirrored to the tracked fixtures dir so the test doesn't depend
on the unstaged `sap worksheets/` workspace.
pyright strict gate not run locally (pyright not installed in this container).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The SAP/EI rating is computed on UK-average weather (Appendix U Tables
U1-U3 region 0) so ratings are nationally comparable, but Appendix U
paragraph 1 (PDF p.124) requires that "other calculations (such as for
energy use and costs on EPCs) are done using local weather. Weather data
for each postcode district are taken from the PCDB". `Sap10Calculator.
calculate` ran ONE cascade (UK-average) and fed it to SAP, CO2 AND primary
energy, so every cert's EPC-displayed CO2/PE were computed on the wrong
climate. Because most of England is warmer than the UK-average, this
systematically OVER-counted heating demand on the emissions/PE outputs.
The two cascades (`cert_to_inputs` rating, `cert_to_demand_inputs`
postcode) already existed; this wires the demand cascade into the
production entry point and grafts its CO2/PE onto the rating result (SAP
unchanged). The corpus gauge's longstanding +5% CO2/PE over-estimate was
mostly this climate bug, NOT (as previously diagnosed) per-cert mapper
fidelity:
CO2 MAE 0.26 -> 0.12 t/yr (bias +0.18 -> +0.04)
PE MAE 13.6 -> 3.8 kWh/m2 (bias +9.0 -> +0.24)
SAP within-0.5 = 69.7% (rating cascade, unchanged)
Worksheet-validated to 1e-4 on simulated case 45 (heat-pump ground-floor
flat, postcode W6): the P960 prints the current dwelling twice — Block 1
on UK-average weather (SAP 60.5318, CO2 692.13) and Block 2 on postcode
weather (CO2 626.78, PE 6581.59). Both reproduce exactly. Added a tracked
case-45 Summary fixture + two-cascade cascade pin as a permanent guard,
and ratcheted the corpus CO2/PE ceilings to 0.13 / 4.2. The e2e Elmhurst
suite (Block-1 line refs) now pins the rating cascade directly; the two
Vaillant overlay snapshots refreshed to demand-cascade CO2/PE.
pyright not installed in this codespace (strict gate not run locally);
change is type-trivial (dataclasses.replace over SapResult).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SAP 10.2 §2 (17)-(18): a measured/design air permeability at 50 Pa from a
Blower Door test routes infiltration via `(18) = AP50/20 + (8)`, in
preference to the components-based (16) estimate. The Elmhurst extractor
read only the AP4 ("Pulse") column of §12.2, so a Blower Door result
(§12.2 "Pressure Test Result (AP50)") fell through to the structural-
infiltration default — over-counting ventilation heat loss.
Surfaced by simulated case 44 (AP50 4.50): effective air change rate was
0.81 vs the worksheet's 0.58 (+38% ventilation loss). The cascade already
supports `air_permeability_ap50` (preferred over AP4); this wires the read
end to end (extractor → ElmhurstSiteNotes → SapVentilation → cert_to_inputs).
Pinned against the case-44 P960 §2 at abs=1e-4: (18) infiltration 0.3417
(= 4.5/20 + 0.1167) and (25) Jan effective ach 0.5812. Worksheet harness
stays 47/47 0-raised.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Schema coverage (datatypes/epc/domain/mapper.py):
- SAP-Schema-18.0.0: full-SAP shape ≡ 17.1 → from_sap_schema_17_1, no normalisation.
- SAP-Schema-16.0: same reduced-field 16.x path; default the omitted `tenure`
field in _normalize_sap_schema_16_x (metadata; SAP cascade never reads it).
Genuinely sparse 16.x certs (missing core fabric fields) still fail loud.
- Regression tests + sap_18_0_0.json / sap_16_0.json fixtures; 0 new pyright errors.
Autonomous triage of the worklist (scripts/hyde/autonomous_run_findings.md):
- Found + diagnosed 2 bugs (flagged, NOT fixed): (1) MAPPER — full-SAP openings
lodged in mm read as m → multi-million-m2 windows → SAP clamps to 1 (uprn_
10093117227 / 10090317693 / 10091636031); (2) CALCULATOR — database heat-pump
fuel code 39 mis-priced as gas, over-rates ~14 (uprn_10093114053).
- Most certs map within +/-4 of lodged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- SAP-Schema-16.3: same reduced-field RdSAP shape as 16.2 — generalise the
normaliser to _normalize_sap_schema_16_x and route both 16.2/16.3 through it.
uprn_44012843 maps → SAP 79 (lodged 81).
- SAP-Schema-17.0: structurally identical to the full-SAP 17.1 schema (measured
sap_opening_types), so it parses with the 17.1 dataclass and reuses
from_sap_schema_17_1 with no normalisation. uprn_10023444324 → 80, uprn_
10023444320 → 81.
- Regression tests (16.3 dispatch, 17.0 dispatch) + sap_16_3.json / sap_17_0.json
fixtures; 0 new pyright errors. All 7 e2e UPRNs now map.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SAP-Schema-16.2 (datatypes/epc/domain/mapper.py):
- 16.2 is structurally an RdSAP-17.1 cert under a different name; add
_normalize_sap_schema_16_2 (field renames + defaults) and dispatch to the
tested from_rdsap_schema_17_1 mapper. uprn_100020933699 maps → SAP 71.
- Honour a "Single glazed" windows description when multiple_glazing_type="ND"
(was defaulting to double) → RdSAP-21 code 5; eng 72→71 (lodged 70).
- 4 regression tests + sap_16_2.json fixture; 0 new pyright errors.
Flat party-wall fix (domain/sap10_calculator/worksheet/heat_transmission.py):
- Full-SAP flats carry flatness in dwelling_type, not property_type, so the
party-wall default fell through to the 0.25 house value instead of the RdSAP
Table-15 flat 0.0. Add _is_flat_or_maisonette_dwelling fallback + regression
test. uprn_10093116529 80→81 (matches the cert's lodged party u_value 0).
Accuracy corpus pins (tests/domain/sap10_calculator/test_real_cert_sap_accuracy.py):
- uprn_10093116543 (SAP-17.1 gas-combi semi): engine 81 (Elmhurst 77; documented
full-SAP→RdSAP residual — measured wall/floor U + PCDB boiler vs RdSAP defaults).
- uprn_10093116529 (SAP-17.1 g/f flat): engine 81 (Elmhurst 78).
devcontainer: add poppler-utils (pdfinfo) for the documents-parser PDF fixtures.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire the non-separated conservatory into the §3 heat-transmission +
§1 dimensions cascade per RdSAP 10 §6.1 (PDF p.49) + Table 25 (p.51):
"The floor area and volume of a non-separated conservatory are added to
the total floor area and volume of the dwelling. Its roof area is taken
as its floor area divided by cos(20°), and wall area is taken as the
product of its exposed perimeter and its height. ... The conservatory
walls and roof are taken as fully glazed ... Glazed walls are taken as
windows, glazed roof as rooflight."
New `worksheet/conservatory.py` derives the geometry:
- height from the equivalent storey count (§6.1: 1 storey → ground-floor
room height; 1½ → ground + 0.25 + 0.5×first; etc.);
- glazed WALL → window (27) at Table 25 U (double 3.1 / single 4.8) with
the §3.2 curtain resistance (R=0.04) → U_eff 2.758;
- glazed ROOF → rooflight (27a) at Table 25 roof U (double 3.4 / single
5.3) + curtain → U_eff 2.993;
- FLOOR → (28a) via BS EN ISO 13370 as an uninsulated SOLID ground floor
with 300 mm walls (§5.12, spec p.43), exposed perimeter = glazed
perimeter → U 0.89;
- glazed wall + roof + floor areas join (31)/(36); the fully-glazed
structure walls/roof add nothing (the glazing IS the window/rooflight).
`dimensions_from_cert` adds the conservatory floor area to TFA (4) and
floor area × height to volume (5) (feeds ventilation (8)), without making
it a storey (avg storey height for §2 infiltration is unchanged).
Pinned against the simulated case-44 P960 §3 at abs=1e-4 — every line ref
EXACT: (4) 95.3800, (5) 257.1630, (27) 96.1169, (27a) 38.2201, (28a)
21.4164, (29a) 35.5852, (30) 7.4688, (31) 294.2900, (33) 207.3274,
(36) 23.5432. The remaining whole-dwelling SAP/CO2 gap is the §6 solar
gains, closed in the next slice. Worksheet harness stays 47/47 0-raised.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
RdSAP 10 §6.1 (PDF p.49) models a non-separated (heated) conservatory as
part of the dwelling. Until now the Summary §5 block was reduced to an
inert `has_conservatory` bool and the geometry (floor area, glazed
perimeter, glazing, storey height) was dropped on both paths.
Plumbing only — no cascade consumer yet (Slices B/C/D wire §3/§6):
- ElmhurstSiteNotesExtractor reads the §5 Conservatory block into a new
`Conservatory` site-notes record (scoped to §5 so the generic
"Floor Area"/"Room Height" labels can't collide with §4 dimensions);
- domain gains a frozen `SapConservatory` (floor area, glazed perimeter,
double/single glazing, thermally-separated guard, equivalent storey
count) on `EpcPropertyData.sap_conservatory`;
- the Elmhurst mapper threads it through, dropping SEPARATED
conservatories per §6.2 ("A separated conservatory ... is disregarded").
Verified against the simulated case-44 Summary (RefNo 001431): extracts
floor_area=12.0, glazed_perimeter=9.0, double_glazed=True, 1 storey.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Golden regression fixture for the multi-feature dwelling that surfaced the
two Elmhurst-extractor bugs in a33707f8. case 43 is a 2-BP mid-terrace with
a DETAILED room-in-roof (two slopes, two flat ceilings, party + exposed
gables, two common walls), a MIXED-insulation multi-section roof (Main
insulated + Extension uninsulated 2.30), a DRY-LINED extension solid wall,
a mains-gas boiler (102 / control 2106) and a House-coal solid-fuel
secondary (633).
Routes the Summary PDF through the WHOLE extractor + mapper + calculator
pipeline (no hand-built EpcPropertyData) and pins the §3 fabric + SAP-rating
block at abs=1e-4: (29a) walls 74.5800, (30) roof 38.5008, (33) fabric
172.7844, continuous SAP 73.2332 = (258), CO2 3518.3037 = (272). Guards the
detailed-RR slope/common_wall surfaces, the dry-lining R=0.17 adjustment,
and the per-part mixed-roof billing together. Summary mirrored to
backend/documents_parser/tests/fixtures/Summary_001431_case43.pdf; provider
module mirrors the _case6/_case21 pattern, assertion in
test_section_cascade_pins. Harness 47/47; regression = the 3 pre-existing
fails; pyright net-zero.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two compensating Summary-extractor bugs surfaced by simulated case 43 (a
2-BP mid-terrace with a detailed room-in-roof + a dry-lined extension wall).
Their fabric errors nearly cancelled (walls net −0.76 W/K), hiding both
behind a deceptively small +0.05 SAP delta.
Bug 1 — main/extension wall dry-lining never read. The §7 "Dry-lining:
Yes/No" line was parsed only for ALTERNATIVE walls; the main/extension
WallDetails dropped it, so a dry-lined solid wall was billed at its
un-adjusted base U. RdSAP 10 §5.8 + Table 14: a dry-lined uninsulated wall
adds R=0.17 → U = 1/(1/U_base + 0.17). Case 43 Ext1: solid brick 1.70 →
1.32. Added `WallDetails.dry_lined`, read it in the extractor (both the
main-wall builder and the As-Main copy), threaded it to the domain
`wall_dry_lined` (emit None when undried — cascade-equivalent to False,
keeps the field absent for the non-dry-lined majority).
Bug 2 — the LAST room-in-roof surface row's U over-read. The per-row token
scan stops at the next RIR-row name; the final surface (no successor) over-
read into the following section, shifting the trailing-token slotting and
silently zeroing its `default_u` (case 43 Common Wall 2: 1.90 → 0.00 → the
2.4 m² common wall billed at U=0 instead of the main-wall 1.90). Stop the
scan at the row's natural end — the "Yes"/"No" u_value_known flag plus the
trailing u_value numeric.
Case 43 now reproduces the P960 EXACTLY: (29a) walls 74.5800, (33) fabric
172.7844, continuous SAP 73.2332 = (258), CO2 3518.30 = (272), all <1e-4
(was SAP +0.0455 / CO2 −8.04). Harness 47/47 0 raised; regression = the 3
pre-existing fails; pyright net-zero (51=51).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`u_roof` only implemented the joist column, so roofs lodged insulated at
rafters (`roof_insulation_location == 1`) were mis-billed at the joist U
on both the API and Summary paths — under-stating loss, over-rating SAP.
RdSAP 10 §5.11.2 Table 16 (spec p.42-43) gives a distinct "insulation at
rafters" column (2): the rafter cavity is shallower than a loft void, so
the same depth yields a higher U (200 mm: rafters 0.29 vs joists 0.21).
§5.11 Table 18 (p.45) likewise carries a rafters column (2) for unknown /
as-built thickness (footnote (1): "The value from the table applies for
unknown and as built") — band A-D = 2.30, E = 1.50, F = 0.68, diverging
from the joist column's 100 mm-equivalent 0.40 default (footnote (4)).
- add `_ROOF_RAFTERS_BY_THICKNESS` (Table 16 col 2) + `_ROOF_RAFTERS_BY_AGE`
(Table 18 col 2) to rdsap_uvalues; `u_roof` selects them via a new
`insulation_at_rafters` flag (ignored for flat / sloping-ceiling roofs).
- `heat_transmission` derives the flag PER BUILDING PART from
`roof_insulation_location` (gov-API int 1 / Summary "R Rafters"), which
also fixes the multi-part dedup-roof-join problem: each part's own
location now drives its U, replacing the unattributable joined
`epc.roofs[]` description.
Worksheet-validated to 1e-4: simulated case 41 (4-bp — Ext1 rafters 200mm
→ 0.29, Ext3 rafters As-Built band F → 0.68; roof total 24.8350) and case
42 (6 variants — rafters 50mm → 0.88, rafters unknown band C → 2.30,
joists/none unchanged). Case 40 stays exact (roof 35.340, total 441.1606);
worksheet harness 47/47.
Corpus within-0.5 66.9% → 66.5% (gates 0.65/1.08 hold) — a spec-correct
shift, NOT a regression: all 15 corpus rafter certs carry redacted (None)
thickness yet lodge roof EER 2-4 (insulated), so the open API blanked a
specified thickness and the spec's unknown-rafter 2.30 default correctly
over-states them. Recovery needs a roof-EER→thickness inference on the
API path (follow-up), not a change to the U-table.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Elmhurst Summary §15.1 lodges "Cylinder Size: Value known" with the
measured volume in the "Cylinder Volume (l)" line — the Summary-path
equivalent of the gov-API "Exact" descriptor. The mapper had no entry for
"Value known" so `_elmhurst_cylinder_size_code` raised UnmappedElmhurstLabel,
and even once mapped the measured volume was never threaded through, so the
cascade dropped the cylinder storage loss (~468 kWh/yr) from (219) water
heating on every measured-volume-cylinder Summary.
Per RdSAP 10 §10.5 Table 28 (p.55) a measured cylinder volume is used
directly. Map "Value known" → cascade code 6 (Exact) and thread the §15.1
"Cylinder Volume (l)" value into SapHeating.cylinder_volume_measured_l, which
`_cylinder_volume_l_from_code` (cert_to_inputs.py:5281) already reads for
code 6 — mirroring the gov-API path (mapper.py:1575/1885).
Pins simulated case 39 (P960-0001-001431): an age-A mid-terrace on direct-
acting electric room heaters (SAP code 691, cat 10, control 2602) with
electric-immersion DHW off a 117 L "Value known" cylinder. The full
extractor→mapper→calculator cascade now reproduces the worksheet's SAP-rating
block EXACTLY — SAP value 36.6365 (band F) and (272) CO2 2056.0731 kg/yr,
with (219) water heating 2637.5049 and (255) total energy cost 1802.0039.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reduced-field window U: heat_transmission derived the synthesised-window
raw U from u_window(all None) -> the 2.5 placeholder regardless of glazing.
Now routes the (uniform) glazing_type code through u_window (RdSAP Table 24)
so e.g. double pre-2002 reads 2.8, not 2.5. Only the pre-SAP10 reduced-field
path is affected (21.0.1 certs carry per-window U upstream) — the RdSAP-21.0.1
corpus gauge is unchanged at 66.9% within-0.5.
test_real_cert_sap_accuracy: pin uprn_10002468137 (RdSAP-17.1, all-electric
storage heaters) at SAP 61, validated against Elmhurst on identical inputs
(dual off-peak immersion, 110 L cylinder, 2 baths). Our engine reproduces
Elmhurst's fuel cost to the penny; lodged 55 is the old SAP-2012 schema.
Tooling to grow the accuracy corpus:
- scripts/fetch_real_life_epc_sample.py — capture a cert by UPRN into the corpus.
- scripts/compare_epc_paths.py — diff gov-API vs Elmhurst-summary EpcPropertyData
and run both through the engine, localising mapper vs calculator differences.
- skill validate-cert-sap-accuracy — the end-to-end loop (capture -> Elmhurst
inputs -> human builds -> compare -> reconcile -> pin in the test).
- skill epc-to-elmhurst-rdsap-inputs reference: corrected immersion (code 1=dual),
cylinder size (code 2 = Normal/110 L), and bath-count (WWHRS sub-tab) mappings.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The realistic re-generation of case 37 (code-117 gas boiler, control 2102,
+ a MAINS-GAS condensing gas-fire secondary code 611, vs case 37's biogas
605). The full extractor -> mapper -> calculator pipeline reproduces the
worksheet's SAP-rating block EXACTLY: continuous SAP 60.9152 (Δ 2e-5) and
(272) CO2 5801.0770 (Δ ~0). This confirms the boiler-efficiency /
control-2102 −5pp interlock / secondary-fuel handling are all correct, and
that case 37's +7 gap was purely the biogas sub-fuel the Summary export
cannot carry.
Summary mirrored into backend/documents_parser/tests/fixtures so the pin
runs without the unstaged workspace. PE not pinned — it is a separate
DPER block (different scope) already guarded by the corpus PE gauge.
Worksheet harness 47/47 unchanged; pyright net-zero.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The §11 layout parser keys a window's wall Location on the glazing-prefix /
orientation tokens around its data row. An alt-wall window lodges its
"Alternative wall 1" Location wrapped across the lines bracketing the W×H×A
row. For a DOUBLE-glazed alt window the prefix line also carries the glazing
phrase ("Double between 2002 Alternative wall"), so the partition breaks
there and the location survives into the window's pre-data slice. For a
SINGLE-glazed alt window the "Alternative wall" line stands alone with no
glazing-type word, so _partition_after_manuf scanned past it and swallowed
it into the PREVIOUS window's suffix — the window then defaulted to
"External wall" and its opening deducted from the wrong wall.
Fix: treat a standalone wall-location line ("Alternative wall" / "External
wall" / "Party wall") as a window boundary in _partition_after_manuf, so it
attaches to the following window's prefix. Surfaced by simulated case 34
(cert 001431 electric-storage flat): 2 of 4 single-glazed alt-wall windows
were mis-allocated, splitting 2.75/10.78 m² instead of the worksheet's
4.63/8.90 corridor/external opening areas.
Elmhurst-extractor only; API gauge unaffected. Regression gate green (3
pre-existing fails unrelated); worksheet harness 47/47 unchanged. Case 34's
alt-wall opening area now matches the worksheet; the corridor wall net area
is correct (the cert's residual is now isolated to the unheated-corridor
door, a separate slice).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Elmhurst Summary §7 lodges "Alternative Wall N Sheltered Wall: Yes" for
a sub-area adjacent to an unheated buffer (e.g. a flat's corridor wall),
but the extractor dropped it and _map_elmhurst_alternative_wall never set
SapAlternativeWall.is_sheltered — so the cascade billed the sub-area at its
full exposed U instead of the RdSAP 10 Table 4 (p.22) sheltered U =
1/(1/U + 0.5).
The calculator already applies is_sheltered (_alt_wall_w_per_k) and the
gov-API path already wires sheltered_wall=="Y"; this brings the Elmhurst
front-end to parity. Three-part change: AlternativeWall.sheltered field +
_alternative_walls_from_lines parse ("Alternative Wall N Sheltered Wall") +
_map_elmhurst_alternative_wall is_sheltered=a.sheltered.
Surfaced by simulated case 34 (cert 001431 electric-storage flat): the
6.02 m² corridor wall billed at full U=1.50 (9.03 W/K) instead of the
sheltered 0.86 (5.18 W/K) — +3.85 W/K, -1.61 SAP. Post-fix the alt wall
matches the worksheet's (29a) 5.177 and case 34 closes from -1.61 to -0.30
(remaining residual is a separate window/wall area-allocation thread).
Elmhurst-mapper only: API SAP gauge unchanged (57.6% within 0.5); worksheet
harness 47/47 unaffected; regression gate green (3 pre-existing fails
unrelated); pyright net-zero.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Elmhurst dwelling-type classifier keyed "Top-floor flat" on a "dwelling
below" floor lodgement. A single-storey flat exposed BOTH top (a real
external roof) AND bottom (floor over partially-heated space, no dwelling
below) therefore fell through to "Ground-floor flat" — which the cascade's
_dwelling_exposure maps to has_exposed_roof=False, dropping the external
roof entirely.
Surfaced by simulated case 34 (cert 001431 reconfigured as a slimline
electric-storage flat): the worksheet bills (30) External roof = 39.98 m²
x U=2.30 = 91.95 W/K — the dominant heat-loss element — but the cascade
dropped it, under-stating space-heating demand by 42% (6550 vs 11357
kWh/yr) and over-predicting SAP by +21.76 (57.07 vs worksheet 35.31).
Fix: an exposed (non-party) roof puts the flat on the top storey
regardless of what is below it. Classify as "Top-floor flat" whenever the
roof is exposed; the flat's exposed floor is recovered downstream by the
existing per-BP is_above_partially_heated_space / is_exposed_floor override
in heat_transmission (§3). Party-roof flats ("another dwelling above") are
unaffected and stay Ground-/Mid-floor.
This is an Elmhurst-mapper (dwelling_type) bug, NOT a calculator bug: the
calculator correctly trusts dwelling_type, and the gov-API path supplies
the position directly (cert 0036 — a genuine ground-floor flat whose API
data lodges a "Pitched, no access" roof construction under another dwelling
— stays party, 2.51 W/K). API SAP gauge unchanged (57.6% within 0.5);
worksheet harness 47/47 unaffected; case 34 roof now exact (residual -1.61
is a separate flat-corridor wall-U thread). Regression gate green (3
pre-existing fails unrelated).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When a property lodges an Alternative Wall, pdftotext interleaves the §11
"Location" column ("Alternative wall 1") into the wrapped glazing-TYPE cell,
producing labels like "Double between 2002 Alternative wall and 2021 1
Alternative wall" (cert 001431 storage-heater variants, simulated case 34).
The existing greedy trailing-suffix strip (\s+Alternative wall.*$) truncates
at the FIRST "Alternative wall", losing "and 2021" and yielding the
unmatchable "Double between 2002". Added a fallback that removes EVERY
"<External|Alternative|Party> wall [n]" fragment and any stray 1-2 digit
location index from the raw label, then retries the lookup. Loss-free: no
glazing-type key contains a wall-location phrase or a bare 1-2 digit number
(install-date years are 4 digits).
Unblocks the Summary cascade for any property with an Alternative Wall;
Summary-path only (the API path receives structured glazing codes, so the
API gauge is unaffected). Regression gate green (1 pre-existing fail
unrelated).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Elmhurst Summary section 15.1 "Hot Water Cylinder" block lodges
"Immersion Heater: Dual" / "Single"; the extractor dropped it, so the
Summary path left immersion_heating_type = None while the API path already
captured it. Capturing it drives SAP Table 13's high-rate-fraction
DHW-cost split (RdSAP 10 section 10.5 p.54: 1 = dual, 2 = single) and
brings the two front-ends to parity.
Three-file change: WaterHeating.immersion_type field +
_extract_water_heating parse (scoped to the 15.1..15.2 slice) +
_elmhurst_immersion_type_code mapper (strict-raise on an unmapped label,
mirroring _elmhurst_cylinder_insulation_code).
Safe to land now that the preceding commit zeroes the high-rate fraction
for 18-/24-hour tariffs: the 20 solid-fuel corpus certs (solid fuel 4-11:
WHC 903 dual immersion, 18-hour meter, 110 L) carry a dual immersion, but
their 18-hour tariff bills 100% low-rate per Table 12a's 7-/10-hour scope
— so they stay EXACT instead of regressing to the 10-hour-column ~0.10.
7-/10-hour Summary immersion certs now correctly cost the Table 13
high-rate fraction instead of falling to the immersion=None 100%-low
default.
Regression gate green (3 pre-existing fails unrelated); API gauge
unchanged (Summary-path-only): 57.6% within 0.5, mean|err| 1.185.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
pdftotext can wrap the §11 building-part column onto the glazing-TYPE
token without an intervening glazing-gap descriptor, e.g. "Double between
2002 and 2021 1st" (the "1st" marks the 1st Extension). The existing
trailing-gap fallback only strips the fragment when preceded by "N mm";
the bare ordinal raised UnmappedElmhurstLabel.
New `_ELMHURST_GLAZING_LABEL_TRAILING_BP_RE` strips a trailing ordinal
("1st"/"2nd"/…) or "Main" and retries the lookup. No glazing-type key
ends in an ordinal or "Main", so it is loss-free. Surfaced by worksheet
`simulated case 33` (direct-acting electric boiler + immersion), which
previously could not be routed through the Summary cascade.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`_separately_timed_dhw` returned True for any boiler+cylinder+from-main
cert, applying the SAP 10.2 Table 2b note b) ×0.9 temperature-factor
reduction unconditionally. For the lpg-boiler "before" worksheet (pre-
1998 LPG boiler SAP code 115 + 210 L cylinder, NO cylinder thermostat,
control 2113 "Room thermostat and TRVs" — no programmer) this dropped
the (53) temperature factor to 0.702 (= 0.60 × 1.3 × 0.9) where the
worksheet lodges 0.78 (= 0.60 × 1.3), under-counting cylinder storage
loss (55) by ~119 kWh/yr and over-rating SAP by ~0.25.
RdSAP 10 §10.5 (PDF p.57) "Hot water separately timed":
No programmer, pre-1998 boiler → No
Programmer, pre-1998 boiler → Yes
Post-1998 boiler → Yes
DHW is therefore NOT separately timed only when a pre-1998 boiler is
paired with a no-programmer control. Add the two SAP 10.2 Table 4c(2) /
Table 4b lookups (controls without a programmer = {2101, 2103, 2111,
2113}; pre-1998 gas/LPG boilers 110-119 + oil 124/125/128) and return
False for that combination; every other boiler+cylinder cert keeps the
separately-timed default, so the change is confined to old low-control
stock and the heating corpus + goldens are unchanged.
Effect: the full chain (Summary PDF → extractor → mapper → cert_to_inputs
→ calculator) now reproduces the lpg-boiler worksheet's §11a unrounded
SAP -6.6499 at abs < 1e-4 (was -6.4013). Full regression suite green bar
the 3 pre-existing unrelated fails.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An LPG-boiler dwelling on the Summary → from_elmhurst_site_notes path
mapped to main_fuel_type=26 (mains gas), making it indistinguishable
from a mains-gas boiler downstream — wrong Table 12/32 cost / CO2 / PE
(bottled LPG is ~10.30 p/kWh vs mains gas 3.48), and it defeats any
"non-gas → gas only with a mains-gas connection" gate (an LPG dwelling
looks already-gas).
Root cause: the recommendation worksheets lodge the boiler carrier as
§15.0 "Water Heating Fuel Type: Bottled gas" (§14.0 carries only SAP
code 115, a Table 4b gas-family row, + "Main gas: Yes" in §14.2 — a
mains-gas CONNECTION, not the heating fuel). "Bottled gas" was absent
from `_ELMHURST_MAIN_FUEL_TO_SAP10`, so the §15.0 fuel resolved to None
and `_elmhurst_gas_boiler_main_fuel` fell through priority-1 to the
mains-gas meter flag → 26.
Map "Bottled gas" → 3 (bottled LPG MAIN heating): code 3 routes via
`API_FUEL_TO_TABLE_32`/`API_FUEL_TO_TABLE_12` → Table-code 3 (10.30 /
9.46 p/kWh). NOT the legacy "LPG bottled": 5 entry — API code 5 =
anthracite, and `canonical_fuel_code` resolves the same-valued Table-32
code 5 to anthracite (3.64 p/kWh), so a 5 here mis-prices the dwelling
as cheap solid fuel (verified: a 5 mapping moved SAP the WRONG way,
42.33 → 45.11; code 3 moves it to -6.40 vs the worksheet's -6.6499).
Also add 3 to `_GAS_LPG_MAIN_FUEL_CODES` so the §15.0-lodged bottled-LPG
water fuel is adopted as the boiler's space-heating carrier (priority 1)
instead of the meter flag.
Effect: main_fuel_type=3 (bottled LPG) and water_heating_fuel=3 (was
None). Mains-gas certs still → 26 (full regression suite green bar the 3
pre-existing unrelated fails); the MissingMainFuelType tripwire still
fires for genuinely-undeterminable carriers.
Spec: SAP 10.2 Table 12 / RdSAP 10 Table 32 (PDF p.95) — bottled LPG
main heating fuel code 3.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`_elmhurst_dwelling_type` derived a flat's roof exposure from
`room_in_roof is not None`, so a top-floor flat whose roof is a plain
external "PS Pitched, sloping ceiling" (no room-in-roof) fell through to
"Mid-floor flat". The cascade's `_dwelling_exposure` then treats a
mid-floor flat's roof as a party ceiling (RdSAP 10 §5 / §3 — party
surfaces carry no heat loss) and drops the entire roof term: cert
001431's 105 m² roof at U=2.3 = 241.68 W/K (30) vanished, collapsing
(33) fabric heat loss 320.06 → 78.38 and over-rating SAP by ~5 points
(on top of the age-band roof-U bug — see prior commit).
Read the roof TYPE instead — the dual of the floor's "Another dwelling
below" signal. A flat's roof is a party ceiling only when its Elmhurst
code is S / A / NR (Same/Another dwelling or Non-residential space
above); F / PN / PA / PS are exposed external roofs, so the dwelling is
on the top storey. `has_exposed_roof = room_in_roof present OR
_elmhurst_roof_is_exposed(roof)` — which is exactly what the function's
own docstring already described as the intent ("RR present or external
roof"), now implemented.
With both upstream fixes the full chain (Summary PDF → extractor →
mapper → cert_to_inputs → calculator) reproduces the worksheet's §11a
unrounded SAP 56.3649 at abs < 1e-4, with (30)/(33)/(37) matching to
the decimal. Only flat fixture reclassified; 000784 (top-floor, RR) and
000910 (ground-floor) unchanged. Regression suite green bar the 3
pre-existing unrelated fails.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Elmhurst Summary §3.0 "Date Built" lodges the per-building-part age
bands; the Main row reads "Main Property" / "C 1930-1949". But "Main
Property" ALSO heads the §4.0 Dimensions table, so the global
`_str_val("Main Property")` collides with it: when pdftotext renders
"3.0 Date Built:" glued onto its "Main Property" row token on one
layout line (as the recommendation worksheets do), the first standalone
"Main Property" match is the §4 dimensions header — returning its next
token "Floor" as the "age band".
That garbage age propagated to `u_roof`: for a "Pitched, sloping
ceiling" (PS) roof with no lodged insulation thickness, `u_roof` returns
the spec uninsulated U=2.3 for the correct age C but U=0.4 for the
unparseable "Floor" — collapsing the roof heat-loss term and inflating
SAP by ~14 points on the affected cert.
Scope the read to the Date-Built block (between "3.0 Date Built" and
"4.0 Dimensions") and take the first age row — a line beginning with a
single A-M band letter + space ("C 1930-1949", "A before 1900",
"J 2003-2006"). Building-part name rows never start that way, and the
Main row precedes any extension / room-in-roof rows.
Regression: full sap10_calculator + documents_parser suite green bar the
3 pre-existing unrelated fails (2 stone-wall U tests, test_total_floor_
area); the multi-bp / "A before 1900" fixtures (000516, 001431_case*,
6035) keep their age bands.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
