5 certs in a 2026 API sample raised `KeyError: 'rr_common_wall_area_m2'`
and were blocked from computing. Root cause: `_part_geometry`'s early
return (taken when a building part lodges no sap_floor_dimensions —
e.g. a party-wall-only or RR-only extension as bp[0]) returned only 6 of
the 9 keys the full return exposes, omitting rr_common_wall_area_m2,
rr_gable_area_m2 and cantilever_floor_area_m2. The §3.9 RR contribution
block reads geom["rr_common_wall_area_m2"] / ["rr_gable_area_m2"] for
EVERY part, so the floorless part's truncated dict raised KeyError at
heat_transmission.py:974.
Fix: the early return now exposes all 9 keys, the three RR/cantilever
geometry values defaulting to 0.0 — correct, since a part with no floor
dimensions has no derivable RR shell or cantilever (no floor area).
Pure contract-completion bug; no spec/U-value change.
Regression test pins the invariant directly: a floorless part's
_part_geometry keys must equal a with-floors part's keys. Validated: all
5 certs now compute (4 within ~2 SAP of lodged; the 5th, 8536, has a
separate residual). §4 suite 2393 passed; heat_transmission.py pyright
unchanged at 12, test file at 71.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Slice 2b. Timber frame (wall_construction=5) takes internal wall insulation but
not external (not constructable — ADR-0019), so the generator offers IWI only.
Cascade pin: the IWI Option reproduces the re-lodged timber-frame after at
abs(diff) <= 1e-4 (general Table 6 insulation-thickness bucket, not the solid-
brick documentary path).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Slice 2a. New recommend_solid_wall emits one Main-wall Recommendation carrying
External + Internal wall-insulation Options for an uninsulated (wall_insulation
_type=4) solid-brick (wall_construction=3) main wall, each priced at the heat-
loss wall area. Cascade pin: the generator's EWI and IWI Options reproduce
their respective re-lodged afters at abs(diff) <= 1e-4.
Detection keys on wall_construction code, not description (ADR-0019 note
corrected): the Elmhurst ingestion path leaves walls[].description empty, so
the code is the only cross-path signal; codes 1-5 are consistent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
above) → None
The 2026 sample lodges roof_construction=6 (1 cert, "Thatched, with
additional insulation") and =7 (6 certs, "(same dwelling above)" /
"(another dwelling above)"), both raising UnmappedApiCode and blocking
the cert. roof_construction_type is read ONLY for the §3 "sloping
ceiling" cos(30°) inclined-surface factor (Slice 89); the base roof
U-value comes from the global roofs[].description. Neither code is a
sloping ceiling:
- 6 = thatched — U set by the description, not this field;
- 7 = same/another dwelling above — an internal ceiling with no roof
heat loss (the roof-side analogue of floor_construction code 0,
governed by the roof_heat_loss / description path).
Map both to None: carries no information the cascade consumes here and
correctly avoids the cos(30°) false-trigger. Empirically inert and
validated — roof W/K is byte-identical whether 6/7 map to None or to an
explicit pitched string across all code-6/7 certs in the sample. 5 of
the 7 now compute (e.g. thatched cert 2276 SAP 62.8 vs lodged 63); the
other 2 also carry a gable_wall_type 2/3 raise (separate, worksheet-
backed slice).
Dict value type widened to Optional[str]. §4 suite 2392 passed; mapper.py
pyright unchanged at 32; new tests suppress reportPrivateUsage (net-zero).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Captures the grill-with-docs session for solid-wall insulation: CONTEXT.md
gains EWI/IWI Measure Types + the Wall Insulation Eligibility rule (+ a
flagged-ambiguity that the three planning flags stay distinct, never recollapsed
to legacy restricted_measures). ADR-0019 records the eligibility policy (cavity
-> cavity only; brick/system -> IWI+EWI; timber -> IWI only; cob/stone -> none;
conservation/flat block EWI, listed/heritage block both). ADR-0020 records
conservation/listed/heritage as three distinct Property attributes sourced by
extending the geospatial S3 repo (flags co-located with lat/long).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Slice 1 of solid-wall insulation. BuildingPartOverlay gains a
wall_insulation_thickness field; the generic applicator already folds it onto
SapBuildingPart by name. With wall_insulation_type=1 (EWI) / 3 (IWI) + 100 mm,
the calculator derives the post-insulation U-value (§5.8 documentary path,
λ=0.04 default) — and for IWI also lowers the thermal-mass parameter. Two new
Elmhurst before/after cascade pins (solid-brick EWI + IWI, cert 001431)
reproduce the re-lodged after at abs(diff) <= 1e-4 across SAP/CO2/PE.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A 2026-register cert (4519-9056-4002-0222-4802) omits the top-level
post_town entirely (its town sits only in address_line_3 "BARNSTAPLE").
RdSapSchema21_0_x declares post_town as a required no-default field, so
from_dict raised "missing required field 'post_town'" and blocked the
whole cert from computing.
post_town is address metadata the SAP cascade never reads (no consumer
in domain/sap10_calculator/), so default an absent post_town to "" in a
from_api_response pre-processor (mirroring _normalize_shower_outlets) —
inert for the calculation, keeps the cert mappable. The schema dataclass
can't simply give post_town a default: it is a plain (non-kw_only)
dataclass with 57 required fields after post_town, so a mid-list default
would break field ordering.
Validated: cert 4519 now maps (post_town="") and computes SAP cont 74.68
vs lodged 75. §4 suite 2392 passed; mapper.py pyright unchanged at 32;
new tests suppress reportPrivateUsage (net-zero).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 2026 sample's second-largest mapper raise: 37 certs lodge
sap_floor_dimensions.floor_construction=0, which raised UnmappedApiCode
and blocked the cert. Code 0 is the "not recorded / not applicable"
sentinel — 33/37 pair it with floor_heat_loss=6 ("another dwelling
below", an upper-floor flat with no ground floor to describe); the rest
carry mixed Solid / unheated-space descriptions. There is no single
construction to assert.
Map code 0 → None, which defers to RdSAP 10 Table 19 ("where floor
construction is unknown" → age-band default) — identical to how an
unlodged floor_construction (the 993 None certs) is already handled, and
honest about the absence (cf. the no-misleading-insulation_type rule).
Empirically inert and validated: across all 37 code-0 certs the cascade
floor W/K is byte-identical whether code 0 maps to None or to an explicit
"Solid" string — the another-dwelling-below floors compute to 0.0 W/K
(handled via floor_heat_loss + property_type=Flat + floors[].description,
per the _API_FLOOR_HEAT_LOSS_TO_FLOOR_TYPE code-6 note), and the few
genuine ground/unheated floors hit the same age-band default either way.
All 37 now compute (were raising).
Dict value type widened to Optional[str] for the None entry; helper
already returns Optional[str]. §4 suite + schema-mapper tests green
(pre-existing test_total_floor_area failure unrelated); mapper.py pyright
unchanged at 32; new test suppresses reportPrivateUsage (net-zero).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A random 1000-cert Jan–May 2026 EPB-register sample surfaced 53 certs
lodging sap_floor_dimensions.floor_construction=3, which raised
UnmappedApiCode and blocked the whole cert from computing (~44 of the
sample's mapper raises). RdSAP 10 field 3-1 "Floor construction"
enumerates the lowest-floor construction as solid / suspended timber /
suspended, not timber, and the spec's "Suspended not timber (structural
infiltration 0)" makes the split load-bearing.
Map code 3 to the canonical "Suspended, not timber" string (the same
value the site-notes mapper already emits — cross-mapper parity):
- u_floor takes the suspended BS EN ISO 13370 branch via the
"Suspended" prefix (_floor_is_suspended_from_description), and
- _has_suspended_timber_floor_per_spec's exact-match
`!= "Suspended timber"` gate correctly does NOT fire, so the §5 (12)
0.1/0.2 floor-infiltration adjustment is skipped (structural
infiltration 0) — exactly the spec rule for not-timber suspended.
Validated: all 5 sampled code-3 certs now compute (e.g.
0340-2877-5570-2606-5965 floor_construction_type="Suspended, not
timber", SAP cont 60.12 vs lodged 60). Confirmed against the cert's own
global floor descriptions ("Suspended, …", floor_heat_loss=7).
Code semantics established from the RdSAP 10 spec + the lodged certs'
human-readable floor descriptions (the EPB /api/codes endpoint carries
no floor_construction enum). §4 suite + schema-mapper tests green
(the pre-existing test_total_floor_area failure is unrelated). mapper.py
pyright unchanged at 32; new test suppresses reportPrivateUsage to keep
net-zero new errors.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Record S0380.218 shipped, bump HEAD/next-slice, note both certs are
0-residual cross-validated golden fixtures and flag the optional
Summary-path regression guard as the cheap follow-up.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two certs fetched fresh from the GOV.UK EPB register, each with an
Elmhurst Summary PDF (input) and a dr87 worksheet PDF (the (1)..(286)
ground truth):
0340-2467-9260-2006-6521 (Summary_000922 / dr87-0001-000922)
5500-5070-0822-0201-3663 (Summary_000920 / dr87-0001-000920)
Both run through BOTH front-ends — from_api_response and
from_elmhurst_site_notes — and through the rating + demand cascades.
Cross-mapper parity holds: the two paths agree to <1e-4 on continuous
SAP, fuel cost, CO2 and PE. Both paths reproduce the worksheet exactly:
0340: (255) cost 776.4295, (272) CO2 2875.0498, (286) PE 16474.5616;
fabric (33) 171.6188, (37) 205.9358; SAP int 70 = lodged.
5500: (255) cost 751.8295, (272) CO2 2423.4547, (286) PE 14397.0118;
fabric (33) 141.1226, (37) 167.3696; SAP int 66 = lodged.
Pinned in two tables of test_golden_fixtures.py:
- _EXPECTATIONS / test_golden_cert_residual_matches_pin — SAP/PE/CO2
residual vs the integer-rounded lodged register (SAP resid +0 both).
- _WORKSHEET_PE_CO2 / test_golden_cert_pe_co2_matches_worksheet —
PE (286)/(4) and CO2 (272) vs the worksheet at +0.0000 (the
load-bearing 1e-4 check; lodged register is integer-rounded).
Dropped-field audit (raw JSON keys vs the schema-21.0.1 dataclass
fields consumed by from_dict) re-run on both fresh JSONs: no new
silently-dropped fields — only created_at metadata and the
shower_outlet_type/shower_wwhrs keys already handled by
_normalize_shower_outlets (mapper.py:2047). No calculator or mapper
change required; this is pure validation + regression-pinning.
Full §4 suite: 2392 passed, 1 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
fetch_epc_bulk_sample streams certificates-<year>.json out of the bulk ZIP via
range requests, keeps the first N SAP-version matches, and writes each cert's
inner document to <out>/<cert>.json for run_property_report. Stops after N, so
only the member prefix transfers, not the 15.7 GB archive (RangeFile.bytes_read
reports the true transfer vs the absolute ZIP offset). Verified on 2026: 100
SAP-10.2 certs -> report ran 81 scorable (MAE 2.03), 46 flagged, 19 raises
(11 full-SAP schema 19.1.0, 7 unmapped floor_construction 0/3, 1 missing
post_town) — real shadow-validation signal vs the curated golden 57.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The bulk endpoint 302-redirects to a 15.7 GB S3 ZIP with one NDJSON member per
year; each line wraps the per-cert payload in a stringified 'document' that
parses to the same RdSAP-Schema-21.0.1 shape from_api_response already handles.
parse_bulk_line unwraps a record; is_sap_version filters to SAP 10.2; RangeFile
exposes the S3 object as a seekable file so zipfile streams a single year's
member (and a sampler stops early) without downloading the whole archive.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Next-agent brief: fetch certs fresh from the EPC API (two new API+Summary+
worksheet triples for cross-mapper parity, plus six dashboard-flagged certs).
Flags the critical reconciliation: the user's flagged numbers don't match the
golden-fixtures cascade (0390-2954-3640 pinned +0 but flagged -6.85; 7536/2130
flags are pre-this-session), so fresh-raw-JSON-vs-curated-fixture or a
different engine must be reconciled before debugging. Documents the EPC API
fetch mechanism, the dropped-field audit method, this session's 4 fixes, and
the conventions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Second silently-dropped field from the 2130 audit: the schema-21
SapBuildingPart never declared `wall_insulation_thermal_conductivity`, so
`from_dict` discarded it. Captured it through schema 21.0.0/21.0.1 → domain
SapBuildingPart → API mapper, and wired it into u_wall's RdSAP 10 §5.8
documentary-evidence R-value calc (both the solid-brick §5.7/§5.8 path and
the cavity-composite path), replacing the bare 0.04 λ constant with a
resolved λ.
Resolver: absent / "Unknown" → the §5.8 default 0.04 W/m·K (mineral wool /
EPS); a mapped code → its λ; an unmapped integer code RAISES so the enum is
confirmed against a worksheet rather than silently mis-factored (same
incremental-coverage discipline as the glazing-type map). Only code 1
(= the default 0.04) is mapped — the sole observed value (cert 2130 Ext1).
Zero cascade effect today: the λ path fires only for solid-brick/cavity
walls with a *measured* wall thickness, and 2130 Ext1 lodges no wall
thickness, so its conductivity is captured-but-unused; all existing §5.8
certs lodge no conductivity → 0.04 default unchanged. The point is to stop
dropping lodged data and make λ correct when a future cert exercises it.
Suite: 2523 passed (1 pre-existing TFA fail); sap10_ml 237 passed (2
pre-existing stone-formula fails). Zero new pyright errors (46=46).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
run_property_report builds the three-section Markdown+CSV report over a dir of
API-shaped EPC JSON, offline (defaults to the golden 57: 57/57 scorable, MAE
0.54, 6 flagged |Δ|>0.5). fetch_epc_dump pulls raw cert JSON from the live API
by --uprn/--postcode (picking the latest cert per match, skipping existing
files), mirroring fetch_cohort2's proven HTTP shape and reading
OPEN_EPC_API_TOKEN. Report artifacts + epc_dump/ are gitignored.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
format_report_csv emits one comma-safe row per property: the calculator-error
fields (lodged/calculated/Δ/flag), the Plan headline figures (baseline+post
SAP/band, measures, cost+contingency, bill & CO2 savings, valuation %), the
flattened measure triggers, and any captured error — sortable in a spreadsheet
for a large dump.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
format_report_markdown emits: (1) cohort parity stats + a per-property
lodged-vs-calculated table flagging |Δ| > 0.5 (errors shown inline),
(2) Plans + costings (SAP/band jump, cost + contingency, bill & CO2 savings,
valuation uplift), (3) each fired measure with the EPC attributes that
triggered it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
build_property_reports models a dump in order (errors captured per-cert);
parity_report_for aggregates the lodged-vs-calculated SAP across the cohort
into the existing ParityReport (MAE/RMSE/bias/worst-N), excluding certs that
couldn't be mapped or scored. Residual convention is the calculator's own
(predicted - actual), the negative of PropertyReport.sap_error.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Section 3 of the report: build_property_report now runs the Modelling stage
and, for every Plan Measure, records the EPC attribute(s) that caused its
generator to fire (MeasureTrigger) — wall_construction/insulation for cavity
fill, roof thickness for loft, floor thickness/construction for floors, the
absent mechanical kind for ventilation. Modelling raises are captured as
plan_error, independent of the calculator-error capture.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Section 1 of the property inspection report: PropertyReport compares the
cert's lodged energy_rating_current to Sap10Calculator's un-rounded SAP and
flags |Δ| > 0.5 (the ADR-0010/0013 shadow-validation design target). A
mapping/scoring raise is captured per-cert as calculator_error, never
propagated, so one bad cert can't abort the sweep.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Updated 2130's pin notes with the case-18 finding: our cascade reproduces
the worksheet PV split to the decimal (gen 2684.17 / onsite 970.77 / export
1713.40), so the Appendix M1 β-split is exact, not the suspected bug. With
the gas PE factor also exact (1.13) and the wall measurement now wired
(S0380.215), 2130's +2/-11.72 is the irreducible API-only lodged residual
(0240-like), not a closable calculator bug. Notes-only; pin unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
pdftotext dumps of hand-entered Elmhurst worksheets wrap the §11 glazing-
GAP column ("16 mm or more") onto the glazing-TYPE token, yielding labels
like "Double between 2002 and 2021 16 mm or [1st]" that
`_elmhurst_glazing_type_code` didn't recognise → UnmappedElmhurstLabel,
blocking the whole Summary from parsing.
Added a fallback: when the lightly-cleaned label isn't a known key, strip a
trailing wrapped gap descriptor (`\s+\d+\s*mm\b.*$`) and retry. Applied
AFTER the direct lookup so explicitly-mapped interleaved variants (e.g.
"Double with unknown 16 mm or install date more", where the gap splits into
the middle) are unaffected. The gap drives the API-path U-value lookup, not
the site-notes glazing-type enum, so dropping it is loss-free for the
cascade.
Unblocks running our cascade on hand-entered worksheet Summaries — used to
validate the PV β-split against simulated case 18 (our split matches the
P960 worksheet exactly: gen 2684.17, onsite 970.77, export 1713.40).
Suite: 2391 passed, 1 skipped. Zero new pyright errors (mapper 32=32).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Next-phase handover: fetch live EPCs via EpcClientService, run the
offline Modelling harness, and save a per-property report covering
(1) lodged-vs-calculated SAP divergence (>0.5), (2) plans + costings,
(3) recommended measures + the EPC attributes that triggered them. Maps
the EPC API client (the user's blocker), the calculator-error ingredients
(parity_report scaffolding), and each generator's exact trigger fields.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The schema-21 SapBuildingPart never declared `wall_insulation_thickness_
measured`, so `from_dict` silently discarded it. When a cert lodges
`wall_insulation_thickness == "measured"` the actual value (mm) lives in
that dropped field, so the cascade fell back to the 50 mm "insulation
present, unknown thickness" default instead of the lodged measurement.
Cert 2130 Ext1 lodges solid brick band B + INTERNAL insulation
"measured"/100 mm. Per RdSAP 10 §5.7 Table 8 (insulated-wall U by age band
+ insulation thickness) the 100 mm row gives U=0.32; the unknown-thickness
fallback gave 0.55. New `_api_resolve_wall_insulation_thickness` substitutes
the measured value for the "measured" sentinel; the existing
`_insulation_bucket`/Table-8 path then computes the correct U. Field added
to schema 21.0.0/21.0.1 SapBuildingPart; domain field widened to
Union[str, int] to match `roof_insulation_thickness`. Isolated: 2130 Ext1
is the only bp lodging "measured" across all 47 fixtures.
This spec-correct fix EXPOSED an offsetting under-count it had been masking
(per the repo's no-special-handling rule — the pre-fix +1 was two bugs
cancelling): 2130 cont SAP 83.35 → 83.78 (resid +1 → +2), PE -7.56 →
-11.72, CO2 -0.045 → -0.095. The exposed -11.72 PE (~-746 kWh/yr) is the
deferred gas-combi-PE + PV-β-credit under-count from S0380.45/.49, now
un-masked — the next slice. Re-pinned 2130 with the cause documented.
Suite: 2391 passed, 1 skipped. Zero new pyright errors (mapper 32=32).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Apply the deep-research off-gas figures so oil/smokeless/wood sit on the
same NEP-Apr-2026 retail / DESNZ DUKES gross-CV basis as the new coal
proxy (fuel-input, not useful-heat): OIL 9.16 -> 12.11 (prior value was
materially low vs current kerosene), SMOKELESS 10.0 -> 8.69, WOOD_LOGS
8.83 -> 8.25, WOOD_PELLETS 7.99 -> 7.38. SEG (15.0, Solar Energy UK) and
LPG (17.61, bottled-propane) kept; gas/electricity (Ofgem cap) unchanged.
CV arithmetic recorded in the snapshot _assumptions. OIL pin updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Coal and heat networks have no national retail/cap rate, so the snapshot
left them null and BillDerivation raised UnpricedFuel — dropping those
certs from an offline cohort run. Add researched proxy rates (fuel-input
basis, sources + arithmetic in the JSON _note/_gaps): COAL 7.13 p/kWh
(NEP Nov 2025 coal uprated + DESNZ DUKES house-coal GCV) and HEAT_NETWORK
16.0 p/kWh + 69.4 p/day (Insite Energy operator sample; indicative, schemes
vary ~8-30). Both flagged proxy/indicative — sense-check estimates, not
market rates. Existing curated fuels are unchanged.
Replaces the unpriced-raises pin for these two with a positive rate pin;
off-peak stays unpriced pending the day/night accessor. Golden cohort now
runs 57/57 offline with zero errors.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
