A cert lodging its horizontal dimensions measured externally
(measurement_type=2) must have its floor area and exposed perimeter converted to
internal dimensions before any SAP use (RdSAP 10 §3.4 + Table 2, p.17-18).
measurement_type was read nowhere (the `# What is this?` field), so external
certs carried a TFA ~11-15% too large — inflating occupancy and understating
every per-m2 output (CO2/m2, PE/m2).
Plumb measurement_type onto EpcPropertyData, and add a post-mapping
_with_internal_dimensions step: compute ONE whole-dwelling area/perimeter ratio
from the ground-floor footprint via the Table 2 built-form equation (§3.4 Note 4
— not a per-storey conversion, which over-corrects multi-part mid-terraces),
apply it to every storey's area/perimeter, reduce party walls by 2w (Note 5),
and refresh total_floor_area_m2 (read for occupancy N). Wall thickness w is the
lodged main-dwelling value, else the Table 3 default (p.19). Room-in-roof floor
area is always internal (§3.1) and left untouched. Scoped to the RdSAP-Schema
reduced-data family (17.0-21.x) where per-storey external dims are the lodgement
standard; legacy SAP-Schema-15.0/16.x are plumbed but not converted.
Corpus: within-0.5 81.0% -> 81.2%, MAE 0.571 -> 0.570, PE 2.5 -> 2.4. Cert
100091435353 TFA 128.7 -> 111.6 (lodged 112), SAP 66.23 -> 65.33 (+1.2 -> +0.3);
mean |dSAP vs lodged| over the 12 external certs 0.556 -> 0.429. Closes#1669.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On the gov-EPC API path a window's metal frame was unrepresentable: both
transmission tables were frameless (U_pvc, g, 0.70), so a cert lodging a metal
frame (pvc_frame:"false", RdSAP input 6-5) was billed at the PVC/wooden U and
frame factor 0.70 — over-rating metal-framed dwellings.
Add the RdSAP 10 Table 24 metal U-column (PDF p.50-51; mirrors u_window's metal
values) and a pvc_frame dimension to _api_glazing_transmission: an explicit
"false" routes the window to the metal U + SAP 10.2 Table 6c frame factor 0.8;
"true"/None/other keep the PVC column + 0.70, so no PVC cert regresses.
Secondary glazing keeps its frame-independent U (only FF changes). Wired into
_api_sap_window and _api_sap_roof_window (both full-schema, lodge pvc_frame);
the reduced-field path has no pvc_frame and is unchanged.
Spec: RdSAP 10 Table 24 metal column, SAP 10.2 Table 6c, RdSAP input 6-5 (p.80).
Corpus: within-0.5 80.6% -> 81.0%, MAE 0.578 -> 0.571. Cert 21067296
68.60 -> 66.96 (toward lodged 67). Closes#1667.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The gov-EPC API lodges a two-main dwelling's main_heating_fraction pair in
two incompatible encodings — a decimal fraction summing to ~1.0 ([0.8, 0.2])
or an integer percent summing to ~100 ([80, 20]) — while every consumer
divides by 100 unconditionally. A decimal pair therefore collapsed the second
main's share to ~1% of its true value, billing almost the whole load to the
first system's fuel/efficiency.
Normalise once at the mapper via a shared pair-sum discriminator
(_normalised_main_heating_fraction): a genuine two-main split summing nearer
1.0 than 100 is scaled to percent; a percent pair and any single main are left
exactly as lodged, so the 3 percent-encoded corpus certs do not regress. Wired
across the seven from_rdsap_schema_* branches and the SAP-17.1 path, whose old
int(fraction) floor silently deleted a decimal-lodged second main (int(0.35)=0).
Spec: RdSAP 10 item 7-5 (percent is canonical), p.81. Corpus: within-0.5
80.3% -> 80.6%, MAE 0.584 -> 0.578. Example cert 10070086972 17.59 -> 18.73
(toward lodged 18). Closes#1666.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
RdSAP 10 item 2-3(h) "another dwelling above" is U=0 (spec p.45: "There is no
heat loss through the roof of a building part that has the same dwelling above
or another dwelling above"). gov-API `roof_construction = 3` is that code, but
the mapper labelled it "Pitched (slates/tiles), no access to loft" (which is
code 5), so the per-part party override at heat_transmission.py:1172 never
matched and the part fell through to the dwelling_type-label-only
has_exposed_roof heuristic.
190 of the 200 corpus code-3 parts sit at index 0 and are already suppressed by
`_cert_lodges_exposed_roof` off the roof_insulation_location == "ND" signal. The
unmitigated set is the 10 parts at index > 0, where extensions default to
exposed and were billed a phantom pitched roof. Also removes a latent PV defect:
the bogus "Pitched" prefix satisfied `is_pitched` and divided PV array area by
cos(35 deg), inflating kWp.
Shipped with the stairwell fix so the corpus pin moves once: within-0.5
78.8% -> 79.6%, SAP MAE 0.626 -> 0.611, PE MAE 2.9 -> 2.7.
The roof change alone trades 4 certs out of the +-0.5 band for 2 in (one moving
4.4 SAP closer). Those 4 were investigated and are NOT a masked bug: the corpus
error distribution is symmetric (mean +0.076, median +0.03, sigma 1.606; 112
certs above +0.5 vs 102 below), ground-floor flats show 7 positive / 8 negative
with no excess over the null, and closing them needs HLC changes of 2.3-5.6% —
below the ~8.8% noise floor of the RHI anchor, calibrated on the 76 certs whose
SAP is exact (space-heating ratio 1.062 +/- 0.088).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A mid-terrace / enclosed-mid-terrace (built_form 4/6) has external walls on
its two opposite (front/rear) faces only (RdSAP 10 §1.1, p.7-8); both end/
gable elevations — and their room-in-roof continuations — abut a neighbour
and are party walls (§1.4.3, p.9). The gov-API path is built_form-blind, so a
cert lodging an "Exposed" RR gable (`gable_wall_type=1`) on such a form had it
billed at the masonry main-wall U (~2.5) instead of the party 0.25 (Table 4,
p.22), over-counting fabric heat loss and under-rating the dwelling.
`_api_type_1_gable_kind` now takes `built_form` and reclassifies an Exposed RR
gable to Party on the both-ends-party forms {4, 6} only; end-terrace {3, 5},
semi {2} and detached {1} keep their genuine exposed gable. The API path lodges
no per-gable U, so this only ever overrides a cascade fallback.
Corroborated three ways: RdSAP §1.1 + Table 4; S10TP-05 Appendix A (a
mid-terrace's roof-gable junction length is zero); and three accredited
Elmhurst mid-terrace worksheets (000477/000480/000516), which all lodge Party
gables. Those worksheets run the site-notes path (already correct) so the 195
Elmhurst pins are untouched.
Corpus (RdSAP-21.0.1, 1000 certs): within-0.5 78.8% -> 79.5%, SAP MAE
0.625 -> 0.599, PE MAE 2.85 -> 2.71, CO2 MAE 0.070 -> 0.068. Cert
100040550095 -10.13 -> +0.37; the 18-cert terraced-external-gable cohort mean
-1.94 -> +0.11; end-terrace/semi/detached untouched. Thresholds ratcheted.
Tests: new test_mapper_rir_gable_terraced (9); 195 Elmhurst pins, 288
cert_to_inputs/real-cert pins, 54 mapper tests, and the corpus gauge all green;
no new pyright errors.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Properties 741843/741872 (uprns 100060714155/100021944594) failed
modelling_e2e subtasks 6ac7841a-9338-4efe-97e5-7e0d19b3055d and
c5450f03-5b0d-4068-a3b3-d1470bc0af57 with a bare StopIteration: their
SAP-Schema-15.0 (2011-era LIG-lodged) certs identify the main dwelling's
building part as "Main building" rather than "Main Dwelling", so
from_api_string fell to OTHER and left the property with no MAIN part —
crashing wall_recommendation.py's unguarded next(...).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
PRD #1385 phase 2. The 20.0.0 and 21.0.0 mappers dropped insulated_door_u_value
(top-level Optional float), unlike 21.0.1. heat_transmission reads it as the
measured door U-value override (cert_to_inputs -> heat_transmission:809), so
dropping it forced every door onto the RdSAP default U-value.
Verification: new test 3 passed (21.0.0 sample lodges 3; 20.0.0 latent -> inject
1.5 + absent -> None); corpus 6002 passed; SAP-accuracy regressions pass; pyright
unchanged (39 -> 39).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PRD #1385 phase 2. The 21.0.0 mapper dropped mechanical_ventilation_index_number
and mechanical_vent_duct_type, unlike 21.0.1. cert_to_inputs reads both (PCDB
Table 4f/329 heat-recovery/SFP lookup + the Table 329 in-use factor), so an
MVHR/MEV cert on the 21.0.0 path fell back to the SAP default efficiency.
Mirrors 21.0.1 exactly -- only the two fields it actually maps. The other four
mechanical_vent_duct_* schema fields have no mapper OR calculator consumer even
on 21.0.1 (verified), so they're deliberately left alone rather than populating
dead domain fields.
Verification: new test 1 passed (harvested 21.0.0 sample lodges index=12,
duct_type=3); corpus 6002 passed; SAP-accuracy regressions pass; pyright
unchanged (39 -> 39).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PRD #1385 phase 2. 21.0.0 was the last mapper still dropping the lodged
percent_draughtproofed (top-level int) — every other path (17.0/…/21.0.1) reads
it. Dropping it understated the SAP §2 draughtproofing/infiltration credit for
21.0.0 certs.
Verification: new test 1 passed (harvested 21.0.0 sample lodges 100); corpus
6002 passed; SAP-accuracy regressions pass; pyright unchanged (39 -> 39).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PRD #1385, building-part fabric family. The 17.0 mapper hardcoded
floor_insulation_thickness / sap_room_in_roof to None and omitted
flat_roof_insulation_thickness, all of which the 17.0 schema lodges and the
19.0/20.0/21.x paths read:
* floor_insulation_thickness: "NI" (no measured thickness) -> None so u_floor
reaches its age-band default, else pass through (mirrors the 19.0 note).
* flat_roof_insulation_thickness: pass through (was dropped).
* sap_room_in_roof: map to SapRoomInRoof(floor_area, construction_age_band),
None if absent (mirrors 19.0).
17.0-only: the 17.1 SapBuildingPart schema has none of these fields, so its None
there is correct (verified against the dataclass). roof_insulation_thickness is
intentionally left raw -- the sloping-ceiling resolve helper moves roof U-values
and is a separate follow-up.
Verification: new test 3 passed; test_mapper_corpus 6002 passed; SAP-accuracy /
RealCertExpectation regressions pass; pyright unchanged (39 -> 39, pre-existing).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PRD #1385, shower/WWHRS family. The 17.0 and 19.0 API mappers left
sap_heating.number_baths / mixer_shower_count as None, so the water-heating
demand (cert_to_inputs) fell back to defaults instead of the lodged bath/shower
room counts. Derive them from schema.sap_heating.instantaneous_wwhrs, mirroring
the 17.1/18.0/20.0 paths:
number_baths = rooms_with_bath_and_or_shower + rooms_with_bath_and_mixer_shower
mixer_shower_count = rooms_with_mixer_shower_no_bath + rooms_with_bath_and_mixer_shower
Guarded on instantaneous_wwhrs is not None (Optional in the schema), like 17.1.
Counts, same meaning across the reduced-field family.
Verification: new test 2 passed; test_mapper_corpus 6002 passed; SAP-accuracy /
RealCertExpectation regressions pass; pyright unchanged (39 -> 39, pre-existing).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PRD #1385, heating-controls family. The 17.0 API mapper omitted
main_heating_controls entirely, leaving the domain field None while the
19.0/20.0/21.x paths map the first lodged control via _map_energy_element. 17.0
lodges main_heating_controls (List[EnergyElement]) in all 23 local fixtures.
The mapped element is the same EnergyElement type 17.0 already carries for
main_heating / window, so this is straight parity (not new logic). Guarded on a
non-empty list -> None, matching the siblings.
Verification: new test 2 passed; test_mapper_corpus 6002 passed; recommendation
+ heating-control tests 85 passed (the domain EnergyElement field is distinct
from the subscriptable dict HeatingControlRecommender reads via the Property
wrapper, so no recommender destabilisation); SAP-accuracy regressions pass;
pyright unchanged (39 -> 39, pre-existing).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PRD #1385, lighting family. The 17.0 and 19.0 API mappers hardcoded every
fixed-lighting bulb count to 0, dropping the lodged lighting mix that the
reference-complete 17.1/18.0/20.0 paths read. The schema lodges total +
low-energy OUTLET counts (ADR-0028); mirror the reference split — low-energy
outlets → low_energy bulbs, the remainder → incandescent — so the SAP
lighting-energy calc sees the real low-energy credit instead of assuming none.
led/cfl stay 0 (reduced-field certs don't split those); 21.0.x use a different
bulb-based lighting schema and are out of scope. The `total - low_energy`
subtraction is safe: all 37 local 17.0/19.0 fixtures have low_energy <= total
(no negative incandescent), matching the reference paths that already do this
unclamped.
Verification: new test 2 passed; test_mapper_corpus 6002 passed; SAP-accuracy /
RealCertExpectation regressions 56 passed (no pinned shifts); pyright unchanged
(39 -> 39, pre-existing).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PRD #1385, ventilation family. The 17.0 and 19.0 API mappers dropped lodged
ventilation fields that the reference-complete 17.1/18.0/20.0 paths read:
* open_fireplaces_count -> open_chimneys_count: hardcoded 0 on both
(understating §2 infiltration).
* percent_draughtproofed: omitted on both.
* 17.0 additionally passed no sap_ventilation block, so the §2 cascade fell
back to NATURAL + its default sheltered_sides=2. Restored from the lodged
mechanical_ventilation + built_form via the existing _api_* helpers.
Semantic parity verified (not just type-equality): built_form and
mechanical_ventilation use the same gov code lists the 17.1/18.0/19.0/20.0/21.x
paths already map through these helpers; all 244 RdSAP fixtures resolve with zero
UnmappedApiCode, and the strict-coverage guard raises on any future divergence.
For the current corpus the 17.0 score-mover is sheltered_sides from built_form
(every 17.0 cert lodges mechanical_ventilation=0 -> NATURAL).
Adds a regression test asserting both paths carry the fields through
from_api_response on real certs.
Verification: new test 2 passed; test_mapper_corpus 6002 passed; SAP-accuracy /
RealCertExpectation regressions pass; pyright unchanged (39 -> 39, pre-existing).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`from_rdsap_schema_17_0` was an older copy that hardcoded a cluster of
lodged `sap_heating` fields to None while every other API path
(19.0/20.0/…) reads them from `schema.sap_heating`. PR #1380 restored the
secondary-heating pair; this restores the rest of the same block, all of
which `RdSapSchema17_0` lodges:
MainHeatingDetail: boiler_flue_type, fan_flue_present,
central_heating_pump_age, main_heating_index_number (PCDB efficiency index)
SapHeating: cylinder_thermostat, cylinder_insulation_thickness
Impact — cylinder_thermostat (12/23 local 17.0 fixtures) read as "no
thermostat", so the water-heating worksheet over-applied its ×1.3 penalty
and `recommend_cylinder_thermostat` could fire on a dwelling that already
has one; cylinder_insulation_thickness (13/23) discarded the measured value
so the cylinder-loss cascade fell back to age-band defaults;
main_heating_index_number lost the PCDB efficiency lookup.
Mirrors the 19.0 path exactly. secondary_*_type left untouched so this PR
stays independent of #1380 (no overlapping lines). Adds a regression test
asserting the whole block survives `from_api_response` on a real 17.0 cert.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The `from_rdsap_schema_17_0` path hardcoded `sap_heating.secondary_heating_type`
and `secondary_fuel_type` to None, unlike every other API path (19.0/20.0/…),
which read them from `schema.sap_heating`. The gov EPC API *does* populate a
secondary heating SAP code for a genuine fixed secondary (e.g. 691 "electric
room heaters", 694), so the hardcode silently discarded it.
Because `recommend_secondary_heating_removal` gates purely on
`sap_heating.secondary_heating_type`, any 17.0 cert carrying a fixed secondary
would never get its removal recommendation. Surfaced while auditing portfolio
814's recommendations; 814 has 17.0 certs but none with a fixed secondary, so
the defect was latent there — this restores parity with the other paths so it
cannot bite.
Mirrors the 19.0 path exactly (secondary_fuel_type via `_api_secondary_fuel_type`).
Adds a regression guard that injects a fixed secondary into a real 17.0 cert and
asserts the code survives the mapping.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_is_elmhurst_roof_window mis-routes a vertical window (Location "External
wall") to sap_roof_windows when the building part's roof is a PARTY ceiling
(A "Another dwelling above" / NR "Non-residential space above"). A party
ceiling has no external roof, so it cannot host a rooflight.
Surfaced by simulated case 53 (cert 000565 re-keyed as a mid-floor electric-
storage flat, roof "A Another dwelling above"): its External-wall window
(U 2.00) routed to sap_roof_windows -> window area not deducted from the wall
(wall over-counted ~7 W/K) + priced as a roof window -> our SAP 74.0 vs
Elmhurst worksheet 75. Elmhurst lodges it Type "Window", Location "External
wall".
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A SAP-16.x cohort cert (9258-4062-7265-2844-7954) lodges a bare int building-
part identifier (the second part as `1` after "Main Dwelling").
`BuildingPartIdentifier.from_api_string` regex-matched it assuming a string and
raised "TypeError: expected string or bytes-like object, got 'int'", failing the
whole property.
Fix: guard the match on `isinstance(api_identifier, str)` so a non-string
identifier falls to OTHER, matching the documented "anything unrecognised ->
OTHER" contract. The baseline SAP fabric sums all building parts regardless of
identifier, so OTHER is SAP-neutral; the identifier only labels parts for
measure targeting. Fixes every mapper (all route through from_api_string).
Cert now maps + calculates (sap 63). Regression test added.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The full RdSAP-Schema-21 label is "Double between 2002 and 2021" (double
glazing installed 2002-2021, SAP 10.2 Table 24 code 3). When the Elmhurst
Summary PDF wraps the trailing "and 2021" into an adjacent table cell that
the extractor joins away, the surfaced label truncates to "Double between
2002" — the same artifact already handled for "Triple post or during".
`_elmhurst_glazing_type_code` raised UnmappedElmhurstLabel on it, blocking
the whole Summary (surfaced on the simulated-case-46 multi-attribute
worksheet). Added the truncated form as a code-3 alias.
pyright not installed in this container.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Elmhurst Summary §15.1 lodges "Cylinder Size: Value known" with the
measured volume in the "Cylinder Volume (l)" line — the Summary-path
equivalent of the gov-API "Exact" descriptor. The mapper had no entry for
"Value known" so `_elmhurst_cylinder_size_code` raised UnmappedElmhurstLabel,
and even once mapped the measured volume was never threaded through, so the
cascade dropped the cylinder storage loss (~468 kWh/yr) from (219) water
heating on every measured-volume-cylinder Summary.
Per RdSAP 10 §10.5 Table 28 (p.55) a measured cylinder volume is used
directly. Map "Value known" → cascade code 6 (Exact) and thread the §15.1
"Cylinder Volume (l)" value into SapHeating.cylinder_volume_measured_l, which
`_cylinder_volume_l_from_code` (cert_to_inputs.py:5281) already reads for
code 6 — mirroring the gov-API path (mapper.py:1575/1885).
Pins simulated case 39 (P960-0001-001431): an age-A mid-terrace on direct-
acting electric room heaters (SAP code 691, cat 10, control 2602) with
electric-immersion DHW off a 117 L "Value known" cylinder. The full
extractor→mapper→calculator cascade now reproduces the worksheet's SAP-rating
block EXACTLY — SAP value 36.6365 (band F) and (272) CO2 2056.0731 kg/yr,
with (219) water heating 2637.5049 and (255) total energy cost 1802.0039.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Elmhurst §15.1 "Insulated: No Insulation" label was lodged but absent
from `_ELMHURST_CYLINDER_INSULATION_LABEL_TO_SAP10`, so
`_elmhurst_cylinder_insulation_code` raised UnmappedElmhurstLabel — blocking
the parse of every Solar PV example cert (the solar `before` lodges "No
Insulation"). An uninsulated cylinder has no insulation *type*, so per the
no-misleading-insulation convention it maps to `cylinder_insulation_type =
None` rather than naming a material; the lodged 0 mm thickness carries the
storage-loss signal the SAP 10.2 Table 2 dispatch needs.
Slice 1 of the Solar PV Recommendation Generator (ADR-0026).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Slice 3a. The inverse of Epc.from_sap_score: the minimum SAP rating in a
band (C → 69, B → 81, …), used as the Optimiser's repair target for an
INCREASING_EPC goal (goal_value "C" → target SAP 69). Keeps the
band-target derivation in the domain rather than re-coupling to
backend.app.utils.epc_to_sap_lower_bound. 8 tests incl. round-trip
through from_sap_score; pyright strict clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
