A water-only community scheme (water_heating_code 950/951/952) is served
by its own heat network, independent of the space main. _water_heating_fuel_code
now resolves the network's SAP Table 12 community fuel (e.g. 41 = heat from
electric heat pump @ 4.24 p/kWh, COP already priced into the row) instead of
defaulting to the space main's fuel. For a direct-electric main this had
mis-billed communal heat-pump DHW at the 13.19p standard-electric rate.
Dovestone Apt 4 (cert 9556-3047-3304-5355-1200): hot-water cost £353 → £113,
SAP 67 → 80.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DX8oAGsGkBHq3U4dsxYRzz
Water-only community hot water (water_heating_code 950/951/952,
community_heating_use 2) carries its own heat source in
sap_community_heating_systems, independent of the space main. Expose the
network's heat-fraction-weighted efficiency (COP×100) + Table-12 fuel on
SapHeating so the calculator can price DHW on the network instead of
dropping it to a direct-electric immersion cylinder.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DX8oAGsGkBHq3U4dsxYRzz
Six properties failed task 4d006fab (portfolio 854 / scenario 1334) with
DegeneratePredictionError. None has a lodged EPC, so each was predicted from a
neighbouring-cert cohort — and every cohort cert lodges its single building part
as `identifier: "complete"`. from_api_string accepted only "Main Dwelling" /
"Main building", so "complete" fell to OTHER, _has_main_part returned False, and
prediction refused to proceed.
"complete" is what SAP 9.91/9.92-era software writes for an unextended dwelling:
semantically identical to "Main Dwelling". Casing varies within a single postcode
(cert 8824-7422-1180-6934-1902 lodges "complete", 199 Highfield Road "Complete"),
so matching is now case-folded.
The API's `identifier` is free text with no schema enum — api.yml declares the
whole cert body `additionalProperties: true` and documents no field of it, and
`identifier` is absent from the 17 vocabularies at /api/codes. So the recognised
set can only grow by discovery, and each miss has cost an incident ("Main
building" → task a40e71c4; "complete" → task 4d006fab). Hence three changes, not
one:
- "complete" → MAIN, case-insensitively.
- Bare "Extension" → EXTENSION_1. The regex required a digit, so 668 corpus
occurrences silently dropped a real extension from the structure. This reverses
a prior deliberate pin whose stated rationale (RdSAP10 §1.2's 4-extension cap)
does not apply — there is no out-of-range number in a bare "Extension".
- _anchor_lone_building_part: a cert lodging exactly ONE part describes the whole
dwelling whatever it is called, so an unclassified lone part becomes MAIN.
Multi-part certs are untouched — there the identifier carries real information
and guessing would invent structure the cert does not state.
Also removes a false claim from DegeneratePredictionError's message: it asserted
the template was "lodged with a null part identifier", which was hardcoded, never
checked against a real cert, and wrong. It misdirected this investigation.
Verified against the live gov API: all 13 cohort certs across BL1 8EB, BL3 6XJ
and BL4 0RA now resolve a MAIN part. The full modelling e2e was NOT run (no local
AWS creds for the geospatial lookup) — this clears the blocking error, it does not
prove the predictions are good.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A PasHub survey can lodge a manufacturer's measured cylinder loss ("What is
the cylinder measured heat loss: 0.94 kWh/24 hours") — the SAP 10.2 §4
(48)-(50) declared-loss factor that must override the Table 2 V×L×VF
insulation computation. The site-notes path dropped it two ways: the
extractor looked for "Cylinder Measured Heat Loss:" (matches 0 fixtures) while
the PDF label is "What is the cylinder measured heat loss:" (92 fixtures), and
the mapper never wired `cylinder_heat_loss` even when captured. So the
calculator fell back to the age-band insulation default — a lossier cylinder
than surveyed — over-costing hot water and under-rating SAP.
Fix the extractor label and add `_pashub_cylinder_measured_heat_loss` (parses
the leading float, returns None for "Not known"), mirroring the gov-API and
Elmhurst paths that already pass `cylinder_heat_loss` through. 3 Woodmans
Court -1.33 → -0.09; LRHA cohort within-0.5 58.3% → 60.2%, MAE 0.515 → 0.496.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Completes the community taxonomy: CHP (302) carries its chp_fraction and
boiler fuel for the SAP 302 worksheet; heat pump (304) rides the existing
fuel/code path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Derive 301 (boilers) / 302 (CHP) / 304 (heat pump) from the lodged
heat_source_type mix and set it on the category-6 main.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Full SAP lodges the heat-network fuel in sap_community_heating_systems (which the
schema dropped); resolve the dominant heat source's Table 12 fuel onto the
category-6 main so the engine prices it as a network, not electricity.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A dwelling with two main heating systems lodges a "Main Heating 2" block and a
"Percentage of space heating provided by Main Heating 1" split (e.g. 20% fan +
80% slimline storage, or 50% storage + 50% panel heaters). The survey model held
a single `main_heating`, and the extractor read the whole §Heating section
first-match — so it captured only Main 1, modelled it as 100% of the load, and
dropped Main 2 and the split (also bleeding Main 2's fields into Main 1). Every
such dwelling over-rated.
- `HeatingAndHotWater` now holds `main_heating_2` + `main_heating_1_percent`.
- The extractor slices the "Main Heating Systems" region per system on the
"Main Heating N" markers and reads Main 1's percentage; a single-main dwelling
(no "Main Heating 2") parses byte-identically to before.
- `_map_sap_heating` factors the per-system detail into `_pashub_main_heating_
detail(main, fraction)` and emits two `MainHeatingDetail`s, setting Main 2's
`main_heating_fraction = 100 - Main 1 %` (the calculator reads Main 2's share
and gives Main 1 the remainder).
5 of the 6 dual-main dwellings converge (within-0.5 53.4% → 55.3%, MAE 0.799);
single-main fixtures and the gov-API / Elmhurst / Guinness / Wythenshawe corpora
are unaffected. The 6th (497655371974, storage+panel) regresses because the
calculator deliberately bills an electric Main 2 at Main 1's rate
(`_main_2_space_heating_fuel_cost_gbp_per_kwh`, deferred §10a off-peak slice) —
the correct two-main mapping now exposes that calculator gap; tracked separately.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A PasHub Manual-Entry electric boiler lodges "Direct acting" in its "Heating
System (Boiler):" age-band cell. `_PASHUB_MANUAL_BOILER_AGE_BAND_TO_TABLE_4B`
covers only gas/liquid Table 4b rows, so the descriptor resolved to None and the
boiler stayed uncoded — taking the generic 0.80 gas-boiler seasonal efficiency
AND, on a Dual meter, `_table_12a_system_for_main`'s None-fallback that bills the
whole main-heating load at the off-peak LOW rate. The two errors pull opposite
ways, so an identical mis-mapping over-rated one dwelling and under-rated another
(fixture 497712825571 +9.5 on a Dual meter; 461386632387 -6.2 on a Single meter).
A direct-acting electric "boiler with radiators" is SAP 10.2 Table 4a code 191
(direct-acting electric boiler, efficiency 1.00, §12 Rule 3 tariff). Code it
directly in the manual-boiler resolver (gated on fuel Electricity + no PCDB
product), so the single code value drives the efficiency table, the Table 12a
pricing row and the tariff dispatch together. Both fixtures converge:
497712825571 → +0.6, 461386632387 → +0.4. LRHA within-0.5 52.4% → 53.4%,
MAE 0.835. Renamed `_pashub_manual_boiler_table_4b_code` → `_pashub_manual_
boiler_code` (it now also returns the Table 4a electric code).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Clears the three non-table-add blocks tracked in #1639 plus a latent age-band
bug they surfaced. LRHA WAVE 3 cohort: 77 → 103 of 104 computing.
Block A (exact cylinder volume): a measured cylinder lodges its litres on a
separate "Exact cylinder volume:" line; parse it in the extractor, carry it on
WaterHeating.cylinder_volume_measured_l, route the "Exact cylinder volume" band
to SAP cylinder-size code 6, and thread the litres into SapHeating (the
calculator reads them only when the code is 6). Verified end-to-end on the real
166 L fixture.
Block B (Controls line-wrap): PasHub wraps a long "Controls:" value across two
PDF lines and _get_in read only the first, truncating the label so the mapper
missed its Table 4e code. Re-join the continuation when the value ends on a
conjunction/comma (grammatically incomplete) so "…room thermostat and TRVs"
matches the existing code 2314. Scoped to Controls: — the shared _get_in is
unchanged; a wider truncation audit is tracked separately.
Age-band normalisation: the heat_network_age_band '2007 - 2011' block was one
facet of a broader bug — PasHub lodges the construction age band in mixed
templates ("K: …" prefixed, tight "1991-1995", space-padded "2007 - 2011") and
only the letter form was normalised, so bare year ranges were SILENTLY
MIS-RATED by the age-band-keyed U-value/DLF cascades. Normalise every template
to its RdSAP letter in _extract_age_band: unblocks the heat-network dwellings
AND corrects ~30 already-computing fixtures (within-0.5 42.9% → 49.5%,
MAE 3.05 → 1.02). Sibling Guinness cohort unchanged (82.9% / 0.36).
Block C (gable 'None') was already resolved on main — no change needed.
Fixture 461178278096 lodges no main heating at all (MissingMainFuelType);
tolerated as a known gap in the harness and ticketed (#1680) for the modelling
decision. LRHA ratchets re-baselined to the corrected cohort (0.49 / 1.05).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A cert lodging its horizontal dimensions measured externally
(measurement_type=2) must have its floor area and exposed perimeter converted to
internal dimensions before any SAP use (RdSAP 10 §3.4 + Table 2, p.17-18).
measurement_type was read nowhere (the `# What is this?` field), so external
certs carried a TFA ~11-15% too large — inflating occupancy and understating
every per-m2 output (CO2/m2, PE/m2).
Plumb measurement_type onto EpcPropertyData, and add a post-mapping
_with_internal_dimensions step: compute ONE whole-dwelling area/perimeter ratio
from the ground-floor footprint via the Table 2 built-form equation (§3.4 Note 4
— not a per-storey conversion, which over-corrects multi-part mid-terraces),
apply it to every storey's area/perimeter, reduce party walls by 2w (Note 5),
and refresh total_floor_area_m2 (read for occupancy N). Wall thickness w is the
lodged main-dwelling value, else the Table 3 default (p.19). Room-in-roof floor
area is always internal (§3.1) and left untouched. Scoped to the RdSAP-Schema
reduced-data family (17.0-21.x) where per-storey external dims are the lodgement
standard; legacy SAP-Schema-15.0/16.x are plumbed but not converted.
Corpus: within-0.5 81.0% -> 81.2%, MAE 0.571 -> 0.570, PE 2.5 -> 2.4. Cert
100091435353 TFA 128.7 -> 111.6 (lodged 112), SAP 66.23 -> 65.33 (+1.2 -> +0.3);
mean |dSAP vs lodged| over the 12 external certs 0.556 -> 0.429. Closes#1669.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On the gov-EPC API path a window's metal frame was unrepresentable: both
transmission tables were frameless (U_pvc, g, 0.70), so a cert lodging a metal
frame (pvc_frame:"false", RdSAP input 6-5) was billed at the PVC/wooden U and
frame factor 0.70 — over-rating metal-framed dwellings.
Add the RdSAP 10 Table 24 metal U-column (PDF p.50-51; mirrors u_window's metal
values) and a pvc_frame dimension to _api_glazing_transmission: an explicit
"false" routes the window to the metal U + SAP 10.2 Table 6c frame factor 0.8;
"true"/None/other keep the PVC column + 0.70, so no PVC cert regresses.
Secondary glazing keeps its frame-independent U (only FF changes). Wired into
_api_sap_window and _api_sap_roof_window (both full-schema, lodge pvc_frame);
the reduced-field path has no pvc_frame and is unchanged.
Spec: RdSAP 10 Table 24 metal column, SAP 10.2 Table 6c, RdSAP input 6-5 (p.80).
Corpus: within-0.5 80.6% -> 81.0%, MAE 0.578 -> 0.571. Cert 21067296
68.60 -> 66.96 (toward lodged 67). Closes#1667.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The gov-EPC API lodges a two-main dwelling's main_heating_fraction pair in
two incompatible encodings — a decimal fraction summing to ~1.0 ([0.8, 0.2])
or an integer percent summing to ~100 ([80, 20]) — while every consumer
divides by 100 unconditionally. A decimal pair therefore collapsed the second
main's share to ~1% of its true value, billing almost the whole load to the
first system's fuel/efficiency.
Normalise once at the mapper via a shared pair-sum discriminator
(_normalised_main_heating_fraction): a genuine two-main split summing nearer
1.0 than 100 is scaled to percent; a percent pair and any single main are left
exactly as lodged, so the 3 percent-encoded corpus certs do not regress. Wired
across the seven from_rdsap_schema_* branches and the SAP-17.1 path, whose old
int(fraction) floor silently deleted a decimal-lodged second main (int(0.35)=0).
Spec: RdSAP 10 item 7-5 (percent is canonical), p.81. Corpus: within-0.5
80.3% -> 80.6%, MAE 0.584 -> 0.578. Example cert 10070086972 17.59 -> 18.73
(toward lodged 18). Closes#1666.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
RdSAP 10 item 2-3(h) "another dwelling above" is U=0 (spec p.45: "There is no
heat loss through the roof of a building part that has the same dwelling above
or another dwelling above"). gov-API `roof_construction = 3` is that code, but
the mapper labelled it "Pitched (slates/tiles), no access to loft" (which is
code 5), so the per-part party override at heat_transmission.py:1172 never
matched and the part fell through to the dwelling_type-label-only
has_exposed_roof heuristic.
190 of the 200 corpus code-3 parts sit at index 0 and are already suppressed by
`_cert_lodges_exposed_roof` off the roof_insulation_location == "ND" signal. The
unmitigated set is the 10 parts at index > 0, where extensions default to
exposed and were billed a phantom pitched roof. Also removes a latent PV defect:
the bogus "Pitched" prefix satisfied `is_pitched` and divided PV array area by
cos(35 deg), inflating kWp.
Shipped with the stairwell fix so the corpus pin moves once: within-0.5
78.8% -> 79.6%, SAP MAE 0.626 -> 0.611, PE MAE 2.9 -> 2.7.
The roof change alone trades 4 certs out of the +-0.5 band for 2 in (one moving
4.4 SAP closer). Those 4 were investigated and are NOT a masked bug: the corpus
error distribution is symmetric (mean +0.076, median +0.03, sigma 1.606; 112
certs above +0.5 vs 102 below), ground-floor flats show 7 positive / 8 negative
with no excess over the null, and closing them needs HLC changes of 2.3-5.6% —
below the ~8.8% noise floor of the RHI anchor, calibrated on the 76 certs whose
SAP is exact (space-heating ratio 1.062 +/- 0.088).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
All six are `from_site_notes`-only (gov-API RdSAP corpus unmoved at 78.9%),
graded against the DB PasHub-assessment oracle (energy_rating_current,
source='lodged'), not the surveyor-entered pre_sap.
A. Manual-Entry boiler efficiency. Extractor read PCDF-Search columns that a
manual survey lacks; mapper returned None for all boilers → the generic
0.80 gas default. Now read `Type of boiler:` / `Heating System (Boiler):`
/ `Is there an open flue?` and resolve the descriptor to its SAP 10.2
Table 4b code. Moves all 7 Wythenshawe manual boilers to the oracle
(4a Hollyhedge back boiler 63.6→53.4=53; group MAE 2.42→0.42).
B. Room-in-roof gable type None → strict-raise. A form variant omits the
gable-type line (genuine field-absence); default None → party `gable_wall`
(U=0.25), mirroring `_api_type_1_gable_kind(None)`. Present-but-unknown
labels still strict-raise. Unblocks 8 & 34 Bucklow Drive.
C. Secondary `Closed room heater` unmapped → add Table 4a code 633.
Unblocks 8 Brinkshaw Avenue.
D. Alternative-wall insulation thickness never extracted → parse
`Wall insulation thickness:` and pass it through instead of the RdSAP 10
§5.4 100 mm favourable default.
E. `Number of heated rooms?` label mismatch → the PDF lodges
`Please enter the number of HEATED rooms:`.
F. HW cylinder thermostat dropped (NEW, systematic). `_map_sap_heating`
never set `cylinder_thermostat`, so a surveyed thermostat defaulted None
and the calculator applied the SAP 10.2 Table 2b Note a) ×1.3 penalty.
Map it ("Y"/"N", gated on a lodged cylinder) as the Elmhurst path does.
Fixes the two worst DB-oracle under-raters (52 Bucklow 59.4→61.8=62;
13 Kerne Grove 65.0→67.1=67).
DB-oracle cohort (148 matched): within-0.5 83.8%→87.8%, MAE 0.427→0.303.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A mid-terrace / enclosed-mid-terrace (built_form 4/6) has external walls on
its two opposite (front/rear) faces only (RdSAP 10 §1.1, p.7-8); both end/
gable elevations — and their room-in-roof continuations — abut a neighbour
and are party walls (§1.4.3, p.9). The gov-API path is built_form-blind, so a
cert lodging an "Exposed" RR gable (`gable_wall_type=1`) on such a form had it
billed at the masonry main-wall U (~2.5) instead of the party 0.25 (Table 4,
p.22), over-counting fabric heat loss and under-rating the dwelling.
`_api_type_1_gable_kind` now takes `built_form` and reclassifies an Exposed RR
gable to Party on the both-ends-party forms {4, 6} only; end-terrace {3, 5},
semi {2} and detached {1} keep their genuine exposed gable. The API path lodges
no per-gable U, so this only ever overrides a cascade fallback.
Corroborated three ways: RdSAP §1.1 + Table 4; S10TP-05 Appendix A (a
mid-terrace's roof-gable junction length is zero); and three accredited
Elmhurst mid-terrace worksheets (000477/000480/000516), which all lodge Party
gables. Those worksheets run the site-notes path (already correct) so the 195
Elmhurst pins are untouched.
Corpus (RdSAP-21.0.1, 1000 certs): within-0.5 78.8% -> 79.5%, SAP MAE
0.625 -> 0.599, PE MAE 2.85 -> 2.71, CO2 MAE 0.070 -> 0.068. Cert
100040550095 -10.13 -> +0.37; the 18-cert terraced-external-gable cohort mean
-1.94 -> +0.11; end-terrace/semi/detached untouched. Thresholds ratcheted.
Tests: new test_mapper_rir_gable_terraced (9); 195 Elmhurst pins, 288
cert_to_inputs/real-cert pins, 54 mapper tests, and the corpus gauge all green;
no new pyright errors.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#1631: the label blocks in the main room-heater type table (Heating
System (Other) field), not the secondary table the issue cited — same
PCDB row either way. Wythenshawe 164→165 computed. The fixture scores
49.5 vs pre_sap 31: its own 2025 accredited cert
6890-0597-0722-0196-3943 lodges the identical codes (603/26/2601) at
band E agreeing with us, while pre_sap matches the dwelling's
superseded electric configuration (2017/2023 certs, band F) — accepted
divergence, MAE ceiling re-baselined 0.75→0.85 with the citation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
#1628: fixture 500796700868 re-blocks on a later label
(first-exception-hit); metrics unchanged, ratchets hold.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Follow-on surfaced by #1626: fixture 500796646644 progressed past the
2404 conflict onto cylinder insulation 'Loose jacket' → code 2 (gov-API
coding, Elmhurst 'Jacket' sibling). Wythenshawe 161→162 computed
(32.6 vs pre 34); ratchets hold at 75.3%/0.744.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
#1626: ground truth resolved per the issue's option 3. The accredited
cert 0764-2894-7173-2720-5535 (62 Mayfair Road, UPRN 77066640 — the
blocked Wythenshawe fixture's dwelling) lodges 402 + 2402 for the same
heater description, and RdSAP 10 p.80 (item 7-3) requires a PCDB index
for any HHR lodgement — absent one, the surveyed type field is
authoritative and 2404 downgrades to 2402 (automatic charge control,
preserving the tick's automatic-charging semantics). Applies only to
the mapped non-HHR types (402/404); any other pairing still fails loud.
LRHA fixture 505091686590 (Fan storage heater, same slip) computes
64.0 vs pre 65; LRHA 76→77 computed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
#1625: 'Same dwelling below' → the "(another dwelling below)" party-
floor suppression string, mirroring the gov-API code-8 precedent
(RdSAP 10 §3: an internal floor between heated storeys has no floor
heat loss). Wythenshawe 151→161 computed; 7/10 newly-unblocked land
within 0.5 — ratchets re-baselined 0.76/0.72 → 0.75/0.75 for the two
tail entries (−3.3/−5.5).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
#1624: Wythenshawe 149→151 computed. 500767954156 lands within 0.5;
500797465814 is a −8.8 tail entry — MAE ceiling re-baselined 0.67→0.72
per the coverage-growth convention (within-0.5 floor 0.76 holds).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
#1637: 'MV - Heat Recovery' → MVHR kind (Table 4g default 46.2% heat
recovery — PasHub lodges no PCDF product/wet-rooms/duct data, so these
land in the accuracy tail as the issue predicts). Unblocking them
surfaced 4 biomass heat networks whose fuel label 'Heat from boilers -
biomass' passed through raw to the calculator's MissingMainFuelType —
now mapped to Table 12 code 43, and any other uncovered community fuel
strict-raises UnmappedPasHubLabel at the boundary (ADR-0015).
LRHA 61→76 computed / 28 blocked / 0 errored, within-0.5 43.4%,
MAE 2.985 — ratchets hold.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
#1636: the 6 single-line-label fixtures unblock and re-block on later
labels (MVHR/cylinder); the 6 line-wrap-truncated '…room thermostat
and' fixtures still need the extractor fix tracked under #1622.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
#1635: 'Programmer, room thermostat and TRVs' on a heat pump → 2210
(PCDB heat_pump_controls), and the lodgement now carries Table 4a
category 4 — without it the calculator's Appendix N3.6/N3.7 gate never
fired and the PCDB heat pump was billed as 100%-efficient direct
electric (6.0/18.5 vs pre 62/57). With the category threaded the two
LRHA WAVE 3 fixtures score 52.1/56.9 (the latter within 0.5).
LRHA 59→61 computed, within-0.5 42.6%, MAE 2.830 — ratchets hold.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
