Six properties failed task 4d006fab (portfolio 854 / scenario 1334) with
DegeneratePredictionError. None has a lodged EPC, so each was predicted from a
neighbouring-cert cohort — and every cohort cert lodges its single building part
as `identifier: "complete"`. from_api_string accepted only "Main Dwelling" /
"Main building", so "complete" fell to OTHER, _has_main_part returned False, and
prediction refused to proceed.
"complete" is what SAP 9.91/9.92-era software writes for an unextended dwelling:
semantically identical to "Main Dwelling". Casing varies within a single postcode
(cert 8824-7422-1180-6934-1902 lodges "complete", 199 Highfield Road "Complete"),
so matching is now case-folded.
The API's `identifier` is free text with no schema enum — api.yml declares the
whole cert body `additionalProperties: true` and documents no field of it, and
`identifier` is absent from the 17 vocabularies at /api/codes. So the recognised
set can only grow by discovery, and each miss has cost an incident ("Main
building" → task a40e71c4; "complete" → task 4d006fab). Hence three changes, not
one:
- "complete" → MAIN, case-insensitively.
- Bare "Extension" → EXTENSION_1. The regex required a digit, so 668 corpus
occurrences silently dropped a real extension from the structure. This reverses
a prior deliberate pin whose stated rationale (RdSAP10 §1.2's 4-extension cap)
does not apply — there is no out-of-range number in a bare "Extension".
- _anchor_lone_building_part: a cert lodging exactly ONE part describes the whole
dwelling whatever it is called, so an unclassified lone part becomes MAIN.
Multi-part certs are untouched — there the identifier carries real information
and guessing would invent structure the cert does not state.
Also removes a false claim from DegeneratePredictionError's message: it asserted
the template was "lodged with a null part identifier", which was hardcoded, never
checked against a real cert, and wrong. It misdirected this investigation.
Verified against the live gov API: all 13 cohort certs across BL1 8EB, BL3 6XJ
and BL4 0RA now resolve a MAIN part. The full modelling e2e was NOT run (no local
AWS creds for the geospatial lookup) — this clears the blocking error, it does not
prove the predictions are good.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
For a dwelling with two main systems heating different parts, the (93)m
Table 4e mean-internal-temperature adjustment was taken from system 1 alone
(`details[0]`). SAP 10.2 p.186 requires the (203)-weighted mean of each
system's control adjustment — (1 - (203)) for system 1, (203) for system 2 —
mirroring the Table 9b responsiveness weighting already applied two lines
above. Keyed to system 1, a dwelling led by a small-fraction system (e.g. a
20% HHRSH, adj 0.0 °C, ahead of an 80% manual-charge storage heater, adj
+0.7 °C) was modelled ~0.7 °C too cool → space heat under-counted → over-rated.
Compute the weighted adjustment at the call site. Shared `cert_to_inputs`
path, so it also lifts the accredited gov-API RdSAP corpus (81.5% → 81.7%,
guardrail). 5 Edmund Close +2.74 → +0.59; 4 Edmund Close -2.51 → -0.55.
LRHA cohort within-0.5 60.2% → 62.1%, MAE 0.496 → 0.445; ratchets re-based to
0.62 / 0.45. Also corrects an oil-combi docstring (462051619031 is a Grant
Vortex 10599, not the Worcester 18415).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A PasHub survey can lodge a manufacturer's measured cylinder loss ("What is
the cylinder measured heat loss: 0.94 kWh/24 hours") — the SAP 10.2 §4
(48)-(50) declared-loss factor that must override the Table 2 V×L×VF
insulation computation. The site-notes path dropped it two ways: the
extractor looked for "Cylinder Measured Heat Loss:" (matches 0 fixtures) while
the PDF label is "What is the cylinder measured heat loss:" (92 fixtures), and
the mapper never wired `cylinder_heat_loss` even when captured. So the
calculator fell back to the age-band insulation default — a lossier cylinder
than surveyed — over-costing hot water and under-rating SAP.
Fix the extractor label and add `_pashub_cylinder_measured_heat_loss` (parses
the leading float, returns None for "Not known"), mirroring the gov-API and
Elmhurst paths that already pass `cylinder_heat_loss` through. 3 Woodmans
Court -1.33 → -0.09; LRHA cohort within-0.5 58.3% → 60.2%, MAE 0.515 → 0.496.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The window channel collapsed a dwelling with genuinely mixed per-window
glazing (distinct `glazing_type` codes, no lodged per-window U) onto
windows[0]'s U, billing the entire opening area at one window's value.
SAP 10.2 worksheet (26)-(27) sums each window's own RdSAP Table 24 U over
its own area. The PasHub site-notes path is the only source that lodges
heterogeneous per-window codes with no per-window U (the gov-API/Elmhurst
paths carry a uniform code or a lodged U), so it was the only one mis-billed.
Add a `windows_have_mixed_codes` branch (factoring `_window_u_raw_from_code`
out of `_synthesised_window_u_raw`) that bills each window individually with
the per-window curtain transform. Scoped to lodge no average → uniform-glazing
dwellings stay byte-identical. Symmetric: corrects over- and under-raters
(e.g. 8 Loveden View -1.39 → window channel 32.31 → 26.67 W/K). LRHA cohort
within-0.5 58.3% → 58.3%, MAE 0.531 → 0.515.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The differ and the HubSpot fetch already named notes_for_surveyor, but the
SQLModel had no such field, so check_for_db_update_trigger raised
AttributeError on every deal and the scrape could not diff at all.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Reproducible RdSAP-10 build (build_100010472748.py) + downloaded Input Summary
and SAP Worksheets (accredited SAP 62 D) + the entry crib sheet, as the
ground-truth for the community-heating mapping fix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An oil STORAGE combi (Worcester Greenstar Heatslave II, PCDB 18415,
store_type 1) zeroed its internal-store loss: pcdb_combi_loss_override
defers on store_type in {1,2,3} and a combi has no cylinder, so line
(56) dropped to zero — under-costing hot water and over-rating SAP by
+2.0..+4.9 across the five LRHA WAVE 3 oil combis.
SAP 10.2 Table 2 note b) now includes the store loss when the combi's
efficiency is from the PCDB: the PCDB store volume (field 42) and
insulation thickness (field 44) feed Table 2/2a with the Table 2b
storage-combi temperature factor, reproducing the accredited Elmhurst
P960 worksheet exactly (5.4175 kWh/day, Jan 167.94, ~1977 kWh/yr).
Cohort: within-0.5 55.3% -> 58.3%, MAE 0.675 -> 0.531; ratchet
tightened accordingly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Closes#1684. `_main_2_space_heating_fuel_cost_gbp_per_kwh` billed an
electric Main 2 at Main 1's space-heating rate. That was a no-op on a
homogeneous off-peak cohort (storage+storage, direct+direct → equal
rates) but mis-billed the HETEROGENEOUS case: an off-peak STORAGE Main 1
(5.5 p/kWh low) + an on-peak DIRECT-ACTING panel Main 2 (15.29 p/kWh
high) rode the storage low rate, under-costing the panel half and
over-rating SAP.
Flip it to bill Main 2 at its OWN Table 12a Grid 1 SH row, the same
per-system resolution Main 1 and a non-electric Main 2 already use — and
the same one `_main_2_high_rate_fraction` already applies, so the scalar
cost and the high-rate split are now consistent.
The docstring's "regresses certs 13/34 (Parkers Hill / Dunley Road)"
warning was stale: those names appear nowhere but that docstring, and
the full guard suite shows zero new regressions (only the two
pre-existing `test_heating_systems_corpus` fails on main). LRHA WAVE 3
fixture 497655371974 (storage+panel) moves +15.3 → -2.5 vs PasHub; cohort
MAE 0.799 → 0.675 (ratchet re-baselined 0.80 → 0.68). within-0.5 holds at
55.3% — the -2.5 residual is PasHub-vs-Elmhurst divergence, not a bug, so
the fixture stays out of the within-0.5 pin. A ground-truth Elmhurst pin
is queued (see docs/HANDOVER_1684_ELMHURST_INPUTS_497655371974.md).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A dwelling with two main heating systems lodges a "Main Heating 2" block and a
"Percentage of space heating provided by Main Heating 1" split (e.g. 20% fan +
80% slimline storage, or 50% storage + 50% panel heaters). The survey model held
a single `main_heating`, and the extractor read the whole §Heating section
first-match — so it captured only Main 1, modelled it as 100% of the load, and
dropped Main 2 and the split (also bleeding Main 2's fields into Main 1). Every
such dwelling over-rated.
- `HeatingAndHotWater` now holds `main_heating_2` + `main_heating_1_percent`.
- The extractor slices the "Main Heating Systems" region per system on the
"Main Heating N" markers and reads Main 1's percentage; a single-main dwelling
(no "Main Heating 2") parses byte-identically to before.
- `_map_sap_heating` factors the per-system detail into `_pashub_main_heating_
detail(main, fraction)` and emits two `MainHeatingDetail`s, setting Main 2's
`main_heating_fraction = 100 - Main 1 %` (the calculator reads Main 2's share
and gives Main 1 the remainder).
5 of the 6 dual-main dwellings converge (within-0.5 53.4% → 55.3%, MAE 0.799);
single-main fixtures and the gov-API / Elmhurst / Guinness / Wythenshawe corpora
are unaffected. The 6th (497655371974, storage+panel) regresses because the
calculator deliberately bills an electric Main 2 at Main 1's rate
(`_main_2_space_heating_fuel_cost_gbp_per_kwh`, deferred §10a off-peak slice) —
the correct two-main mapping now exposes that calculator gap; tracked separately.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A PasHub Manual-Entry electric boiler lodges "Direct acting" in its "Heating
System (Boiler):" age-band cell. `_PASHUB_MANUAL_BOILER_AGE_BAND_TO_TABLE_4B`
covers only gas/liquid Table 4b rows, so the descriptor resolved to None and the
boiler stayed uncoded — taking the generic 0.80 gas-boiler seasonal efficiency
AND, on a Dual meter, `_table_12a_system_for_main`'s None-fallback that bills the
whole main-heating load at the off-peak LOW rate. The two errors pull opposite
ways, so an identical mis-mapping over-rated one dwelling and under-rated another
(fixture 497712825571 +9.5 on a Dual meter; 461386632387 -6.2 on a Single meter).
A direct-acting electric "boiler with radiators" is SAP 10.2 Table 4a code 191
(direct-acting electric boiler, efficiency 1.00, §12 Rule 3 tariff). Code it
directly in the manual-boiler resolver (gated on fuel Electricity + no PCDB
product), so the single code value drives the efficiency table, the Table 12a
pricing row and the tariff dispatch together. Both fixtures converge:
497712825571 → +0.6, 461386632387 → +0.4. LRHA within-0.5 52.4% → 53.4%,
MAE 0.835. Renamed `_pashub_manual_boiler_table_4b_code` → `_pashub_manual_
boiler_code` (it now also returns the Table 4a electric code).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An electric room/panel-heater dwelling (main SAP code 691) on a Dual meter is a
10-hour tariff by RdSAP 10 §12 Rule 3, so its main heating bills the secondary's
identical electricity at a 0.50-blended 11.09 p/kWh. Its SECONDARY heating,
however, resolved its tariff from the bare meter string — where a Dual meter
defaults to 7-hour (`tariff_from_meter_type`) — and billed at the 7-hour
all-high-rate 15.29 p/kWh. The ~10% secondary fraction was over-priced by ~4 p/kWh,
under-rating SAP by ~1 point across the LRHA WAVE 3 code-691 cohort.
Route the dwelling's resolved `_rdsap_tariff(epc)` into `_secondary_fuel_cost_gbp_
per_kwh` / `_secondary_off_peak_rate_gbp_per_kwh`, exactly as the main-heating cost
path (`_space_heating_fuel_cost_gbp_per_kwh`) and the secondary CO2 path
(`_secondary_heating_co2_factor_kg_per_kwh`) already do — removing the internal
meter-only resolution that diverged from the dwelling's single §12 tariff.
Storage-heater dwellings (7-hour by §12) are unchanged — their secondary correctly
stays at 15.29 p (regression-pinned in test_cert_to_inputs). LRHA WAVE 3 within-0.5
49.5% → 52.4%, MAE 0.977; gov-API, Elmhurst RealCert, Guinness and Wythenshawe
corpora unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Clears the three non-table-add blocks tracked in #1639 plus a latent age-band
bug they surfaced. LRHA WAVE 3 cohort: 77 → 103 of 104 computing.
Block A (exact cylinder volume): a measured cylinder lodges its litres on a
separate "Exact cylinder volume:" line; parse it in the extractor, carry it on
WaterHeating.cylinder_volume_measured_l, route the "Exact cylinder volume" band
to SAP cylinder-size code 6, and thread the litres into SapHeating (the
calculator reads them only when the code is 6). Verified end-to-end on the real
166 L fixture.
Block B (Controls line-wrap): PasHub wraps a long "Controls:" value across two
PDF lines and _get_in read only the first, truncating the label so the mapper
missed its Table 4e code. Re-join the continuation when the value ends on a
conjunction/comma (grammatically incomplete) so "…room thermostat and TRVs"
matches the existing code 2314. Scoped to Controls: — the shared _get_in is
unchanged; a wider truncation audit is tracked separately.
Age-band normalisation: the heat_network_age_band '2007 - 2011' block was one
facet of a broader bug — PasHub lodges the construction age band in mixed
templates ("K: …" prefixed, tight "1991-1995", space-padded "2007 - 2011") and
only the letter form was normalised, so bare year ranges were SILENTLY
MIS-RATED by the age-band-keyed U-value/DLF cascades. Normalise every template
to its RdSAP letter in _extract_age_band: unblocks the heat-network dwellings
AND corrects ~30 already-computing fixtures (within-0.5 42.9% → 49.5%,
MAE 3.05 → 1.02). Sibling Guinness cohort unchanged (82.9% / 0.36).
Block C (gable 'None') was already resolved on main — no change needed.
Fixture 461178278096 lodges no main heating at all (MissingMainFuelType);
tolerated as a known gap in the harness and ticketed (#1680) for the modelling
decision. LRHA ratchets re-baselined to the corrected cohort (0.49 / 1.05).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The service was the last one living wholly under backend/. It now follows
the same layering as abri and the other newer services:
domain/pashub_fetcher/ core file classification, subfolders
infrastructure/pashub_fetcher/ PasHub client, token getter, wire DTOs
orchestration/ PashubFetcherOrchestrator (was PashubService)
applications/pashub_fetcher/ lambda handler, trigger request, dev tooling
core_files.py is split along the layer boundary: the domain module keeps the
filename/evidence-category classification rules and no longer imports
infrastructure.postgres, while the CoreFiles -> FileTypeEnum translation moves
to infrastructure/pashub_fetcher/core_file_types.py.
Tests move into the tests/ tree by layer. Note this puts them in the only
suite CI currently runs (unit_tests.yml is disabled), so these 73 tests now
execute on PRs for the first time; they were previously reachable only via
the legacy pytest.ini testpaths.
sharepoint_renamer's image now copies just domain/pashub_fetcher/ rather than
the whole service, since SharepointSubfolders is all it needed.
Behaviour is unchanged. tests/ goes 9927 -> 10000 passed (+73, exactly the
tests that moved in); the legacy suite keeps its same 17 pre-existing failures
and 11 errors. tests/test_lambda_packaging.py confirms both changed
Dockerfiles still copy their handler's full import closure.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A second export selection alongside the per-Scenario one, chosen by
`plan_selection` on the export request:
- "scenario" (default, unchanged): one sheet per Scenario, freshest Plan per
(property, scenario); scenario_ids required.
- "default" (new): a single "Default Plans" sheet of each home's is_default
Plan (one-per-property across all Scenarios, ADR-0012/0017) — the portfolio's
current state, one row per home; scenario_ids ignored.
Repository gains `default_rows_for(portfolio_id, property_ids)`: same read-model
and Effective-EPC join as `rows_for`, with the plan-selection WHERE swapped from
`scenario_id = :scenario_id` to `is_default = TRUE`. Orchestrator branches on
plan_selection; the router validates it and only requires scenario_ids for the
scenario selection. A recipe without plan_selection defaults to "scenario", so
the change is backward-compatible.
Both selections now surface the chosen Plan's `name` as a `plan_name` column.
Tests: repository default_rows_for (scenario-independent selection, plan_name,
non-default excluded); orchestrator default path (one "Default Plans" sheet;
empty selection is a recorded failure). 62 passed, pyright --strict clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two outputs of validating the Wythenshawe cohort against accredited Elmhurst:
1. Harness note: the DB-oracle residual tail is not engine defects. The two
worst under-raters, 6 Barry Road (ours 60.12, oracle 62) and 87 Alderue
(ours 61.80, oracle 63), were re-keyed into accredited Elmhurst on our exact
inputs — Elmhurst returned 60 and 62, matching our engine and both below the
lodged oracle. So our engine computes correct RdSAP; the lodged certs carried
a lighter input set (dropped the surveyed 17.4 m2 solid-brick alternative
wall, +1.7 SAP). It's input-provenance drift, not a calculation gap — don't
chase these as engine bugs or tune the engine to close them. Corrects the
earlier "spec-literal alt-wall" note.
2. epc-to-elmhurst skill gotcha: give Elmhurst the RAW surveyed room height, not
the engine's storey height. The mapper already adds +0.25 m to upper floors
(_UPPER_FLOOR_HEIGHT_ADD_M), so feeding sap_floor_dimensions.room_height_m
makes Elmhurst double-add the joist void (~1 SAP too low — it under-scored
6 Barry 60->59 until the first-floor height was corrected 2.76 -> raw 2.51).
Comment/doc only; no behaviour change (harness still 89.2%/0.297, pyright clean).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`test_pashub_sap_accuracy_wythenshawe` graded against the manifest `pre_sap`
(`hubspot_deal_data.pre_sap`) — surveyor hand-entered and frequently wrong,
which floated a misleading 79% / MAE 0.75. Grade instead against the DB
PasHub-assessment SAP (`epc_property_energy_performance.energy_rating_current`,
`ep.source='lodged'`), matched by UPRN or postcode+house-number.
`scripts/enrich_pashub_wythenshawe_oracle.py` resolves each fixture's oracle SAP
once (DB required, not run in CI) and bakes `oracle_sap` / `oracle_match` into
`manifest.json`, so the harness stays DB-free. 148/168 resolve (39 UPRN + 109
address); the 20 unmatched carry `oracle_sap` null and are excluded from the
aggregate until PasHub sources their SAP.
With the six from_site_notes fixes, the 148 matched score within-0.5 89.2%
(132/148), MAE 0.297 (was, vs pre_sap, 79.2% / 0.754). Ratchets re-baselined
0.75→0.88 floor, 0.85→0.31 ceiling. The residual tail is the integer-oracle
rounding floor (spec-correct dwellings) plus a couple of spec-literal gaps, not
engine defects.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
All six are `from_site_notes`-only (gov-API RdSAP corpus unmoved at 78.9%),
graded against the DB PasHub-assessment oracle (energy_rating_current,
source='lodged'), not the surveyor-entered pre_sap.
A. Manual-Entry boiler efficiency. Extractor read PCDF-Search columns that a
manual survey lacks; mapper returned None for all boilers → the generic
0.80 gas default. Now read `Type of boiler:` / `Heating System (Boiler):`
/ `Is there an open flue?` and resolve the descriptor to its SAP 10.2
Table 4b code. Moves all 7 Wythenshawe manual boilers to the oracle
(4a Hollyhedge back boiler 63.6→53.4=53; group MAE 2.42→0.42).
B. Room-in-roof gable type None → strict-raise. A form variant omits the
gable-type line (genuine field-absence); default None → party `gable_wall`
(U=0.25), mirroring `_api_type_1_gable_kind(None)`. Present-but-unknown
labels still strict-raise. Unblocks 8 & 34 Bucklow Drive.
C. Secondary `Closed room heater` unmapped → add Table 4a code 633.
Unblocks 8 Brinkshaw Avenue.
D. Alternative-wall insulation thickness never extracted → parse
`Wall insulation thickness:` and pass it through instead of the RdSAP 10
§5.4 100 mm favourable default.
E. `Number of heated rooms?` label mismatch → the PDF lodges
`Please enter the number of HEATED rooms:`.
F. HW cylinder thermostat dropped (NEW, systematic). `_map_sap_heating`
never set `cylinder_thermostat`, so a surveyed thermostat defaulted None
and the calculator applied the SAP 10.2 Table 2b Note a) ×1.3 penalty.
Map it ("Y"/"N", gated on a lodged cylinder) as the Elmhurst path does.
Fixes the two worst DB-oracle under-raters (52 Bucklow 59.4→61.8=62;
13 Kerne Grove 65.0→67.1=67).
DB-oracle cohort (148 matched): within-0.5 83.8%→87.8%, MAE 0.427→0.303.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fetch the property from the HubSpot deals API, persist it on
hubspot_deal_data (column added by the frontend repo's drizzle
migration), and include it in the differ's FIELD_MAP so a change
to the field triggers a DB reload.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
#1631: the label blocks in the main room-heater type table (Heating
System (Other) field), not the secondary table the issue cited — same
PCDB row either way. Wythenshawe 164→165 computed. The fixture scores
49.5 vs pre_sap 31: its own 2025 accredited cert
6890-0597-0722-0196-3943 lodges the identical codes (603/26/2601) at
band E agreeing with us, while pre_sap matches the dwelling's
superseded electric configuration (2017/2023 certs, band F) — accepted
divergence, MAE ceiling re-baselined 0.75→0.85 with the citation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
#1625: 'Same dwelling below' → the "(another dwelling below)" party-
floor suppression string, mirroring the gov-API code-8 precedent
(RdSAP 10 §3: an internal floor between heated storeys has no floor
heat loss). Wythenshawe 151→161 computed; 7/10 newly-unblocked land
within 0.5 — ratchets re-baselined 0.76/0.72 → 0.75/0.75 for the two
tail entries (−3.3/−5.5).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
#1624: Wythenshawe 149→151 computed. 500767954156 lands within 0.5;
500797465814 is a −8.8 tail entry — MAE ceiling re-baselined 0.67→0.72
per the coverage-growth convention (within-0.5 floor 0.76 holds).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add pashub_accuracy_wythenshawe fixtures + a sibling accuracy module,
built from project_code
[Wythenshawe WH:SHF RAs - WH:SF - 240426 - 494493718748] the same way as
LRHA WAVE 3 (#1620). First run: 149 computed / 19 blocked / 0 errored,
within-0.5 76.5%, MAE 0.665. Ratchets baselined to this cohort; the 19
blocks are separately-ticketed mapper-label gaps (floor 'Same dwelling
below' ×10 dominating), not tuning.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Teaches the pashub `from_site_notes` mapper the electric-storage-heater and
room-heater families, following the existing boiler/heat-network split exactly
(fix at the mapper boundary, ADR-0015 — never mis-rate, never strict-raise deep
in the calculator). Adds:
- Table 4e Group 4 storage charge-control codes (2401/2402/2404) and Group 6
room-heater control codes (2602/2605), each gated on its own system group so a
control label never takes a wrong-group 2xxx code.
- The Table 4a system code from the surveyed heater type lodged in the "Heating
System (Other)" field (`community_heat_source`): Fan storage 404, slimline
402, PCDF-Search product ⇒ HHR 409 (RdSAP 10 p.80), room heaters 691/694.
- A 2404⇔409 coherence guard: high-heat-retention controls on a non-HHR storage
type is a surveyor contradiction and fails loud rather than rating a mismatch.
All codes taken from the SAP 10.2 spec (14-03-2025 PDF) and already present in
the calculator's inventories — no calculator change.
LRHA WAVE 3 (lincs rural) accuracy: 8 computed / 96 blocked → 45 computed / 59
blocked / 0 errored (within-0.5 46.7%, MAE 2.989); ratchets re-baselined to the
new coverage. Guinness cohort (208) + gov-API corpus unchanged; pyright --strict
adds zero new errors.
Closes#1619
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
5 Paxton Place (deal 507639151845, uprn 10070864419) had a STALE fixture.
The same PAS Hub survey (report B74AA5A4-…, inspected 29 Jun 2026) has two
accredited revisions in S3: V2 (uploaded 6 Jul, ventilation "Natural",
SAP 80) and V5 (uploaded 14–15 Jul, ventilation "Mechanical Extract -
Centralised", SAP 76 — the current version). Our committed fixture was the
superseded V2; the portfolio-838 oracle tracks the latest V5 (76). Our
engine faithfully reproduces BOTH (V2 → 80.4, V5 → 75.6) — the +4.38 miss
vs the DB oracle was a fixture-version mismatch, NOT an engine over-rate
(a parallel over-credit sweep found no spec-correct from_site_notes lever;
every channel is faithful).
Refreshed the fixture to the V5 PDF (image-stripped, text layer preserved,
6.7MB→0.27MB, per scripts/build_pashub_accuracy_fixtures.py) and set the
manifest to C76. 5 Paxton flips from the cohort's largest miss to a
within-0.5 hit: DB-oracle gauge 86.9%→87.4% within-0.5, MAE 0.348→0.327.
Pinned via test_paxton_fixture_is_v5_resurvey_sap_76 (asserts a literal 76
against the V5 fixture) — a canary, because hubspot_deal_data.pre_sap is
still stale at C80 and a fixture-builder rerun would revert the manifest.
Confirmed against pashub: the current survey is SAP 76 (Khalim).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
13 Walsden (deal 499630239947, uprn 77179782) has TWO distinct gov-EPC
assessments: the PAS Hub site-note re-survey we parse (26 May 2026,
accredited SAP 64, backed by rd_sap_site_note files) and a SEPARATE
later lodged cert (24 Jun 2026, SAP 68, registered, no site-note). The
portfolio-838 baseline latched onto the 24-Jun SAP-68 cert; our engine
models the survey it was given and reproduces its own accredited 64
(64.19). 6 of 7 Walsden Street homes have pbp oracle == PAS Hub survey
SAP exactly — #13 is the lone collision.
A durable regression pin (asserts our SAP within 0.5 of a literal 64,
not the manifest value the fixture-builder can revert to a colliding DB
row) documents the provenance so nobody "corrects" the target to 68.
13 Walsden is a known re-survey-diverges outlier, not an engine
under-rate; confirmed against pashub (Khalim, 2026-07-15e).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`_PASHUB_FLOOR_EXPOSURE` mapped the surveyed "Semi Exposed (unheated)"
floor label to `is_above_partially_heated_space` (RdSAP §5.14, constant
U=0.7). A floor semi-exposed to an enclosed UNHEATED space belongs in
§5.13 Table 20 `u_exposed_floor` (band-B as-built U=1.20). §5.14's U=0.7
is reserved for a floor above a *partially* heated space. #1602
transposed the two branches; the too-low U under-counted floor loss on
the 9 label-carrying cohort fixtures → a shared over-credit.
Triangulated against all three siblings + the calculator docstring: the
gov-API code-2 "To unheated space" (mapper.py:4797) and Elmhurst
`_is_floor_exposed_to_unheated_space` (mapper.py:5970) both route to
Table 20; rdsap_uvalues.py:1416 documents Table 20 as collapsing exposed
and semi-exposed-to-unheated into one lookup. One-line data flip,
from_site_notes-only (gov-API RdSAP corpus untouched, still 78.9%).
Adjacent Ronald St mid-terrace pair (the clustered signal): 10 Ronald
60.35→59.02 (oracle 59), 12 Ronald 60.79→59.36 (oracle 59). Cohort vs
DB oracle (portfolio 838 effective_sap) 83.3%→86.9% within-0.5, MAE
0.377→0.348, no built form regresses, all 7 verified dwellings unchanged
(none carry the label). pashub harness 79.5%→82.4%, MAE 0.389→0.367;
ratchets tightened 0.78→0.82 / 0.40→0.37. pyright zero-new.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Electric-immersion DHW: bill at 100%, not the space boiler's efficiency 🟩
A separate electric immersion heater (SAP water-heating code 903) is 100%
efficient (SAP 10.2 Table 4a) and the space-heating boiler provides no water
heating, so Appendix D2.1 Eq D1 (the boiler seasonal-efficiency cascade) must
not apply to it. But `_water_heating_main` resolves DHW to the SPACE main — a
gas/oil boiler keeps its PCDB record — so three water-efficiency branches in
cert_to_inputs billed the immersion-heated cylinder at the boiler's ~87% summer
efficiency (a mongrel: electric fuel price x gas-boiler efficiency):
1. the scalar `water_eff = water_pcdb_main.summer_efficiency_pct / 100`,
2. the SAP §9.4.11 / Table 4c(2) -5pp no-interlock adjustment, and
3. the Eq D1 (winter, summer) seasonal pair from `pcdb_main`.
Gate all three on `not dhw_is_electric_immersion`; the correct immersion path
(`_water_efficiency_with_category_inherit` -> 1.0, Eq D1 off) then applies,
mirroring the Table 3 zero-primary-loss gate already present for WHC 903.
General bug — fires for any WHC-903 dwelling whose space main is a PCDB gas/oil
boiler. It surfaced via the PasHub campaign's #1600 no-water-heating default
(WHC 999 -> 903) on a gas-combi dwelling: 58 Hackle St M11 4WU, SAP 55.30 ->
57.67 (pre_sap 58, verified 59).
Guardrails: the gov-API RdSAP corpus IMPROVES 78.8% -> 78.9% within-0.5, MAE
0.625 -> 0.622 (a handful of corpus certs with a boiler space main + electric
immersion move closer to accredited) — MAE ceiling ratcheted 0.626 -> 0.625.
Regression pinned in test_cert_to_inputs (RED before / GREEN after). pyright
0-new (cert_to_inputs baseline 30).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* PasHub SAP accuracy: thread low-energy lights + draught lobby (from_site_notes) 🟩
Two systematic silent-drops in the PAS Hub `from_site_notes` path, each a field
the gov-API/Elmhurst mappers thread but site-notes dropped, letting a favourable
default reach the SAP-10.2 calculator. Validated against the correct pashub
oracle (property_baseline_performance.effective_sap_score, portfolio 838):
cohort within-0.5 55.1% → 62.6%, MAE 0.599 → 0.521.
- Low-energy lights: when "exact LED/CFL known = No", PAS Hub lodges an aggregate
"Number of fixed low energy lights?" count. Previously dropped (no dataclass
field / extractor key / mapper thread) → calculator saw 0 low-energy bulbs and
applied the pessimistic L5b/L8c no-data default, under-rating SAP. Now threaded
into low_energy_fixed_lighting_bulbs_count, mirroring from_elmhurst_site_notes.
~6% of the cohort; resolves 58 Hackle (−0.43 vs oracle 59 with #1615).
- Draught lobby: _map_sap_ventilation set only the legacy `draught_lobby` field,
never the canonical `has_draught_lobby` §2 (13) gate the cascade reads, so the
surveyed lobby was ignored and infiltration over-stated. Now mirrors Elmhurst.
Two other audited levers were REJECTED against the pashub oracle (they matched
the gov-cert/RdSAP convention but pashub does not apply them): percent_draughtproofed
(cohort 30.8%) and the §A.2.2 assumed secondary heater (19.2%). A heat-network
control-code fix (2306→2303 for 16 Bingley) is HELD pending spec/Elmhurst
adjudication — it contradicts ADR-0053; see the NOTE on _PASHUB_HEAT_NETWORK_CONTROL_TO_SAP10.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* PasHub heat-network control: room-thermostat-only maps to 2308, not 2306 🟩
The Table 4e Group 3 label "charging linked to use of community heating, room
thermostat only" was mapped to 2306 — the linked-to-use *with-TRVs* code
(control type 3, space charging 1.00). The label explicitly excludes TRVs, so
it is control type 2 / space 1.05 = code 2308/2309 (linked-to-use → DHW 1.00).
2306 asserted TRVs the survey doesn't have and over-rated the cohort's sole
heat-network dwelling (16 Bingley Close 56.89 → 54.01 vs pashub oracle 52).
Verified against SAP 10.2 Table 4e as encoded in cert_to_inputs.py
(_CONTROL_TYPE_BY_CODE 2308→2, space-charging 2308→1.05, DHW 2308→1.00) and the
RdSAP control-label vocabulary in MainheatControlAttributes.py. 2303 (the earlier
audit's guess) is rejected: it is a flat-rate code (DHW 1.05) whose oracle match
was a coincidence of two offsetting spec violations. The residual +2.0 to oracle
52 is the documented SAP-10.2-engine-vs-lodged offset, not a fuel/flags gap
(those were threaded by the #1590 follow-up) — supersedes the ADR-0053 /
HANDOVER_838 "community fuel/flags" attribution.
16 Bingley is the only heat-network fixture in portfolio 838, so this moves one
fixture strictly toward its oracle. Harness ratcheted: within-0.5 0.51→0.58,
MAE 0.625→0.521 (observed 58.5% / 0.520 with all three 2026-07-15 levers).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* PasHub SAP accuracy: restore 3 dropped §2 infiltration inputs (from_site_notes)
`from_site_notes` uniquely dropped three ventilation inputs that the gov-API
and Elmhurst sibling mappers all set, causing a systematic cohort-wide SAP
under-rate vs the company's own accredited SAP-10.2 certs
(property_baseline_performance.effective_sap_score, portfolio 838):
1. percent_draughtproofed — never set, so §2(15) window infiltration was
pinned at its 0.25-ACH worst case on every dwelling. Reconstructed as the
area-weighted % of draught-proofed windows (mirrors Elmhurst
`draught_proofing_percent`). Dominant term.
2. Upper-storey +0.25 m joist void — omitted; now added, matching
`_UPPER_FLOOR_HEIGHT_ADD_M` on the gov-API/Elmhurst paths. This
RE-ADJUDICATES #1601 ("keep raw"): that call was confounded by the
then-present draught-proofing drop suppressing every verified dwelling.
3. sheltered_sides — left None → calculator's flat default of 2, which
over-shelters end/semi/detached (RdSAP §S5 = 1/1/0). Now derived from
built form.
The three are NON-ADDITIVE — each overshoots alone (which is why all three
were previously rejected individually) — but together they land all 7
verified ground-truth dwellings toward truth (none regress) and every built
form near zero. Cohort 62.1% -> 83.3% within-0.5, MAE 0.507 -> 0.377.
Guardrails: gov-API RdSAP corpus unchanged (78.9%, these are from_site_notes-
only helpers). pashub harness 58.5% -> 79.5% / MAE 0.520 -> 0.389; ratchets
tightened 0.58->0.78 and 0.521->0.40. pyright zero-new (mapper baseline 39).
New focused tests in TestFromSiteNotesInfiltrationFixes cover all three
fields incl. area-weighting and the built-form shelter map; goldens updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolve#1601 and land #1602/#1603 on the merged-#1609 tree, adjudicated
against Khalim's verified ground truth (not pre_sap).
#1601 (upper-floor +0.25 m joist void) — RESOLVED as keep-raw. The
gov-API/Elmhurst mappers add 0.25 m to upper-storey heights because their
lodged height is a clear internal ceiling. Against verified truth RAW beats
+0.25 on all 7 verified dwellings (16 Stillwater raw -0.35 within-band vs
+0.25 -0.75; every verified property is already under truth and +0.25 only
lowers SAP). pashub surveys internal dimensions (202/205 "Measurements
Location: Internal") and its upper-ground height gap is modal 0.00, so the
joist void is not differentially present and a blanket +0.25 is unsafe.
_map_floor_dimensions keeps the raw surveyed height with a RESOLVED note.
#1602 — semi-exposed / exposed floor exposure. The surveyed "Floor type"
"Semi Exposed (unheated)" now sets is_above_partially_heated_space (RdSAP
§5.14 U=0.7) and "Exposed Floor" sets is_exposed_floor (§5.13 Table 20), on
the lowest storey (floor==0) only, mirroring the gov-API/Elmhurst siblings.
New strict-raise _pashub_floor_exposure (ADR-0015).
#1603a — alternative walls. New AlternativeWall survey dataclass +
extract_alternative_walls (the "Alternative Wall"->"Windows" section) +
_map_pashub_alternative_walls -> sap_alternative_wall_1/2. The cascade
deducts each alt-wall area from the part's main opaque wall (no double
count), so a mixed-facade dwelling is billed at each sub-area's own
construction. The alt-wall block lodges "As Built" (capital B) — aliased to
the existing default insulation code.
#1603b — extension floor construction. ExtensionConstruction gains a floor
block (parsed like the main building) and _map_extension_building_part now
threads floor_type/construction/insulation/u_value_known, which the main
path already did and the extension silently dropped.
Cohort: within-0.5 50.2% -> 51.2%, MAE 0.652 -> 0.624, 0 strict-raise
errors across all 205. Verified-truth safe: none of the 7 verified
dwellings have alt walls, and #1602 moves the one it touches (2 Philips
exposed ext floor) by -0.00. pyright 0-new (mapper baseline 39). Goldens
test_full_building_construction + end_to_end EXTENSION_1 updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
