An oil STORAGE combi (Worcester Greenstar Heatslave II, PCDB 18415,
store_type 1) zeroed its internal-store loss: pcdb_combi_loss_override
defers on store_type in {1,2,3} and a combi has no cylinder, so line
(56) dropped to zero — under-costing hot water and over-rating SAP by
+2.0..+4.9 across the five LRHA WAVE 3 oil combis.
SAP 10.2 Table 2 note b) now includes the store loss when the combi's
efficiency is from the PCDB: the PCDB store volume (field 42) and
insulation thickness (field 44) feed Table 2/2a with the Table 2b
storage-combi temperature factor, reproducing the accredited Elmhurst
P960 worksheet exactly (5.4175 kWh/day, Jan 167.94, ~1977 kWh/yr).
Cohort: within-0.5 55.3% -> 58.3%, MAE 0.675 -> 0.531; ratchet
tightened accordingly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Closes#1684. `_main_2_space_heating_fuel_cost_gbp_per_kwh` billed an
electric Main 2 at Main 1's space-heating rate. That was a no-op on a
homogeneous off-peak cohort (storage+storage, direct+direct → equal
rates) but mis-billed the HETEROGENEOUS case: an off-peak STORAGE Main 1
(5.5 p/kWh low) + an on-peak DIRECT-ACTING panel Main 2 (15.29 p/kWh
high) rode the storage low rate, under-costing the panel half and
over-rating SAP.
Flip it to bill Main 2 at its OWN Table 12a Grid 1 SH row, the same
per-system resolution Main 1 and a non-electric Main 2 already use — and
the same one `_main_2_high_rate_fraction` already applies, so the scalar
cost and the high-rate split are now consistent.
The docstring's "regresses certs 13/34 (Parkers Hill / Dunley Road)"
warning was stale: those names appear nowhere but that docstring, and
the full guard suite shows zero new regressions (only the two
pre-existing `test_heating_systems_corpus` fails on main). LRHA WAVE 3
fixture 497655371974 (storage+panel) moves +15.3 → -2.5 vs PasHub; cohort
MAE 0.799 → 0.675 (ratchet re-baselined 0.80 → 0.68). within-0.5 holds at
55.3% — the -2.5 residual is PasHub-vs-Elmhurst divergence, not a bug, so
the fixture stays out of the within-0.5 pin. A ground-truth Elmhurst pin
is queued (see docs/HANDOVER_1684_ELMHURST_INPUTS_497655371974.md).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An electric room/panel-heater dwelling (main SAP code 691) on a Dual meter is a
10-hour tariff by RdSAP 10 §12 Rule 3, so its main heating bills the secondary's
identical electricity at a 0.50-blended 11.09 p/kWh. Its SECONDARY heating,
however, resolved its tariff from the bare meter string — where a Dual meter
defaults to 7-hour (`tariff_from_meter_type`) — and billed at the 7-hour
all-high-rate 15.29 p/kWh. The ~10% secondary fraction was over-priced by ~4 p/kWh,
under-rating SAP by ~1 point across the LRHA WAVE 3 code-691 cohort.
Route the dwelling's resolved `_rdsap_tariff(epc)` into `_secondary_fuel_cost_gbp_
per_kwh` / `_secondary_off_peak_rate_gbp_per_kwh`, exactly as the main-heating cost
path (`_space_heating_fuel_cost_gbp_per_kwh`) and the secondary CO2 path
(`_secondary_heating_co2_factor_kg_per_kwh`) already do — removing the internal
meter-only resolution that diverged from the dwelling's single §12 tariff.
Storage-heater dwellings (7-hour by §12) are unchanged — their secondary correctly
stays at 15.29 p (regression-pinned in test_cert_to_inputs). LRHA WAVE 3 within-0.5
49.5% → 52.4%, MAE 0.977; gov-API, Elmhurst RealCert, Guinness and Wythenshawe
corpora unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The ADR-0030 component-accuracy and expired-pairs gates re-derive the *actual*
EpcPropertyData through the same mapper the leave-one-out scorer predicts against.
#1669 (measurement_type=2 external->internal conversion, RdSAP §3.4 + Table 2)
gives the fixture's external certs sharper internal dimensions, which tips the
geo-proximity-weighted donor mode on three components by single-target amounts:
total_window_area 3.7184 -> 3.7484, door_count 0.3333 -> 0.3737, has_pv floor
0.8929 -> 0.8571 (one pair 25/28 -> 24/28).
This is a ground-truth-method change, not a prediction-logic loosening: the fix
is spec-correct, corpus MAE improved (0.571 -> 0.570) and the real-cert accuracy
pin held. Per the gate convention the affected floors/ceilings are re-baselined
to the measured values with the attribution recorded; tighten-only resumes here.
Bisected: #1666/#1667 leave the gates green; #1669 introduces all three shifts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The service was the last one living wholly under backend/. It now follows
the same layering as abri and the other newer services:
domain/pashub_fetcher/ core file classification, subfolders
infrastructure/pashub_fetcher/ PasHub client, token getter, wire DTOs
orchestration/ PashubFetcherOrchestrator (was PashubService)
applications/pashub_fetcher/ lambda handler, trigger request, dev tooling
core_files.py is split along the layer boundary: the domain module keeps the
filename/evidence-category classification rules and no longer imports
infrastructure.postgres, while the CoreFiles -> FileTypeEnum translation moves
to infrastructure/pashub_fetcher/core_file_types.py.
Tests move into the tests/ tree by layer. Note this puts them in the only
suite CI currently runs (unit_tests.yml is disabled), so these 73 tests now
execute on PRs for the first time; they were previously reachable only via
the legacy pytest.ini testpaths.
sharepoint_renamer's image now copies just domain/pashub_fetcher/ rather than
the whole service, since SharepointSubfolders is all it needed.
Behaviour is unchanged. tests/ goes 9927 -> 10000 passed (+73, exactly the
tests that moved in); the legacy suite keeps its same 17 pre-existing failures
and 11 errors. tests/test_lambda_packaging.py confirms both changed
Dockerfiles still copy their handler's full import closure.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wet underfloor responsiveness was pinned at R=0.75 for the lumped gov-EPC emitter
code 2, regardless of floor construction. SAP 10.2 Table 4d splits it three ways
and RdSAP 10 Table 29 (p.56-57) selects from floor construction + age band:
timber floor 1.0, screed above insulation 0.75, concrete slab 0.25. A
concrete-slab system (solid floor, age A-E) was billed as screed, over-stating
responsiveness (higher MIT) and over-rating the dwelling; R=0.25 was unreachable.
_responsiveness now takes the cert and, for the lumped underfloor code, derives
the Table 29 subtype via _underfloor_responsiveness from the main part's floor
construction + age band: suspended TIMBER -> 1.0; solid A-E -> 0.25 (concrete
slab); solid F-M / suspended-not-timber / unknown / no ground floor -> 0.75
(screed, the prior default, so ambiguous no-ground-floor certs do not move). The
mapper stays a pass-through.
Spec: SAP 10.2 Table 4d (p.170), RdSAP 10 Table 29 (p.56-57). Corpus: MAE
0.570 -> 0.565 (net toward lodged), within-0.5 81.2%. Cert 100100137438 (solid,
band B) 72 -> 69, exactly lodged. Closes#1668 on the API path.
NOTE: the issue's worksheet adjudication (run 100100137438 through accredited
Elmhurst to confirm concrete-slab R=0.25) is still outstanding — it corrects to
lodged 69 exactly, but is not yet Elmhurst-worksheet-confirmed. The Elmhurst
site-notes path (_resolve_elmhurst_underfloor_subtype still raises for solid A-E)
is untouched — no corpus/fixture exercises it; separate follow-up.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
RdSAP 10 §5.9 "U-values of sheltered walls" (PDF p.43): a wall between a
flat/maisonette and an unheated buffer takes SAP 10.2 §3.3's U = 1/(1/U0 + Ru),
and Ru depends on the BUFFER TYPE — "Use Ru = 0.5 m²K/W for corridors and
Ru = 2.1 m²K/W for stairwell."
The buffer type is lodged on `SapFlatDetails.heat_loss_corridor` (0-based per
item 1-11, PDF p.70: 0 none, 1 heated, 2 unheated corridor, 3 unheated
stairwell). That field was read NOWHERE under domain/ — corridor presence was
inferred only from a lodged sheltered alternative wall — so every sheltered wall
took the corridor 0.5 and every stairwell flat over-billed its sheltered wall,
under-rating the dwelling.
Distinct from `_SHELTERED_GABLE_ADDED_RESISTANCE_M2K_W`, which is RdSAP 10
Table 4 (p.22) for a room-in-roof sheltered gable — same 0.5, different rule,
deliberately untouched.
Corpus gauge ratcheted UP: within-0.5 78.8% -> 79.8% (+10 certs), SAP MAE
0.626 -> 0.613, PE MAE 2.9 -> 2.8. Failure set and pyright count identical to
main. Portfolio 824 is stairwell-heavy (60 properties at code 3); property
749898 over-bills 4.44 W/K on a 48 m2 flat.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
RdSAP 10 §3.12 (PDF p.26), the flats/maisonettes floor rules: "There is no heat
loss through the floor if there is another flat below. Otherwise the floor area
of the flat ... is ... a semi-exposed floor if there are unheated premises below
it (e.g. an enclosed garage) ... Semi-exposed (sheltered) floors are treated as
if they were fully exposed." §5.13 (p.48) confirms the U-value identity: Table 20
applies to exposed and semi-exposed alike.
API floor_heat_loss=2 -> floor_type "To unheated space" set no exposure signal,
so on a mid-/top-floor flat the dwelling-level heuristic (has_exposed_floor=False,
assuming a heated dwelling below) went unopposed and the ENTIRE floor was billed
at 0 W/K. 11 properties across portfolios 796+824, mean +4.02 SAP, 10 of 11
over-rating — we judged dwellings more efficient than the accredited assessment
and would withhold retrofit measures they need.
Exemplars: 750018 space heating -53.2% -> -8.1% vs the cert's own RHI figure,
SAP +8.22 -> +1.16; 749898 -43.0% -> +0.9%, SAP +6.29 -> -1.11.
SCOPED TO FLATS ON PURPOSE. §3.12 is the flats/maisonettes section, and the
accredited Elmhurst worksheet disagrees with Table 20 for a house: golden cert
7536-3827-0600-0600-0276 (detached) lodges floor_heat_loss=2 on its Main part and
the worksheet bills it on the BS EN ISO 13370 ground-floor cascade at U 0.97, not
Table 20's 1.20. An unscoped mapper-level fix broke that pin and the
uprn_100050881708 accuracy pin; the scoped fix leaves both untouched.
No regressions: identical failure set and pyright count vs main; the RdSAP-21.0.1
accuracy corpus gauge is unchanged at 78.8% within-0.5 / MAE 0.625 (correctly —
all 7 corpus certs lodging code 2 are houses).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The "latest EPC per UPRN" selection ordered by e.id DESC, but row-id does not
track recency: a gov-EPC cert re-ingested after a PasHub survey lands with a
higher e.id, so the header/perf reads silently picked the stale gov cert (wrong
TFA, lodgement, property_type, current band) on 3 of portfolio 838. Order by the
effective lodgement date (registration -> completion -> inspection) DESC NULLS
LAST, e.id DESC tiebreak.
Compose the descriptive fabric from the structured SAP fields a re-survey lodges
(no gov-EPC prose): walls/roof/floor from the main epc_building_part, heating and
hot water from epc_main_heating_detail + the water-heating codes, decoded with
the SAP engine's own code space (fabric_description.py). Structured fabric wins
where present and falls back to the gov-EPC prose otherwise; windows and lighting
stay on prose (their per-element codes do not reconstruct the dwelling phrase).
Add built_form (epc_property, code-decoded) and property_age_band (building part
construction_age_band -> RdSAP date range) columns.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
6035 is built_form 4 (Mid-Terrace), so this PR routes its Exposed Type-1
RR gable to a party gable (U=0.25) instead of the main-wall U. Heat loss
drops: PE resid -0.14 -> -0.92, CO2 -0.0362 -> -0.0549 (SAP still exact 70).
Residuals stay within noise (PE <0.5% of lodged 191). All 113 golden
fixtures pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Electric-immersion DHW: bill at 100%, not the space boiler's efficiency 🟩
A separate electric immersion heater (SAP water-heating code 903) is 100%
efficient (SAP 10.2 Table 4a) and the space-heating boiler provides no water
heating, so Appendix D2.1 Eq D1 (the boiler seasonal-efficiency cascade) must
not apply to it. But `_water_heating_main` resolves DHW to the SPACE main — a
gas/oil boiler keeps its PCDB record — so three water-efficiency branches in
cert_to_inputs billed the immersion-heated cylinder at the boiler's ~87% summer
efficiency (a mongrel: electric fuel price x gas-boiler efficiency):
1. the scalar `water_eff = water_pcdb_main.summer_efficiency_pct / 100`,
2. the SAP §9.4.11 / Table 4c(2) -5pp no-interlock adjustment, and
3. the Eq D1 (winter, summer) seasonal pair from `pcdb_main`.
Gate all three on `not dhw_is_electric_immersion`; the correct immersion path
(`_water_efficiency_with_category_inherit` -> 1.0, Eq D1 off) then applies,
mirroring the Table 3 zero-primary-loss gate already present for WHC 903.
General bug — fires for any WHC-903 dwelling whose space main is a PCDB gas/oil
boiler. It surfaced via the PasHub campaign's #1600 no-water-heating default
(WHC 999 -> 903) on a gas-combi dwelling: 58 Hackle St M11 4WU, SAP 55.30 ->
57.67 (pre_sap 58, verified 59).
Guardrails: the gov-API RdSAP corpus IMPROVES 78.8% -> 78.9% within-0.5, MAE
0.625 -> 0.622 (a handful of corpus certs with a boiler space main + electric
immersion move closer to accredited) — MAE ceiling ratcheted 0.626 -> 0.625.
Regression pinned in test_cert_to_inputs (RED before / GREEN after). pyright
0-new (cert_to_inputs baseline 30).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* PasHub SAP accuracy: thread low-energy lights + draught lobby (from_site_notes) 🟩
Two systematic silent-drops in the PAS Hub `from_site_notes` path, each a field
the gov-API/Elmhurst mappers thread but site-notes dropped, letting a favourable
default reach the SAP-10.2 calculator. Validated against the correct pashub
oracle (property_baseline_performance.effective_sap_score, portfolio 838):
cohort within-0.5 55.1% → 62.6%, MAE 0.599 → 0.521.
- Low-energy lights: when "exact LED/CFL known = No", PAS Hub lodges an aggregate
"Number of fixed low energy lights?" count. Previously dropped (no dataclass
field / extractor key / mapper thread) → calculator saw 0 low-energy bulbs and
applied the pessimistic L5b/L8c no-data default, under-rating SAP. Now threaded
into low_energy_fixed_lighting_bulbs_count, mirroring from_elmhurst_site_notes.
~6% of the cohort; resolves 58 Hackle (−0.43 vs oracle 59 with #1615).
- Draught lobby: _map_sap_ventilation set only the legacy `draught_lobby` field,
never the canonical `has_draught_lobby` §2 (13) gate the cascade reads, so the
surveyed lobby was ignored and infiltration over-stated. Now mirrors Elmhurst.
Two other audited levers were REJECTED against the pashub oracle (they matched
the gov-cert/RdSAP convention but pashub does not apply them): percent_draughtproofed
(cohort 30.8%) and the §A.2.2 assumed secondary heater (19.2%). A heat-network
control-code fix (2306→2303 for 16 Bingley) is HELD pending spec/Elmhurst
adjudication — it contradicts ADR-0053; see the NOTE on _PASHUB_HEAT_NETWORK_CONTROL_TO_SAP10.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* PasHub heat-network control: room-thermostat-only maps to 2308, not 2306 🟩
The Table 4e Group 3 label "charging linked to use of community heating, room
thermostat only" was mapped to 2306 — the linked-to-use *with-TRVs* code
(control type 3, space charging 1.00). The label explicitly excludes TRVs, so
it is control type 2 / space 1.05 = code 2308/2309 (linked-to-use → DHW 1.00).
2306 asserted TRVs the survey doesn't have and over-rated the cohort's sole
heat-network dwelling (16 Bingley Close 56.89 → 54.01 vs pashub oracle 52).
Verified against SAP 10.2 Table 4e as encoded in cert_to_inputs.py
(_CONTROL_TYPE_BY_CODE 2308→2, space-charging 2308→1.05, DHW 2308→1.00) and the
RdSAP control-label vocabulary in MainheatControlAttributes.py. 2303 (the earlier
audit's guess) is rejected: it is a flat-rate code (DHW 1.05) whose oracle match
was a coincidence of two offsetting spec violations. The residual +2.0 to oracle
52 is the documented SAP-10.2-engine-vs-lodged offset, not a fuel/flags gap
(those were threaded by the #1590 follow-up) — supersedes the ADR-0053 /
HANDOVER_838 "community fuel/flags" attribution.
16 Bingley is the only heat-network fixture in portfolio 838, so this moves one
fixture strictly toward its oracle. Harness ratcheted: within-0.5 0.51→0.58,
MAE 0.625→0.521 (observed 58.5% / 0.520 with all three 2026-07-15 levers).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* PasHub SAP accuracy: restore 3 dropped §2 infiltration inputs (from_site_notes)
`from_site_notes` uniquely dropped three ventilation inputs that the gov-API
and Elmhurst sibling mappers all set, causing a systematic cohort-wide SAP
under-rate vs the company's own accredited SAP-10.2 certs
(property_baseline_performance.effective_sap_score, portfolio 838):
1. percent_draughtproofed — never set, so §2(15) window infiltration was
pinned at its 0.25-ACH worst case on every dwelling. Reconstructed as the
area-weighted % of draught-proofed windows (mirrors Elmhurst
`draught_proofing_percent`). Dominant term.
2. Upper-storey +0.25 m joist void — omitted; now added, matching
`_UPPER_FLOOR_HEIGHT_ADD_M` on the gov-API/Elmhurst paths. This
RE-ADJUDICATES #1601 ("keep raw"): that call was confounded by the
then-present draught-proofing drop suppressing every verified dwelling.
3. sheltered_sides — left None → calculator's flat default of 2, which
over-shelters end/semi/detached (RdSAP §S5 = 1/1/0). Now derived from
built form.
The three are NON-ADDITIVE — each overshoots alone (which is why all three
were previously rejected individually) — but together they land all 7
verified ground-truth dwellings toward truth (none regress) and every built
form near zero. Cohort 62.1% -> 83.3% within-0.5, MAE 0.507 -> 0.377.
Guardrails: gov-API RdSAP corpus unchanged (78.9%, these are from_site_notes-
only helpers). pashub harness 58.5% -> 79.5% / MAE 0.520 -> 0.389; ratchets
tightened 0.58->0.78 and 0.521->0.40. pyright zero-new (mapper baseline 39).
New focused tests in TestFromSiteNotesInfiltrationFixes cover all three
fields incl. area-weighting and the built-form shelter map; goldens updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
pytest.approx is untyped under pyright strict, so the new assertions added
3 errors against the repo's zero-new-errors bar. Switches to the abs(...)
<= tol form already dominant in this file. No behaviour change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
950 boilers / 951 CHP / 952 community heat pump differ only in plant
efficiency, which SAP 10.2 §4.3 puts in the Table 12 heat price — so the
dwelling's delivered-heat demand must not vary with it. Passes on the
previous commit's expression; verified RED against the prior impl, where
951 leaked 0.80/0.75 and 952 under-billed its DHW fuel by 3.15×.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SAP 10.2 §4.3 (p.24) puts the water-heating efficiency in the Table 12
heat price, and C4 (p.53) scopes the plant efficiency to CO2/PE only, so
worksheet (310) = (64) × (305a) × (306) carries no efficiency term. The
dwelling-side divisor was applying it a second time.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
