* Electric-immersion DHW: bill at 100%, not the space boiler's efficiency 🟩
A separate electric immersion heater (SAP water-heating code 903) is 100%
efficient (SAP 10.2 Table 4a) and the space-heating boiler provides no water
heating, so Appendix D2.1 Eq D1 (the boiler seasonal-efficiency cascade) must
not apply to it. But `_water_heating_main` resolves DHW to the SPACE main — a
gas/oil boiler keeps its PCDB record — so three water-efficiency branches in
cert_to_inputs billed the immersion-heated cylinder at the boiler's ~87% summer
efficiency (a mongrel: electric fuel price x gas-boiler efficiency):
1. the scalar `water_eff = water_pcdb_main.summer_efficiency_pct / 100`,
2. the SAP §9.4.11 / Table 4c(2) -5pp no-interlock adjustment, and
3. the Eq D1 (winter, summer) seasonal pair from `pcdb_main`.
Gate all three on `not dhw_is_electric_immersion`; the correct immersion path
(`_water_efficiency_with_category_inherit` -> 1.0, Eq D1 off) then applies,
mirroring the Table 3 zero-primary-loss gate already present for WHC 903.
General bug — fires for any WHC-903 dwelling whose space main is a PCDB gas/oil
boiler. It surfaced via the PasHub campaign's #1600 no-water-heating default
(WHC 999 -> 903) on a gas-combi dwelling: 58 Hackle St M11 4WU, SAP 55.30 ->
57.67 (pre_sap 58, verified 59).
Guardrails: the gov-API RdSAP corpus IMPROVES 78.8% -> 78.9% within-0.5, MAE
0.625 -> 0.622 (a handful of corpus certs with a boiler space main + electric
immersion move closer to accredited) — MAE ceiling ratcheted 0.626 -> 0.625.
Regression pinned in test_cert_to_inputs (RED before / GREEN after). pyright
0-new (cert_to_inputs baseline 30).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* PasHub SAP accuracy: thread low-energy lights + draught lobby (from_site_notes) 🟩
Two systematic silent-drops in the PAS Hub `from_site_notes` path, each a field
the gov-API/Elmhurst mappers thread but site-notes dropped, letting a favourable
default reach the SAP-10.2 calculator. Validated against the correct pashub
oracle (property_baseline_performance.effective_sap_score, portfolio 838):
cohort within-0.5 55.1% → 62.6%, MAE 0.599 → 0.521.
- Low-energy lights: when "exact LED/CFL known = No", PAS Hub lodges an aggregate
"Number of fixed low energy lights?" count. Previously dropped (no dataclass
field / extractor key / mapper thread) → calculator saw 0 low-energy bulbs and
applied the pessimistic L5b/L8c no-data default, under-rating SAP. Now threaded
into low_energy_fixed_lighting_bulbs_count, mirroring from_elmhurst_site_notes.
~6% of the cohort; resolves 58 Hackle (−0.43 vs oracle 59 with #1615).
- Draught lobby: _map_sap_ventilation set only the legacy `draught_lobby` field,
never the canonical `has_draught_lobby` §2 (13) gate the cascade reads, so the
surveyed lobby was ignored and infiltration over-stated. Now mirrors Elmhurst.
Two other audited levers were REJECTED against the pashub oracle (they matched
the gov-cert/RdSAP convention but pashub does not apply them): percent_draughtproofed
(cohort 30.8%) and the §A.2.2 assumed secondary heater (19.2%). A heat-network
control-code fix (2306→2303 for 16 Bingley) is HELD pending spec/Elmhurst
adjudication — it contradicts ADR-0053; see the NOTE on _PASHUB_HEAT_NETWORK_CONTROL_TO_SAP10.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* PasHub heat-network control: room-thermostat-only maps to 2308, not 2306 🟩
The Table 4e Group 3 label "charging linked to use of community heating, room
thermostat only" was mapped to 2306 — the linked-to-use *with-TRVs* code
(control type 3, space charging 1.00). The label explicitly excludes TRVs, so
it is control type 2 / space 1.05 = code 2308/2309 (linked-to-use → DHW 1.00).
2306 asserted TRVs the survey doesn't have and over-rated the cohort's sole
heat-network dwelling (16 Bingley Close 56.89 → 54.01 vs pashub oracle 52).
Verified against SAP 10.2 Table 4e as encoded in cert_to_inputs.py
(_CONTROL_TYPE_BY_CODE 2308→2, space-charging 2308→1.05, DHW 2308→1.00) and the
RdSAP control-label vocabulary in MainheatControlAttributes.py. 2303 (the earlier
audit's guess) is rejected: it is a flat-rate code (DHW 1.05) whose oracle match
was a coincidence of two offsetting spec violations. The residual +2.0 to oracle
52 is the documented SAP-10.2-engine-vs-lodged offset, not a fuel/flags gap
(those were threaded by the #1590 follow-up) — supersedes the ADR-0053 /
HANDOVER_838 "community fuel/flags" attribution.
16 Bingley is the only heat-network fixture in portfolio 838, so this moves one
fixture strictly toward its oracle. Harness ratcheted: within-0.5 0.51→0.58,
MAE 0.625→0.521 (observed 58.5% / 0.520 with all three 2026-07-15 levers).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* PasHub SAP accuracy: restore 3 dropped §2 infiltration inputs (from_site_notes)
`from_site_notes` uniquely dropped three ventilation inputs that the gov-API
and Elmhurst sibling mappers all set, causing a systematic cohort-wide SAP
under-rate vs the company's own accredited SAP-10.2 certs
(property_baseline_performance.effective_sap_score, portfolio 838):
1. percent_draughtproofed — never set, so §2(15) window infiltration was
pinned at its 0.25-ACH worst case on every dwelling. Reconstructed as the
area-weighted % of draught-proofed windows (mirrors Elmhurst
`draught_proofing_percent`). Dominant term.
2. Upper-storey +0.25 m joist void — omitted; now added, matching
`_UPPER_FLOOR_HEIGHT_ADD_M` on the gov-API/Elmhurst paths. This
RE-ADJUDICATES #1601 ("keep raw"): that call was confounded by the
then-present draught-proofing drop suppressing every verified dwelling.
3. sheltered_sides — left None → calculator's flat default of 2, which
over-shelters end/semi/detached (RdSAP §S5 = 1/1/0). Now derived from
built form.
The three are NON-ADDITIVE — each overshoots alone (which is why all three
were previously rejected individually) — but together they land all 7
verified ground-truth dwellings toward truth (none regress) and every built
form near zero. Cohort 62.1% -> 83.3% within-0.5, MAE 0.507 -> 0.377.
Guardrails: gov-API RdSAP corpus unchanged (78.9%, these are from_site_notes-
only helpers). pashub harness 58.5% -> 79.5% / MAE 0.520 -> 0.389; ratchets
tightened 0.58->0.78 and 0.521->0.40. pyright zero-new (mapper baseline 39).
New focused tests in TestFromSiteNotesInfiltrationFixes cover all three
fields incl. area-weighting and the built-form shelter map; goldens updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Enumerates the 3 corpus certs that can move (all WHC 950) individually per
#1592 sub-task D: two improve (one crossing inside 0.5), one worsens and is
handed to sub-task C for an accredited oracle. No loosening required.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
pytest.approx is untyped under pyright strict, so the new assertions added
3 errors against the repo's zero-new-errors bar. Switches to the abs(...)
<= tol form already dominant in this file. No behaviour change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
950 boilers / 951 CHP / 952 community heat pump differ only in plant
efficiency, which SAP 10.2 §4.3 puts in the Table 12 heat price — so the
dwelling's delivered-heat demand must not vary with it. Passes on the
previous commit's expression; verified RED against the prior impl, where
951 leaked 0.80/0.75 and 952 under-billed its DHW fuel by 3.15×.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SAP 10.2 §4.3 (p.24) puts the water-heating efficiency in the Table 12
heat price, and C4 (p.53) scopes the plant efficiency to CO2/PE only, so
worksheet (310) = (64) × (305a) × (306) carries no efficiency term. The
dwelling-side divisor was applying it a second time.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Read stored lodged EPCs whatever refetch_epc says, so a stored assessment
can win the ADR-0001 Recency Tie-Break against the gov register.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ADR-0054 gives a prediction conditioned by an expired Historic EPC the source
"expired", so reporting can tell "no EPC at all" apart from "only an expired
one". It was implemented in IngestionOrchestrator — but the pipeline that
actually runs is modelling_e2e, which predicts through its own _predict_epc and
never adopted it. Result: zero `expired` rows have ever been written, and
`epc_property.source` holds only 'lodged' and 'predicted'.
Two things kept the flavour unreachable here, and both had to go:
- _predict_epc never looked at the historic backup at all, so no prediction
was ever conditioned.
- _flush_writes passed the literal source="predicted", and _PropertyWrite had
no field to carry a flavour, so one would have been dropped before the write
even if computed.
_predict_epc now mirrors IngestionOrchestrator._predict: the expired cert's
stable attributes fill the gaps Landlord Overrides left (overrides still win
where both speak) and condition the cohort, and it returns the source alongside
the EPC. _PropertyWrite carries it; _flush_writes persists it. save_batch
already groups deletes by source family, so a mixed predicted/expired batch
clears the shared slot correctly.
Ships dark: the reader is built only when HISTORIC_EPC_S3_ROOT is set, so
without it every prediction stays plain "predicted", exactly as today.
DEFAULT_S3_ROOT names the dev bucket, so defaulting to it would have a prod
lambda silently reading dev data — Terraform sets the var per environment.
Note this also rescues properties that currently fail to model outright: an
expired cert can supply the property_type no override resolved, where today
that raises UnresolvedPropertyTypeError.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
inspection_date is non-null on epc_property, so it is a better winner for
a shared-UPRN collision than insertion order: the latest-surveyed lodged
row wins. id DESC stays as the deterministic secondary, so a same-UPRN,
same-inspection_date tie falls to the highest id.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An epc_property row can carry a null property_id (FE/historic ingestion
persists EPC rows never linked to a property row), so a property_id-keyed
read silently misses them. UPRN is the durable key both the property row
and the epc_property row share. Repoint the lodged reads (get_for_property
+ get_for_properties) onto UPRN; the predicted reads stay on property_id
because a predicted EPC deep-copies a neighbour's UPRN, so its UPRN column
is never the property's own.
UPRN is not unique on epc_property, so the read tie-break is now load-
bearing (it was dormant under property_id, where the write path guarantees
one lodged row per property): the most recently ingested (highest-id) row
wins. Write path stays keyed on (property_id, source).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Locks the gate edges — a lodged pitched loft below 270mm is recommended, a
loft at 270mm is left alone, and the 'Nmm+' form ('300mm+') parses to its
number and stays ineligible. These passed on arrival (they fell out of the
numeric-parse gate added in the previous commit); pinned as regression guards.
ADR-0063.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Properties 741843/741872 (uprns 100060714155/100021944594) failed
modelling_e2e subtasks 6ac7841a-9338-4efe-97e5-7e0d19b3055d and
c5450f03-5b0d-4068-a3b3-d1470bc0af57 with a bare StopIteration: their
SAP-Schema-15.0 (2011-era LIG-lodged) certs identify the main dwelling's
building part as "Main building" rather than "Main Dwelling", so
from_api_string fell to OTHER and left the property with no MAIN part —
crashing wall_recommendation.py's unguarded next(...).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Review findings on PR #1527:
- The overlay constants, ScoredOption builder, ventilation dependency and
selected_types helper come from the shared _optimiser_fixtures module
(landed on the fabric-first base) instead of local copies; the boiler
overlay is the shared BOILER_OVERLAY (SAP Table 4a code 104, a mains-gas
combi) rather than code 201, which is neither a boiler nor a heat pump.
_IWI_OVERLAY (solid-wall internal, type 3) stays local — no shared
equivalent — and the carbon stubs stay bespoke (the shared StubScorer has
no CO2 knob).
- The optimise_package_fabric_first import is lifted to module scope.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
An unrecognised schema_type used to raise ValueError and abort the whole
call; now from_api_response logs a warning and returns None so one
unmapped cert doesn't break a batch. Schema 15.0 already has a mapper
(PR #1531) so it's unaffected; only genuinely unmapped versions skip.
OpenHousing logs a job but silently drops the appointment when no bookable
resource is sent, so the survey date never lands. log_job already sends the
configured default_resource; amend_job omitted the parameter unless the
request carried an explicit surveyor. Default amend's resource to the
configured surveyor too, and fail loudly at config load on a blank
ABRI_RELAY_DEFAULT_RESOURCE so the misconfiguration can't ship an empty
resource again.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Review findings on PR #1526:
- tests/domain/modelling/_optimiser_fixtures.py is the one home for the
overlay constants, the ScoredOption builder, the additive per-kind
StubScorer and the forced ventilation dependency; test_optimiser.py and
test_optimiser_fabric_first.py had byte-identical copies of each
(and _StubScorer / _VentStubScorer fold into one parameterised stub).
- Fixture worlds are domain-plausible per team convention: the fabric-vs-
heating contrast is a £12,000 EWI against a £3,200 gas boiler rather
than a £500 heat pump undercutting a £1,000 cavity wall; heating
overlays carry real identities (SAP Table 4a code 104 for the boiler,
a PCDF index for the heat pump) instead of code 201 doubling as both;
whole-dwelling double glazing is £3,500, not £500.
- Dead knobs removed: the unused _ROOF_OVERLAY, the always-zero roof
gain, the duplicate _BOILER_OVERLAY, and the nested conditional
expressions in the interaction stubs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Re-attempted entering this SAP-15.0 cert's 4 real N/E/S/W windows (gov-API
sap_windows, 2.516 m^2 each) as separate Elmhurst Openings rows instead of
the accepted single combined South row, to test whether it closes the
residual ~2pt gap (engine 53 vs Elmhurst worksheet 51).
Reproduced live the documented elmhurst_lib.py window-grid limitation:
adding a 2nd row wipes the previously-added row's width/height to 0.00,
in both same-session and fresh-session-per-window attempts. Not fixable
via current Playwright automation. Restored the assessment to its original
single South 10.06 m^2 row (no PDFs re-downloaded, no worksheet change).
Separately confirmed our own engine already models this cert's windows
per-orientation correctly from gov-API sap_windows via
solar_gains.py's ORIENTATION_BY_SAP10_CODE cascade -- this is an Elmhurst
build-tooling ceiling, not an engine or mapper bug. sap_score=53 remains
unchanged; no calculator/mapper code was touched. Full accuracy suite
re-run clean (66 passed/67 skipped/1 xfailed + corpus test passed).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Both flagged mismatches were Elmhurst input errors (same silent-stale-value
contamination class as the earlier chimneys/wall-thickness bugs), not
parser or calculator bugs:
- cylinder_size: build_100010086084.py's water_heating() selected
DropDownListCylinderSize by raw value "2", but this DOM's option values
ARE their visible litre-band text (no "2" among them) — Playwright
silently no-ops on a non-matching value, leaving a prior cert's "Medium
(131-170L)" selection in place. Fixed to match by text ("Normal"), and
added the missing mapper dict entries (Normal/Large litre-suffixed
labels) to _ELMHURST_CYLINDER_SIZE_LABEL_TO_SAP10 (a real mapper-coverage
gap — the calculator raises UnmappedElmhurstLabel rather than silently
mis-mapping).
- boiler_flue_type: the boiler-code search dialog's combined "Balanced/Open
Flue" Table 4b category doesn't drive the separate, independently
selectable RadioButtonListFlueType field, which was left at an inherited
"Balanced" from a prior cert. Fixed space_heating() to explicitly select
"Open" to match the lodged gov-API code. This field isn't consumed by
Sap10Calculator (ML/generator-only), so it had zero effect on the SAP
score.
Re-downloaded elmhurst_summary.pdf/elmhurst_worksheet.pdf after the fixes
(Recommendations page confirmed clean). Elmhurst-PDF-inputs path moved
46->47 SAP from the cylinder-volume correction; gov-API SAP (53) and
Elmhurst's own worksheet (51) are unchanged. Full accuracy suites re-run
clean (67 passed, 67 skipped, 1 xfailed, same as before).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The Elmhurst rebuild for this cert had two carryover bugs from the shared
Playwright session: TextBoxOpenChimneys silently inherited "2" from a prior
cert's build (this cert's register lodges open_fireplaces_count=0, which
the gov-API mapper already reads correctly), and wall thickness inherited
a stale 280mm. Also fixed a real bug in the space_heating() PCDB-ref-clearing
logic: input_value() returns the string "0" when already cleared, which is
truthy in Python, causing a spurious navigate-away that broke the next
E.goto() call.
Rebuilt cleanly (chimneys=0, wall thickness=220mm per RdSAP convention for
unrecorded solid-wall thickness, since CheckBoxWallThicknessUnknown does not
persist via Playwright in this tool version). Elmhurst's own worksheet score
moves from a contaminated 46/48 to 51, closing the gap against this engine's
53-on-gov-API-inputs from ~5-7pt to ~2pt. No mapper.py change was needed --
the gov-API mapper's open_chimneys_count=0 was correct all along.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Built UPRN 100010086084 (the largest gov-API-vs-lodged outlier from the
20-cert batch) in Elmhurst's accredited RdSAP10 tool to get real ground
truth, per feedback that lodged rating isn't a valid comparison target.
Elmhurst worksheet: SAP 46-48 (orientation-dependent), vs this engine's
53 on gov-API inputs -- a real ~5-7pt residual gap. Ruled out with
direct empirical tests: total_floor_area, missing party_wall_length,
region/climate (SAP rating always uses UK-average weather per Appendix
U -- confirmed inert), boiler/secondary efficiency (exact match, 66%/
63% both sides). Confirmed real but NOT blind-fixed (need more than one
cert's evidence, and a wall-U-value patch tested the wrong direction in
isolation): a wall-thickness-unknown U-value fallback mismatch (this
engine 1.70, Elmhurst's own "unknown" resolution 1.40) and a window
U-value divergence (1.85 vs 2.52 W/m2K for the same nominal glazing
pick). Single-window orientation confirmed a ~2pt partial contributor.
Also fixes a small, separate, evidenced gap the build surfaced: the
Elmhurst summary parser didn't recognise the live tool's "Medium
(131-170L)" cylinder-size label (only the bare "Medium").
Full write-up is in the RealCertExpectation comment for this cert in
test_real_cert_sap_accuracy.py and the worklist. Residual gap is
intentionally left open for follow-up, not force-closed.
These are 2011 SAP-2009 lodgements; this calculator implements current
RdSAP10, which has since revised several U-value tables. Closeness to
lodged was never evidence of mapper correctness (nor is a gap evidence
against it) -- the prior commit's comments and worklist entries wrongly
framed 16/20 "matching lodged" as validation and the other 4 as
anomalies needing explanation. Corrected the comments to state what the
batch actually shows: all 20 structurally distinct certs map without
error and produce plausible scores. Real accuracy validation still
needs an Elmhurst RdSAP10 rebuild on the same methodology as the
engine, which remains open.
Sampled all 20 unique property_ids that failed with "Unsupported EPC
schema: 'SAP-Schema-15.0'" across the sub_task history (114 total across
72 subtask rows), captured 20 real live certs, and pinned each as a
RealCertExpectation regression in the accuracy corpus.
16/20 land exact or within 1-3 SAP of the lodged rating, confirming the
door_count/WWHRS/FGHRS/draught-proofing defaults generalise beyond the
original two properties. The other 4 (all solid-brick, uninsulated,
age band C) diverge +6 to +12 — traced to a legitimate SAP-2009-vintage-
lodgement vs RdSAP10-recalculation methodology gap (u_wall() checked
directly against RdSAP10's own Table 6, returns the correct 1.70 — no
mapper defect), not a mapper bug, so pinned to the engine's observed
value rather than tuned to lodged.
