RdSAP 10 §5.9 "U-values of sheltered walls" (PDF p.43): a wall between a
flat/maisonette and an unheated buffer takes SAP 10.2 §3.3's U = 1/(1/U0 + Ru),
and Ru depends on the BUFFER TYPE — "Use Ru = 0.5 m²K/W for corridors and
Ru = 2.1 m²K/W for stairwell."
The buffer type is lodged on `SapFlatDetails.heat_loss_corridor` (0-based per
item 1-11, PDF p.70: 0 none, 1 heated, 2 unheated corridor, 3 unheated
stairwell). That field was read NOWHERE under domain/ — corridor presence was
inferred only from a lodged sheltered alternative wall — so every sheltered wall
took the corridor 0.5 and every stairwell flat over-billed its sheltered wall,
under-rating the dwelling.
Distinct from `_SHELTERED_GABLE_ADDED_RESISTANCE_M2K_W`, which is RdSAP 10
Table 4 (p.22) for a room-in-roof sheltered gable — same 0.5, different rule,
deliberately untouched.
Corpus gauge ratcheted UP: within-0.5 78.8% -> 79.8% (+10 certs), SAP MAE
0.626 -> 0.613, PE MAE 2.9 -> 2.8. Failure set and pyright count identical to
main. Portfolio 824 is stairwell-heavy (60 properties at code 3); property
749898 over-bills 4.44 W/K on a 48 m2 flat.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
RdSAP 10 §3.12 (PDF p.26), the flats/maisonettes floor rules: "There is no heat
loss through the floor if there is another flat below. Otherwise the floor area
of the flat ... is ... a semi-exposed floor if there are unheated premises below
it (e.g. an enclosed garage) ... Semi-exposed (sheltered) floors are treated as
if they were fully exposed." §5.13 (p.48) confirms the U-value identity: Table 20
applies to exposed and semi-exposed alike.
API floor_heat_loss=2 -> floor_type "To unheated space" set no exposure signal,
so on a mid-/top-floor flat the dwelling-level heuristic (has_exposed_floor=False,
assuming a heated dwelling below) went unopposed and the ENTIRE floor was billed
at 0 W/K. 11 properties across portfolios 796+824, mean +4.02 SAP, 10 of 11
over-rating — we judged dwellings more efficient than the accredited assessment
and would withhold retrofit measures they need.
Exemplars: 750018 space heating -53.2% -> -8.1% vs the cert's own RHI figure,
SAP +8.22 -> +1.16; 749898 -43.0% -> +0.9%, SAP +6.29 -> -1.11.
SCOPED TO FLATS ON PURPOSE. §3.12 is the flats/maisonettes section, and the
accredited Elmhurst worksheet disagrees with Table 20 for a house: golden cert
7536-3827-0600-0600-0276 (detached) lodges floor_heat_loss=2 on its Main part and
the worksheet bills it on the BS EN ISO 13370 ground-floor cascade at U 0.97, not
Table 20's 1.20. An unscoped mapper-level fix broke that pin and the
uprn_100050881708 accuracy pin; the scoped fix leaves both untouched.
No regressions: identical failure set and pyright count vs main; the RdSAP-21.0.1
accuracy corpus gauge is unchanged at 78.8% within-0.5 / MAE 0.625 (correctly —
all 7 corpus certs lodging code 2 are houses).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
6035 is built_form 4 (Mid-Terrace), so this PR routes its Exposed Type-1
RR gable to a party gable (U=0.25) instead of the main-wall U. Heat loss
drops: PE resid -0.14 -> -0.92, CO2 -0.0362 -> -0.0549 (SAP still exact 70).
Residuals stay within noise (PE <0.5% of lodged 191). All 113 golden
fixtures pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Electric-immersion DHW: bill at 100%, not the space boiler's efficiency 🟩
A separate electric immersion heater (SAP water-heating code 903) is 100%
efficient (SAP 10.2 Table 4a) and the space-heating boiler provides no water
heating, so Appendix D2.1 Eq D1 (the boiler seasonal-efficiency cascade) must
not apply to it. But `_water_heating_main` resolves DHW to the SPACE main — a
gas/oil boiler keeps its PCDB record — so three water-efficiency branches in
cert_to_inputs billed the immersion-heated cylinder at the boiler's ~87% summer
efficiency (a mongrel: electric fuel price x gas-boiler efficiency):
1. the scalar `water_eff = water_pcdb_main.summer_efficiency_pct / 100`,
2. the SAP §9.4.11 / Table 4c(2) -5pp no-interlock adjustment, and
3. the Eq D1 (winter, summer) seasonal pair from `pcdb_main`.
Gate all three on `not dhw_is_electric_immersion`; the correct immersion path
(`_water_efficiency_with_category_inherit` -> 1.0, Eq D1 off) then applies,
mirroring the Table 3 zero-primary-loss gate already present for WHC 903.
General bug — fires for any WHC-903 dwelling whose space main is a PCDB gas/oil
boiler. It surfaced via the PasHub campaign's #1600 no-water-heating default
(WHC 999 -> 903) on a gas-combi dwelling: 58 Hackle St M11 4WU, SAP 55.30 ->
57.67 (pre_sap 58, verified 59).
Guardrails: the gov-API RdSAP corpus IMPROVES 78.8% -> 78.9% within-0.5, MAE
0.625 -> 0.622 (a handful of corpus certs with a boiler space main + electric
immersion move closer to accredited) — MAE ceiling ratcheted 0.626 -> 0.625.
Regression pinned in test_cert_to_inputs (RED before / GREEN after). pyright
0-new (cert_to_inputs baseline 30).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* PasHub SAP accuracy: thread low-energy lights + draught lobby (from_site_notes) 🟩
Two systematic silent-drops in the PAS Hub `from_site_notes` path, each a field
the gov-API/Elmhurst mappers thread but site-notes dropped, letting a favourable
default reach the SAP-10.2 calculator. Validated against the correct pashub
oracle (property_baseline_performance.effective_sap_score, portfolio 838):
cohort within-0.5 55.1% → 62.6%, MAE 0.599 → 0.521.
- Low-energy lights: when "exact LED/CFL known = No", PAS Hub lodges an aggregate
"Number of fixed low energy lights?" count. Previously dropped (no dataclass
field / extractor key / mapper thread) → calculator saw 0 low-energy bulbs and
applied the pessimistic L5b/L8c no-data default, under-rating SAP. Now threaded
into low_energy_fixed_lighting_bulbs_count, mirroring from_elmhurst_site_notes.
~6% of the cohort; resolves 58 Hackle (−0.43 vs oracle 59 with #1615).
- Draught lobby: _map_sap_ventilation set only the legacy `draught_lobby` field,
never the canonical `has_draught_lobby` §2 (13) gate the cascade reads, so the
surveyed lobby was ignored and infiltration over-stated. Now mirrors Elmhurst.
Two other audited levers were REJECTED against the pashub oracle (they matched
the gov-cert/RdSAP convention but pashub does not apply them): percent_draughtproofed
(cohort 30.8%) and the §A.2.2 assumed secondary heater (19.2%). A heat-network
control-code fix (2306→2303 for 16 Bingley) is HELD pending spec/Elmhurst
adjudication — it contradicts ADR-0053; see the NOTE on _PASHUB_HEAT_NETWORK_CONTROL_TO_SAP10.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* PasHub heat-network control: room-thermostat-only maps to 2308, not 2306 🟩
The Table 4e Group 3 label "charging linked to use of community heating, room
thermostat only" was mapped to 2306 — the linked-to-use *with-TRVs* code
(control type 3, space charging 1.00). The label explicitly excludes TRVs, so
it is control type 2 / space 1.05 = code 2308/2309 (linked-to-use → DHW 1.00).
2306 asserted TRVs the survey doesn't have and over-rated the cohort's sole
heat-network dwelling (16 Bingley Close 56.89 → 54.01 vs pashub oracle 52).
Verified against SAP 10.2 Table 4e as encoded in cert_to_inputs.py
(_CONTROL_TYPE_BY_CODE 2308→2, space-charging 2308→1.05, DHW 2308→1.00) and the
RdSAP control-label vocabulary in MainheatControlAttributes.py. 2303 (the earlier
audit's guess) is rejected: it is a flat-rate code (DHW 1.05) whose oracle match
was a coincidence of two offsetting spec violations. The residual +2.0 to oracle
52 is the documented SAP-10.2-engine-vs-lodged offset, not a fuel/flags gap
(those were threaded by the #1590 follow-up) — supersedes the ADR-0053 /
HANDOVER_838 "community fuel/flags" attribution.
16 Bingley is the only heat-network fixture in portfolio 838, so this moves one
fixture strictly toward its oracle. Harness ratcheted: within-0.5 0.51→0.58,
MAE 0.625→0.521 (observed 58.5% / 0.520 with all three 2026-07-15 levers).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* PasHub SAP accuracy: restore 3 dropped §2 infiltration inputs (from_site_notes)
`from_site_notes` uniquely dropped three ventilation inputs that the gov-API
and Elmhurst sibling mappers all set, causing a systematic cohort-wide SAP
under-rate vs the company's own accredited SAP-10.2 certs
(property_baseline_performance.effective_sap_score, portfolio 838):
1. percent_draughtproofed — never set, so §2(15) window infiltration was
pinned at its 0.25-ACH worst case on every dwelling. Reconstructed as the
area-weighted % of draught-proofed windows (mirrors Elmhurst
`draught_proofing_percent`). Dominant term.
2. Upper-storey +0.25 m joist void — omitted; now added, matching
`_UPPER_FLOOR_HEIGHT_ADD_M` on the gov-API/Elmhurst paths. This
RE-ADJUDICATES #1601 ("keep raw"): that call was confounded by the
then-present draught-proofing drop suppressing every verified dwelling.
3. sheltered_sides — left None → calculator's flat default of 2, which
over-shelters end/semi/detached (RdSAP §S5 = 1/1/0). Now derived from
built form.
The three are NON-ADDITIVE — each overshoots alone (which is why all three
were previously rejected individually) — but together they land all 7
verified ground-truth dwellings toward truth (none regress) and every built
form near zero. Cohort 62.1% -> 83.3% within-0.5, MAE 0.507 -> 0.377.
Guardrails: gov-API RdSAP corpus unchanged (78.9%, these are from_site_notes-
only helpers). pashub harness 58.5% -> 79.5% / MAE 0.520 -> 0.389; ratchets
tightened 0.58->0.78 and 0.521->0.40. pyright zero-new (mapper baseline 39).
New focused tests in TestFromSiteNotesInfiltrationFixes cover all three
fields incl. area-weighting and the built-form shelter map; goldens updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
pytest.approx is untyped under pyright strict, so the new assertions added
3 errors against the repo's zero-new-errors bar. Switches to the abs(...)
<= tol form already dominant in this file. No behaviour change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
950 boilers / 951 CHP / 952 community heat pump differ only in plant
efficiency, which SAP 10.2 §4.3 puts in the Table 12 heat price — so the
dwelling's delivered-heat demand must not vary with it. Passes on the
previous commit's expression; verified RED against the prior impl, where
951 leaked 0.80/0.75 and 952 under-billed its DHW fuel by 3.15×.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SAP 10.2 §4.3 (p.24) puts the water-heating efficiency in the Table 12
heat price, and C4 (p.53) scopes the plant efficiency to CO2/PE only, so
worksheet (310) = (64) × (305a) × (306) carries no efficiency term. The
dwelling-side divisor was applying it a second time.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Locks the gate edges — a lodged pitched loft below 270mm is recommended, a
loft at 270mm is left alone, and the 'Nmm+' form ('300mm+') parses to its
number and stays ineligible. These passed on arrival (they fell out of the
numeric-parse gate added in the previous commit); pinned as regression guards.
ADR-0063.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review findings on PR #1527:
- The overlay constants, ScoredOption builder, ventilation dependency and
selected_types helper come from the shared _optimiser_fixtures module
(landed on the fabric-first base) instead of local copies; the boiler
overlay is the shared BOILER_OVERLAY (SAP Table 4a code 104, a mains-gas
combi) rather than code 201, which is neither a boiler nor a heat pump.
_IWI_OVERLAY (solid-wall internal, type 3) stays local — no shared
equivalent — and the carbon stubs stay bespoke (the shared StubScorer has
no CO2 knob).
- The optimise_package_fabric_first import is lifted to module scope.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review findings on PR #1526:
- tests/domain/modelling/_optimiser_fixtures.py is the one home for the
overlay constants, the ScoredOption builder, the additive per-kind
StubScorer and the forced ventilation dependency; test_optimiser.py and
test_optimiser_fabric_first.py had byte-identical copies of each
(and _StubScorer / _VentStubScorer fold into one parameterised stub).
- Fixture worlds are domain-plausible per team convention: the fabric-vs-
heating contrast is a £12,000 EWI against a £3,200 gas boiler rather
than a £500 heat pump undercutting a £1,000 cavity wall; heating
overlays carry real identities (SAP Table 4a code 104 for the boiler,
a PCDF index for the heat pump) instead of code 201 doubling as both;
whole-dwelling double glazing is £3,500, not £500.
- Dead knobs removed: the unused _ROOF_OVERLAY, the always-zero roof
gain, the duplicate _BOILER_OVERLAY, and the nested conditional
expressions in the interaction stubs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Re-attempted entering this SAP-15.0 cert's 4 real N/E/S/W windows (gov-API
sap_windows, 2.516 m^2 each) as separate Elmhurst Openings rows instead of
the accepted single combined South row, to test whether it closes the
residual ~2pt gap (engine 53 vs Elmhurst worksheet 51).
Reproduced live the documented elmhurst_lib.py window-grid limitation:
adding a 2nd row wipes the previously-added row's width/height to 0.00,
in both same-session and fresh-session-per-window attempts. Not fixable
via current Playwright automation. Restored the assessment to its original
single South 10.06 m^2 row (no PDFs re-downloaded, no worksheet change).
Separately confirmed our own engine already models this cert's windows
per-orientation correctly from gov-API sap_windows via
solar_gains.py's ORIENTATION_BY_SAP10_CODE cascade -- this is an Elmhurst
build-tooling ceiling, not an engine or mapper bug. sap_score=53 remains
unchanged; no calculator/mapper code was touched. Full accuracy suite
re-run clean (66 passed/67 skipped/1 xfailed + corpus test passed).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Both flagged mismatches were Elmhurst input errors (same silent-stale-value
contamination class as the earlier chimneys/wall-thickness bugs), not
parser or calculator bugs:
- cylinder_size: build_100010086084.py's water_heating() selected
DropDownListCylinderSize by raw value "2", but this DOM's option values
ARE their visible litre-band text (no "2" among them) — Playwright
silently no-ops on a non-matching value, leaving a prior cert's "Medium
(131-170L)" selection in place. Fixed to match by text ("Normal"), and
added the missing mapper dict entries (Normal/Large litre-suffixed
labels) to _ELMHURST_CYLINDER_SIZE_LABEL_TO_SAP10 (a real mapper-coverage
gap — the calculator raises UnmappedElmhurstLabel rather than silently
mis-mapping).
- boiler_flue_type: the boiler-code search dialog's combined "Balanced/Open
Flue" Table 4b category doesn't drive the separate, independently
selectable RadioButtonListFlueType field, which was left at an inherited
"Balanced" from a prior cert. Fixed space_heating() to explicitly select
"Open" to match the lodged gov-API code. This field isn't consumed by
Sap10Calculator (ML/generator-only), so it had zero effect on the SAP
score.
Re-downloaded elmhurst_summary.pdf/elmhurst_worksheet.pdf after the fixes
(Recommendations page confirmed clean). Elmhurst-PDF-inputs path moved
46->47 SAP from the cylinder-volume correction; gov-API SAP (53) and
Elmhurst's own worksheet (51) are unchanged. Full accuracy suites re-run
clean (67 passed, 67 skipped, 1 xfailed, same as before).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The Elmhurst rebuild for this cert had two carryover bugs from the shared
Playwright session: TextBoxOpenChimneys silently inherited "2" from a prior
cert's build (this cert's register lodges open_fireplaces_count=0, which
the gov-API mapper already reads correctly), and wall thickness inherited
a stale 280mm. Also fixed a real bug in the space_heating() PCDB-ref-clearing
logic: input_value() returns the string "0" when already cleared, which is
truthy in Python, causing a spurious navigate-away that broke the next
E.goto() call.
Rebuilt cleanly (chimneys=0, wall thickness=220mm per RdSAP convention for
unrecorded solid-wall thickness, since CheckBoxWallThicknessUnknown does not
persist via Playwright in this tool version). Elmhurst's own worksheet score
moves from a contaminated 46/48 to 51, closing the gap against this engine's
53-on-gov-API-inputs from ~5-7pt to ~2pt. No mapper.py change was needed --
the gov-API mapper's open_chimneys_count=0 was correct all along.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Built UPRN 100010086084 (the largest gov-API-vs-lodged outlier from the
20-cert batch) in Elmhurst's accredited RdSAP10 tool to get real ground
truth, per feedback that lodged rating isn't a valid comparison target.
Elmhurst worksheet: SAP 46-48 (orientation-dependent), vs this engine's
53 on gov-API inputs -- a real ~5-7pt residual gap. Ruled out with
direct empirical tests: total_floor_area, missing party_wall_length,
region/climate (SAP rating always uses UK-average weather per Appendix
U -- confirmed inert), boiler/secondary efficiency (exact match, 66%/
63% both sides). Confirmed real but NOT blind-fixed (need more than one
cert's evidence, and a wall-U-value patch tested the wrong direction in
isolation): a wall-thickness-unknown U-value fallback mismatch (this
engine 1.70, Elmhurst's own "unknown" resolution 1.40) and a window
U-value divergence (1.85 vs 2.52 W/m2K for the same nominal glazing
pick). Single-window orientation confirmed a ~2pt partial contributor.
Also fixes a small, separate, evidenced gap the build surfaced: the
Elmhurst summary parser didn't recognise the live tool's "Medium
(131-170L)" cylinder-size label (only the bare "Medium").
Full write-up is in the RealCertExpectation comment for this cert in
test_real_cert_sap_accuracy.py and the worklist. Residual gap is
intentionally left open for follow-up, not force-closed.
These are 2011 SAP-2009 lodgements; this calculator implements current
RdSAP10, which has since revised several U-value tables. Closeness to
lodged was never evidence of mapper correctness (nor is a gap evidence
against it) -- the prior commit's comments and worklist entries wrongly
framed 16/20 "matching lodged" as validation and the other 4 as
anomalies needing explanation. Corrected the comments to state what the
batch actually shows: all 20 structurally distinct certs map without
error and produce plausible scores. Real accuracy validation still
needs an Elmhurst RdSAP10 rebuild on the same methodology as the
engine, which remains open.
Sampled all 20 unique property_ids that failed with "Unsupported EPC
schema: 'SAP-Schema-15.0'" across the sub_task history (114 total across
72 subtask rows), captured 20 real live certs, and pinned each as a
RealCertExpectation regression in the accuracy corpus.
16/20 land exact or within 1-3 SAP of the lodged rating, confirming the
door_count/WWHRS/FGHRS/draught-proofing defaults generalise beyond the
original two properties. The other 4 (all solid-brick, uninsulated,
age band C) diverge +6 to +12 — traced to a legitimate SAP-2009-vintage-
lodgement vs RdSAP10-recalculation methodology gap (u_wall() checked
directly against RdSAP10's own Table 6, returns the correct 1.70 — no
mapper defect), not a mapper bug, so pinned to the engine's observed
value rather than tuned to lodged.
Review feedback: cavity and internal wall insulation are competing options,
so a package selecting both read as nonsense even in a synthetic fixture.
The behaviour under test (a dependency triggered in both phases injects
once) now uses an airtightness pair that genuinely coexists — cavity wall
in phase 1, double glazing re-entering in phase 2 — with the same numbers.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Documents behaviour already delivered: with no SAP target both phases run
max-gain, so the envelope still consumes the budget first. Test passed on
arrival.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Documents behaviour already delivered: an empty phase 1 meets no target, so
phase 2 optimises every group — identical to a plain run. Test passed on
arrival.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Documents behaviour already delivered: phase 2 optimises every group phase 1
did not consume, with signals re-scored against the fabric-applied dwelling,
so glazing skipped on raw-baseline merit re-enters when it closes the target.
Test passed on arrival.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Documents behaviour already delivered by the phase-2 structure: the leftover
budget is the envelope minus the committed fabric cost, so a heating system
the plain optimiser would buy can be priced out. Test passed on arrival.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
'Cancelled' and 'no show' were separate strings in the outcome vocabulary,
but HubSpot only ever emits the one dropdown value 'Cancelled / No Show' -
so neither phantom string could ever match. Replace both with the real
value (mapped to NOACCESS, best-guess pending client), which also fixes the
long-standing differ mismatch: the abandonment trigger now actually fires on
Cancelled / No Show. Drops the now-unused REQT enum member.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Complete the outcome-to-code table so all five UNSUCCESSFUL_OUTCOMES map:
cancelled -> REQT (new enum member) and no show -> NOACCESS join the
existing three. Only no answer -> CARDED is firm; the rest are best-guesses
for client confirmation, noted inline with alternatives.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the CARDED placeholder in abandon_reason_for_outcome with the
real outcome-to-code table from the client: No Answer->CARDED,
Tenant Refusal->CRA (ambiguous, client follow-up pending),
Not Viable->NOACCESS. Only these three outcomes trip the abandonment
trigger today; any empty/unrecognised outcome raises
UnmappableOutcomeError since abandon_reason is mandatory.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
