5.9 KiB
Follow-ups from the full-SAP baseline-downgrade work (ADR-0037)
Open items surfaced while fixing the full-SAP mapper / portfolio-796 downgrades.
Each is separate from the full-SAP fix (fix/baseline-downgrades).
1. RdSAP ventilation mapping is inconsistent across schemas
Severity: accuracy bug for MEV/MVHR dwellings. The mappers handle
sap_ventilation four different ways:
17_0+ full-SAP — build via_sap_17_1_ventilation, which mapsmechanical_ventilation_kind. ✅21_0_1— rich inlineSapVentilation(...)incl. the kind. ✅17_1/18_0/20_0_0— builtSapVentilation(sheltered_sides=…)and dropped the kind. Fixed in #1333 (mirror_api_mechanical_ventilation_kind).19_0/21_0_0— set nosap_ventilationat all → theEpcPropertyDatadefault (None). They dropped the entire ventilation block (sheltered sides + kind), not just the kind. Fixed infeat/rdsap-19-21-ventilation: both now constructSapVentilation(...)mirroring 21.0.1, but only for the fields each schema lodges — the top-levelmechanical_ventilationenum (→ kind) andsheltered_sides(frombuilt_form). Neither lodges the fan/flue/vent counts, so those stay unset (RdSAP Table-5 age defaults — see counts note below).
Either way, an MEV/MVHR cert (mechanical_ventilation ≠ 0) is treated as
natural by the affected mappers — wrong §2 ventilation cascade (and heat
recovery). Natural certs (code 0/5 → None) are unaffected.
- For natural-ventilation certs (
mechanical_ventilation = 0, e.g. UPRN 100020603823 / property 726605) it's benign — unmapped →Nonedefaults to natural in the §2 cascade. - For MEV/MVHR certs (
mechanical_ventilation ≠ 0) the calculator treats the dwelling as naturally ventilated, getting the SAP §2 ventilation cascade (and any heat recovery) wrong → mis-scored SAP.
The granular counts (fans/flues/vents) are not a bug: older RdSAP open-data
certs don't lodge them, and the calc correctly uses RdSAP Table-5 age defaults.
percent_draughtproofed is mapped (top-level) and read by the calc.
19.0 / 21.0.0 fix (done, feat/rdsap-19-21-ventilation): both build
SapVentilation(sheltered_sides=…, mechanical_ventilation_kind=…) mirroring
21.0.1. Guarded by the extended mapper-level MVHR test + a sheltered_sides
regression, with the corpus (held 73.3% / MAE 0.774 — it's 21.0.1-only so the
change isn't exercised by it) and the 6002-case mapper-corpus staying green.
Outstanding SAP-accuracy fast-follow: an Elmhurst-anchored MEV/MVHR
RealCertExpectation (the corpus is natural-vent-dominated, so the kind
change isn't exercised by it). Blast radius (still to quantify): count
portfolio-796 certs on schema 19.0 / 21.0.0 with mechanical_ventilation ≠ 0.
2. FE "Main Fuel: Unknown" is FE-side, not a Model mapper gap
main_fuel_type (the gov fuel code) is populated Model-side — UPRN
10093412452 (709791) → 1, UPRN 100020603823 (726605) → 26 — and is persisted
(epc_main_heating_detail.main_fuel_type). So the panel's "Main Fuel: Unknown"
is the front-end either not mapping the code → fuel name or reading a field we
don't populate. Needs an FE-repo (Drizzle/Next) check, not a Model change.
3. Survivor clusters from the post-fix population sweep
scripts/hyde/case_a_population_sweep.py: the representative sample rebaselined
cleanly (0 survivors), but the worst-old-drop sample held 28 survivors
(lodged A/B → new C/D, 15–25 SAP) in tight UPRN clusters (new-build blocks),
spanning multiple schemas (16.0/16.1/17.1/18.0.0) and heating types. No single
mapper-gap signature → likely genuine SAP-2012→10.2 drops for very-high-lodged
new-builds, but triage one in Elmhurst to confirm genuine vs a residual calc
issue before trusting the cohort.
4. Predicted-property display path (e.g. property 721167)
721167 has no lodged EPC (predicted). Its Heating-Control / Main-Fuel /
Ventilation Unknowns come from the prediction synthesis path
(domain/epc_prediction/epc_prediction.py), not the lodged-cert mappers and
not the landlord-override overlay (overlays_from is simulation-time only,
ADR-0032 — it never writes the persisted/displayed EPC). Investigated and fixed
on fix/predicted-property-display-unknowns:
- Heating Control — FIXED (Model).
_apply_heating_donorcopied the donor's calcsap_heatingcluster but left the display rows (main_heating,main_heating_controls) on the size-representative structural template — incoherent with the donated system, and "Unknown" whenever the template lodged no control row (exactly 721167: nomain_heating_controlsenergy element persisted). Now carries the donor's display heating + control alongside the calc cluster. - Ventilation — FIXED (Model). The prediction never synthesised ventilation
— it kept the template's
sap_ventilation, so a predicted dwelling in an MEV/MVHR neighbourhood was scored + displayed as natural. New_apply_ventilation_modemodesmechanical_ventilation_kindacross the cohort (recency/geo-weighted, mirroring glazing). Natural-vent cohorts mode to None and stay natural (§2 default) — so a genuinely natural dwelling like 721167 correctly stays natural; the fix moves only real MEV/MVHR neighbourhoods. - Main Fuel — NOT a prediction gap.
epc_main_heating_detail.main_fuel_typeis persisted for predicted properties (721167 →26= mains gas, carried from the heating donor). The "Unknown" is the same FE code→name gap as item 2 (the Drizzle/Next repo not mapping the fuel code), not a Model issue.
Both fixes are display-only for the calculator (component-accuracy gate, corpus
harness, and prediction e2e all green); persistence already carries
main_heating_controls (energy element) and ventilation_mechanical_ventilation_kind,
so the synthesised values reach the passport on the next prediction run.