refetch_epc=True fetched a fresh cert but the recency tie-break (strict >)
dropped it whenever the stored row shared the cert's inspection date — so a
row written before a schema field/table existed (e.g. room-in-roof geometry)
could never be healed by re-fetching. The dwelling was modelled on stale
fabric indefinitely.
Reconcile instead of just comparing dates: on an equal inspection date,
refresh a public gov cert (uploaded_file_id NULL) when the fetched cert
differs from the stored one, so a plain re-fetch re-ingests it. One of our
own surveys (PasHub / ECMK / Elmhurst — uploaded_file_id set) is preserved on
the tie, keeping the survey-wins rule (ADR-0001 / #1589); an unchanged public
cert is still not re-persisted.
`_reconcile_lodged` layers the content refresh over `_newer_lodged` (recency
untouched). New `EpcRepository.survey_lodged_uprns` surfaces the row-level
uploaded_file_id provenance the hydrated EpcPropertyData drops. Together with
the property_id-anchored persist, a refetch_epc=True re-run now heals both a
missing epc_property row and a stale one.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The lodged EPC read keys on UPRN (ADR-0001 recency tie-break), but writes
and the front end both key on property_id. When a dwelling is onboarded
under two portfolios, the shared UPRN surfaces the sibling's lodged row,
so `lodged_epc_is_new` stays False and the modelling run skips persisting
a copy for this property — leaving it with no epc_property row the FE can
read. The EPC is modelled but invisible (portfolio 850 / property 792235,
uprn 100021979925, also under property 753896 in portfolio 830).
Drive the lodged persist off "does THIS property own a lodged row?"
(property_id-scoped) rather than "is the cert new?" (UPRN-scoped): add
`EpcRepository.property_ids_with_lodged_epc` and persist a per-property
copy whenever the cert is new OR the property has no lodged row of its
own. Idempotent (save_batch is replace-by-property_id), and the recency
skip still holds for a property that already owns its row.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Apt 2 (Tracey's Place) through from_api_response -> cert_to_inputs ->
calculate_sap_from_inputs: priced as mains gas (fuel 51), SAP 67 D — recovered
from the pre-fix 27 F. Accredited Elmhurst RdSAP is 62 D; lodged 71 C.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An oil STORAGE combi (Worcester Greenstar Heatslave II, PCDB 18415,
store_type 1) zeroed its internal-store loss: pcdb_combi_loss_override
defers on store_type in {1,2,3} and a combi has no cylinder, so line
(56) dropped to zero — under-costing hot water and over-rating SAP by
+2.0..+4.9 across the five LRHA WAVE 3 oil combis.
SAP 10.2 Table 2 note b) now includes the store loss when the combi's
efficiency is from the PCDB: the PCDB store volume (field 42) and
insulation thickness (field 44) feed Table 2/2a with the Table 2b
storage-combi temperature factor, reproducing the accredited Elmhurst
P960 worksheet exactly (5.4175 kWh/day, Jan 167.94, ~1977 kWh/yr).
Cohort: within-0.5 55.3% -> 58.3%, MAE 0.675 -> 0.531; ratchet
tightened accordingly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Closes#1684. `_main_2_space_heating_fuel_cost_gbp_per_kwh` billed an
electric Main 2 at Main 1's space-heating rate. That was a no-op on a
homogeneous off-peak cohort (storage+storage, direct+direct → equal
rates) but mis-billed the HETEROGENEOUS case: an off-peak STORAGE Main 1
(5.5 p/kWh low) + an on-peak DIRECT-ACTING panel Main 2 (15.29 p/kWh
high) rode the storage low rate, under-costing the panel half and
over-rating SAP.
Flip it to bill Main 2 at its OWN Table 12a Grid 1 SH row, the same
per-system resolution Main 1 and a non-electric Main 2 already use — and
the same one `_main_2_high_rate_fraction` already applies, so the scalar
cost and the high-rate split are now consistent.
The docstring's "regresses certs 13/34 (Parkers Hill / Dunley Road)"
warning was stale: those names appear nowhere but that docstring, and
the full guard suite shows zero new regressions (only the two
pre-existing `test_heating_systems_corpus` fails on main). LRHA WAVE 3
fixture 497655371974 (storage+panel) moves +15.3 → -2.5 vs PasHub; cohort
MAE 0.799 → 0.675 (ratchet re-baselined 0.80 → 0.68). within-0.5 holds at
55.3% — the -2.5 residual is PasHub-vs-Elmhurst divergence, not a bug, so
the fixture stays out of the within-0.5 pin. A ground-truth Elmhurst pin
is queued (see docs/HANDOVER_1684_ELMHURST_INPUTS_497655371974.md).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An electric room/panel-heater dwelling (main SAP code 691) on a Dual meter is a
10-hour tariff by RdSAP 10 §12 Rule 3, so its main heating bills the secondary's
identical electricity at a 0.50-blended 11.09 p/kWh. Its SECONDARY heating,
however, resolved its tariff from the bare meter string — where a Dual meter
defaults to 7-hour (`tariff_from_meter_type`) — and billed at the 7-hour
all-high-rate 15.29 p/kWh. The ~10% secondary fraction was over-priced by ~4 p/kWh,
under-rating SAP by ~1 point across the LRHA WAVE 3 code-691 cohort.
Route the dwelling's resolved `_rdsap_tariff(epc)` into `_secondary_fuel_cost_gbp_
per_kwh` / `_secondary_off_peak_rate_gbp_per_kwh`, exactly as the main-heating cost
path (`_space_heating_fuel_cost_gbp_per_kwh`) and the secondary CO2 path
(`_secondary_heating_co2_factor_kg_per_kwh`) already do — removing the internal
meter-only resolution that diverged from the dwelling's single §12 tariff.
Storage-heater dwellings (7-hour by §12) are unchanged — their secondary correctly
stays at 15.29 p (regression-pinned in test_cert_to_inputs). LRHA WAVE 3 within-0.5
49.5% → 52.4%, MAE 0.977; gov-API, Elmhurst RealCert, Guinness and Wythenshawe
corpora unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#1672 (which includes #1666's main_heating_fraction normalisation at the mapper)
is merged into main. The decimal-fraction cohort now maps to integer percent
before save, so the INTEGER column no longer truncates and all 1000 corpus certs
round-trip their SAP to 1e-9. Remove the temporary _is_pre_1666_decimal_fraction
skip so the gate asserts a hard zero over the whole corpus.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The accuracy corpus scores straight off the mapped fixture and never saves, so it
is structurally blind to any field the calculator reads but the schema drops
(#1665). This gate maps -> saves to the ephemeral test DB -> reads back ->
re-scores every corpus cert and asserts the reloaded SAP equals the in-memory SAP
to 1e-9. It would have caught cylinder_heat_loss / room-in-roof / roof-windows /
wall_u_value on day one; a new drop now fails here immediately.
Save+read+rollback per cert keeps the graph off disk (bounded, ~18s over 1000
certs). One documented, precise skip: the pre-#1666 decimal main_heating_fraction
cohort, whose INTEGER column truncates the raw decimal on save (0.8 -> 1) — fixed
upstream by #1666 (mapper normalisation, on #1672). Remove the skip once #1672 is
in main so the gate asserts a hard zero.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SapBuildingPart.wall_u_value (the gov-EPC lodged wall U, authoritative over the
derived default) had no column and dropped on save, so the reloaded dwelling fell
back to the age-band default U — -2.32 SAP on the 2 affected RdSAP corpus certs.
Surfaced by the new DB-round-trip gate. The prior _UNPERSISTED_ALLOWLIST
justification ("deliberate — full-SAP only; RdSAP re-model uses default U-values")
was stale: the cascade DOES honour the lodged U on the API path.
Add wall_u_value to EpcBuildingPartModel, write in from_domain, reconstruct in
_to_building_part, drop the allowlist entry. Verified: certs 39036600 / 38137278
round-trip 72.43 / 69.11 exactly. Needs an added FE column
(epc_building_part.wall_u_value, double precision NULL) on assessment-model#443.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SapRoomInRoof.detailed_surfaces (+ the gable/common-wall scalars) had no columns,
so 107 corpus certs lost their whole §3.9/§3.10 room-in-roof geometry on the DB
round-trip and fell back to the Simplified all-elements age-band default —
worst case -15.21 SAP (#1665 / #1664).
Add EpcRoomInRoofModel (0..1 per building part, unique FK) + EpcRoomInRoofSurface
Model (0..n, surface_index order). Save nests via RETURNING (building part -> RIR
-> surfaces); delete clears bottom-up (surfaces -> room_in_roof -> parts) since
the FE FKs are ON DELETE no action; both read paths group + reconstruct in a new
_to_room_in_roof / _to_rir_surface, replacing the two-field flat rebuild in
_to_building_part. The flat room_in_roof_* columns on epc_building_part are
superseded and dropped in a follow-up (#1664). Nullable is preserved on
insulation_thickness_mm / u_value (null vs 0 is the Table 17-vs-18 branch); area
is unconstrained (a §3.9.2 absent-gable adjustment is signed). Drop the 8
_UNPERSISTED_ALLOWLIST entries. Verified: worst RIR cert 10012119141 round-trip
-15.21 -> 0.0000 with detailed_surfaces at deep equality.
Companion FE migration: assessment-model#443 (epc_room_in_roof +
epc_room_in_roof_surface). Deploy gate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
EpcPropertyData.sap_roof_windows (List[SapRoofWindow]) had no table, so 55
corpus certs lost their rooflights on the DB round-trip and were re-scored
without them (§3 (27a) heat transmission + §6 solar gain). Add EpcRoofWindowModel
(mirrors EpcPhotovoltaicArrayModel) with a roof_window_index order column and the
resolved SapRoofWindow fields; wire save (batch insert), delete (both delete
paths), both read paths (get + get_many) and _compose reconstruction; drop the
two _UNPERSISTED_ALLOWLIST entries.
u_value_raw / g_perpendicular / frame_factor are stored resolved, not re-derived:
the Table 24 lookup is keyed on glazing_type AND glazing_gap, and glazing_gap is
not retained on SapRoofWindow, so they can't be reconstructed from the persisted
fields. window_location is JSONB (int on API / str on site-notes). Verified: a
roof-window cert round-trips sap_roof_windows to deep equality.
Companion FE migration: assessment-model#443 (epc_roof_window). Deploy gate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SapAlternativeWall.u_value (heat_transmission.py:1616), .wall_thickness_mm
(:1645) and .is_basement (:1618) were read by the calculator but had no column,
so they dropped on save. Latent today (0 gov-API certs lodge them — live only on
the Elmhurst/site-notes path), but is_basement is the nastiest shape: it does not
null out, it FLIPS meaning (True -> None -> is_basement_wall reads False),
silently switching off the RdSAP §5.17 / Table 23 basement-wall U path.
Add alt_wall_{1,2}_{u_value,thickness_mm,is_basement} to EpcBuildingPartModel,
write in from_domain, reconstruct in _to_alt_wall, and drop the three
_UNPERSISTED_ALLOWLIST entries so the structural guard enforces them. is_basement
is nullable (None "not stated" != False "not a basement"; a NOT NULL DEFAULT
false re-enables the code-6 heuristic — same reasoning as #1661's wall_is_basement).
Companion assessment-model migration: six nullable columns on epc_building_part
(deploy gate). Round-trip + field-coverage guard green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
