refetch_epc=True fetched a fresh cert but the recency tie-break (strict >)
dropped it whenever the stored row shared the cert's inspection date — so a
row written before a schema field/table existed (e.g. room-in-roof geometry)
could never be healed by re-fetching. The dwelling was modelled on stale
fabric indefinitely.
Reconcile instead of just comparing dates: on an equal inspection date,
refresh a public gov cert (uploaded_file_id NULL) when the fetched cert
differs from the stored one, so a plain re-fetch re-ingests it. One of our
own surveys (PasHub / ECMK / Elmhurst — uploaded_file_id set) is preserved on
the tie, keeping the survey-wins rule (ADR-0001 / #1589); an unchanged public
cert is still not re-persisted.
`_reconcile_lodged` layers the content refresh over `_newer_lodged` (recency
untouched). New `EpcRepository.survey_lodged_uprns` surfaces the row-level
uploaded_file_id provenance the hydrated EpcPropertyData drops. Together with
the property_id-anchored persist, a refetch_epc=True re-run now heals both a
missing epc_property row and a stale one.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The lodged EPC read keys on UPRN (ADR-0001 recency tie-break), but writes
and the front end both key on property_id. When a dwelling is onboarded
under two portfolios, the shared UPRN surfaces the sibling's lodged row,
so `lodged_epc_is_new` stays False and the modelling run skips persisting
a copy for this property — leaving it with no epc_property row the FE can
read. The EPC is modelled but invisible (portfolio 850 / property 792235,
uprn 100021979925, also under property 753896 in portfolio 830).
Drive the lodged persist off "does THIS property own a lodged row?"
(property_id-scoped) rather than "is the cert new?" (UPRN-scoped): add
`EpcRepository.property_ids_with_lodged_epc` and persist a per-property
copy whenever the cert is new OR the property has no lodged row of its
own. Idempotent (save_batch is replace-by-property_id), and the recency
skip still holds for a property that already owns its row.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The differ and the HubSpot fetch already named notes_for_surveyor, but the
SQLModel had no such field, so check_for_db_update_trigger raised
AttributeError on every deal and the scrape could not diff at all.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Reproducible RdSAP-10 build (build_100010472748.py) + downloaded Input Summary
and SAP Worksheets (accredited SAP 62 D) + the entry crib sheet, as the
ground-truth for the community-heating mapping fix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Apt 2 (Tracey's Place) through from_api_response -> cert_to_inputs ->
calculate_sap_from_inputs: priced as mains gas (fuel 51), SAP 67 D — recovered
from the pre-fix 27 F. Accredited Elmhurst RdSAP is 62 D; lodged 71 C.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Completes the community taxonomy: CHP (302) carries its chp_fraction and
boiler fuel for the SAP 302 worksheet; heat pump (304) rides the existing
fuel/code path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Derive 301 (boilers) / 302 (CHP) / 304 (heat pump) from the lodged
heat_source_type mix and set it on the category-6 main.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Full SAP lodges the heat-network fuel in sap_community_heating_systems (which the
schema dropped); resolve the dominant heat source's Table 12 fuel onto the
category-6 main so the engine prices it as a network, not electricity.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An oil STORAGE combi (Worcester Greenstar Heatslave II, PCDB 18415,
store_type 1) zeroed its internal-store loss: pcdb_combi_loss_override
defers on store_type in {1,2,3} and a combi has no cylinder, so line
(56) dropped to zero — under-costing hot water and over-rating SAP by
+2.0..+4.9 across the five LRHA WAVE 3 oil combis.
SAP 10.2 Table 2 note b) now includes the store loss when the combi's
efficiency is from the PCDB: the PCDB store volume (field 42) and
insulation thickness (field 44) feed Table 2/2a with the Table 2b
storage-combi temperature factor, reproducing the accredited Elmhurst
P960 worksheet exactly (5.4175 kWh/day, Jan 167.94, ~1977 kWh/yr).
Cohort: within-0.5 55.3% -> 58.3%, MAE 0.675 -> 0.531; ratchet
tightened accordingly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Closes#1684. `_main_2_space_heating_fuel_cost_gbp_per_kwh` billed an
electric Main 2 at Main 1's space-heating rate. That was a no-op on a
homogeneous off-peak cohort (storage+storage, direct+direct → equal
rates) but mis-billed the HETEROGENEOUS case: an off-peak STORAGE Main 1
(5.5 p/kWh low) + an on-peak DIRECT-ACTING panel Main 2 (15.29 p/kWh
high) rode the storage low rate, under-costing the panel half and
over-rating SAP.
Flip it to bill Main 2 at its OWN Table 12a Grid 1 SH row, the same
per-system resolution Main 1 and a non-electric Main 2 already use — and
the same one `_main_2_high_rate_fraction` already applies, so the scalar
cost and the high-rate split are now consistent.
The docstring's "regresses certs 13/34 (Parkers Hill / Dunley Road)"
warning was stale: those names appear nowhere but that docstring, and
the full guard suite shows zero new regressions (only the two
pre-existing `test_heating_systems_corpus` fails on main). LRHA WAVE 3
fixture 497655371974 (storage+panel) moves +15.3 → -2.5 vs PasHub; cohort
MAE 0.799 → 0.675 (ratchet re-baselined 0.80 → 0.68). within-0.5 holds at
55.3% — the -2.5 residual is PasHub-vs-Elmhurst divergence, not a bug, so
the fixture stays out of the within-0.5 pin. A ground-truth Elmhurst pin
is queued (see docs/HANDOVER_1684_ELMHURST_INPUTS_497655371974.md).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A dwelling with two main heating systems lodges a "Main Heating 2" block and a
"Percentage of space heating provided by Main Heating 1" split (e.g. 20% fan +
80% slimline storage, or 50% storage + 50% panel heaters). The survey model held
a single `main_heating`, and the extractor read the whole §Heating section
first-match — so it captured only Main 1, modelled it as 100% of the load, and
dropped Main 2 and the split (also bleeding Main 2's fields into Main 1). Every
such dwelling over-rated.
- `HeatingAndHotWater` now holds `main_heating_2` + `main_heating_1_percent`.
- The extractor slices the "Main Heating Systems" region per system on the
"Main Heating N" markers and reads Main 1's percentage; a single-main dwelling
(no "Main Heating 2") parses byte-identically to before.
- `_map_sap_heating` factors the per-system detail into `_pashub_main_heating_
detail(main, fraction)` and emits two `MainHeatingDetail`s, setting Main 2's
`main_heating_fraction = 100 - Main 1 %` (the calculator reads Main 2's share
and gives Main 1 the remainder).
5 of the 6 dual-main dwellings converge (within-0.5 53.4% → 55.3%, MAE 0.799);
single-main fixtures and the gov-API / Elmhurst / Guinness / Wythenshawe corpora
are unaffected. The 6th (497655371974, storage+panel) regresses because the
calculator deliberately bills an electric Main 2 at Main 1's rate
(`_main_2_space_heating_fuel_cost_gbp_per_kwh`, deferred §10a off-peak slice) —
the correct two-main mapping now exposes that calculator gap; tracked separately.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A PasHub Manual-Entry electric boiler lodges "Direct acting" in its "Heating
System (Boiler):" age-band cell. `_PASHUB_MANUAL_BOILER_AGE_BAND_TO_TABLE_4B`
covers only gas/liquid Table 4b rows, so the descriptor resolved to None and the
boiler stayed uncoded — taking the generic 0.80 gas-boiler seasonal efficiency
AND, on a Dual meter, `_table_12a_system_for_main`'s None-fallback that bills the
whole main-heating load at the off-peak LOW rate. The two errors pull opposite
ways, so an identical mis-mapping over-rated one dwelling and under-rated another
(fixture 497712825571 +9.5 on a Dual meter; 461386632387 -6.2 on a Single meter).
A direct-acting electric "boiler with radiators" is SAP 10.2 Table 4a code 191
(direct-acting electric boiler, efficiency 1.00, §12 Rule 3 tariff). Code it
directly in the manual-boiler resolver (gated on fuel Electricity + no PCDB
product), so the single code value drives the efficiency table, the Table 12a
pricing row and the tariff dispatch together. Both fixtures converge:
497712825571 → +0.6, 461386632387 → +0.4. LRHA within-0.5 52.4% → 53.4%,
MAE 0.835. Renamed `_pashub_manual_boiler_table_4b_code` → `_pashub_manual_
boiler_code` (it now also returns the Table 4a electric code).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
