Apt 2 (Tracey's Place) through from_api_response -> cert_to_inputs ->
calculate_sap_from_inputs: priced as mains gas (fuel 51), SAP 67 D — recovered
from the pre-fix 27 F. Accredited Elmhurst RdSAP is 62 D; lodged 71 C.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An oil STORAGE combi (Worcester Greenstar Heatslave II, PCDB 18415,
store_type 1) zeroed its internal-store loss: pcdb_combi_loss_override
defers on store_type in {1,2,3} and a combi has no cylinder, so line
(56) dropped to zero — under-costing hot water and over-rating SAP by
+2.0..+4.9 across the five LRHA WAVE 3 oil combis.
SAP 10.2 Table 2 note b) now includes the store loss when the combi's
efficiency is from the PCDB: the PCDB store volume (field 42) and
insulation thickness (field 44) feed Table 2/2a with the Table 2b
storage-combi temperature factor, reproducing the accredited Elmhurst
P960 worksheet exactly (5.4175 kWh/day, Jan 167.94, ~1977 kWh/yr).
Cohort: within-0.5 55.3% -> 58.3%, MAE 0.675 -> 0.531; ratchet
tightened accordingly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Closes#1684. `_main_2_space_heating_fuel_cost_gbp_per_kwh` billed an
electric Main 2 at Main 1's space-heating rate. That was a no-op on a
homogeneous off-peak cohort (storage+storage, direct+direct → equal
rates) but mis-billed the HETEROGENEOUS case: an off-peak STORAGE Main 1
(5.5 p/kWh low) + an on-peak DIRECT-ACTING panel Main 2 (15.29 p/kWh
high) rode the storage low rate, under-costing the panel half and
over-rating SAP.
Flip it to bill Main 2 at its OWN Table 12a Grid 1 SH row, the same
per-system resolution Main 1 and a non-electric Main 2 already use — and
the same one `_main_2_high_rate_fraction` already applies, so the scalar
cost and the high-rate split are now consistent.
The docstring's "regresses certs 13/34 (Parkers Hill / Dunley Road)"
warning was stale: those names appear nowhere but that docstring, and
the full guard suite shows zero new regressions (only the two
pre-existing `test_heating_systems_corpus` fails on main). LRHA WAVE 3
fixture 497655371974 (storage+panel) moves +15.3 → -2.5 vs PasHub; cohort
MAE 0.799 → 0.675 (ratchet re-baselined 0.80 → 0.68). within-0.5 holds at
55.3% — the -2.5 residual is PasHub-vs-Elmhurst divergence, not a bug, so
the fixture stays out of the within-0.5 pin. A ground-truth Elmhurst pin
is queued (see docs/HANDOVER_1684_ELMHURST_INPUTS_497655371974.md).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An electric room/panel-heater dwelling (main SAP code 691) on a Dual meter is a
10-hour tariff by RdSAP 10 §12 Rule 3, so its main heating bills the secondary's
identical electricity at a 0.50-blended 11.09 p/kWh. Its SECONDARY heating,
however, resolved its tariff from the bare meter string — where a Dual meter
defaults to 7-hour (`tariff_from_meter_type`) — and billed at the 7-hour
all-high-rate 15.29 p/kWh. The ~10% secondary fraction was over-priced by ~4 p/kWh,
under-rating SAP by ~1 point across the LRHA WAVE 3 code-691 cohort.
Route the dwelling's resolved `_rdsap_tariff(epc)` into `_secondary_fuel_cost_gbp_
per_kwh` / `_secondary_off_peak_rate_gbp_per_kwh`, exactly as the main-heating cost
path (`_space_heating_fuel_cost_gbp_per_kwh`) and the secondary CO2 path
(`_secondary_heating_co2_factor_kg_per_kwh`) already do — removing the internal
meter-only resolution that diverged from the dwelling's single §12 tariff.
Storage-heater dwellings (7-hour by §12) are unchanged — their secondary correctly
stays at 15.29 p (regression-pinned in test_cert_to_inputs). LRHA WAVE 3 within-0.5
49.5% → 52.4%, MAE 0.977; gov-API, Elmhurst RealCert, Guinness and Wythenshawe
corpora unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#1672 (which includes #1666's main_heating_fraction normalisation at the mapper)
is merged into main. The decimal-fraction cohort now maps to integer percent
before save, so the INTEGER column no longer truncates and all 1000 corpus certs
round-trip their SAP to 1e-9. Remove the temporary _is_pre_1666_decimal_fraction
skip so the gate asserts a hard zero over the whole corpus.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The accuracy corpus scores straight off the mapped fixture and never saves, so it
is structurally blind to any field the calculator reads but the schema drops
(#1665). This gate maps -> saves to the ephemeral test DB -> reads back ->
re-scores every corpus cert and asserts the reloaded SAP equals the in-memory SAP
to 1e-9. It would have caught cylinder_heat_loss / room-in-roof / roof-windows /
wall_u_value on day one; a new drop now fails here immediately.
Save+read+rollback per cert keeps the graph off disk (bounded, ~18s over 1000
certs). One documented, precise skip: the pre-#1666 decimal main_heating_fraction
cohort, whose INTEGER column truncates the raw decimal on save (0.8 -> 1) — fixed
upstream by #1666 (mapper normalisation, on #1672). Remove the skip once #1672 is
in main so the gate asserts a hard zero.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SapBuildingPart.wall_u_value (the gov-EPC lodged wall U, authoritative over the
derived default) had no column and dropped on save, so the reloaded dwelling fell
back to the age-band default U — -2.32 SAP on the 2 affected RdSAP corpus certs.
Surfaced by the new DB-round-trip gate. The prior _UNPERSISTED_ALLOWLIST
justification ("deliberate — full-SAP only; RdSAP re-model uses default U-values")
was stale: the cascade DOES honour the lodged U on the API path.
Add wall_u_value to EpcBuildingPartModel, write in from_domain, reconstruct in
_to_building_part, drop the allowlist entry. Verified: certs 39036600 / 38137278
round-trip 72.43 / 69.11 exactly. Needs an added FE column
(epc_building_part.wall_u_value, double precision NULL) on assessment-model#443.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SapRoomInRoof.detailed_surfaces (+ the gable/common-wall scalars) had no columns,
so 107 corpus certs lost their whole §3.9/§3.10 room-in-roof geometry on the DB
round-trip and fell back to the Simplified all-elements age-band default —
worst case -15.21 SAP (#1665 / #1664).
Add EpcRoomInRoofModel (0..1 per building part, unique FK) + EpcRoomInRoofSurface
Model (0..n, surface_index order). Save nests via RETURNING (building part -> RIR
-> surfaces); delete clears bottom-up (surfaces -> room_in_roof -> parts) since
the FE FKs are ON DELETE no action; both read paths group + reconstruct in a new
_to_room_in_roof / _to_rir_surface, replacing the two-field flat rebuild in
_to_building_part. The flat room_in_roof_* columns on epc_building_part are
superseded and dropped in a follow-up (#1664). Nullable is preserved on
insulation_thickness_mm / u_value (null vs 0 is the Table 17-vs-18 branch); area
is unconstrained (a §3.9.2 absent-gable adjustment is signed). Drop the 8
_UNPERSISTED_ALLOWLIST entries. Verified: worst RIR cert 10012119141 round-trip
-15.21 -> 0.0000 with detailed_surfaces at deep equality.
Companion FE migration: assessment-model#443 (epc_room_in_roof +
epc_room_in_roof_surface). Deploy gate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
EpcPropertyData.sap_roof_windows (List[SapRoofWindow]) had no table, so 55
corpus certs lost their rooflights on the DB round-trip and were re-scored
without them (§3 (27a) heat transmission + §6 solar gain). Add EpcRoofWindowModel
(mirrors EpcPhotovoltaicArrayModel) with a roof_window_index order column and the
resolved SapRoofWindow fields; wire save (batch insert), delete (both delete
paths), both read paths (get + get_many) and _compose reconstruction; drop the
two _UNPERSISTED_ALLOWLIST entries.
u_value_raw / g_perpendicular / frame_factor are stored resolved, not re-derived:
the Table 24 lookup is keyed on glazing_type AND glazing_gap, and glazing_gap is
not retained on SapRoofWindow, so they can't be reconstructed from the persisted
fields. window_location is JSONB (int on API / str on site-notes). Verified: a
roof-window cert round-trips sap_roof_windows to deep equality.
Companion FE migration: assessment-model#443 (epc_roof_window). Deploy gate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SapAlternativeWall.u_value (heat_transmission.py:1616), .wall_thickness_mm
(:1645) and .is_basement (:1618) were read by the calculator but had no column,
so they dropped on save. Latent today (0 gov-API certs lodge them — live only on
the Elmhurst/site-notes path), but is_basement is the nastiest shape: it does not
null out, it FLIPS meaning (True -> None -> is_basement_wall reads False),
silently switching off the RdSAP §5.17 / Table 23 basement-wall U path.
Add alt_wall_{1,2}_{u_value,thickness_mm,is_basement} to EpcBuildingPartModel,
write in from_domain, reconstruct in _to_alt_wall, and drop the three
_UNPERSISTED_ALLOWLIST entries so the structural guard enforces them. is_basement
is nullable (None "not stated" != False "not a basement"; a NOT NULL DEFAULT
false re-enables the code-6 heuristic — same reasoning as #1661's wall_is_basement).
Companion assessment-model migration: six nullable columns on epc_building_part
(deploy gate). Round-trip + field-coverage guard green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SapHeating.cylinder_heat_loss is read by the calculator (cert_to_inputs §4
cylinder loss) but had no column, so it was silently dropped on save: a dwelling
reloaded from Postgres lost its lodged cylinder loss and fell back to the Table
2b age-band default — worst case -15.53 SAP over the corpus round-trip (#1665).
Add heating_cylinder_heat_loss (nullable double precision) to EpcPropertyModel,
write it in from_domain and reconstruct it in the SapHeating compose path,
mirroring heating_cylinder_volume_measured_l. Remove the now-false
_UNPERSISTED_ALLOWLIST entry ("dormant — not read by the calculator") so the
ADR structural guard enforces it. Verified: worst cylinder cert 10091630692
round-trip -15.53 -> 0.0000.
Companion assessment-model migration (heating_cylinder_heat_loss column) must
land + be applied per-environment before the backend deploys (deploy gate).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The ADR-0030 component-accuracy and expired-pairs gates re-derive the *actual*
EpcPropertyData through the same mapper the leave-one-out scorer predicts against.
#1669 (measurement_type=2 external->internal conversion, RdSAP §3.4 + Table 2)
gives the fixture's external certs sharper internal dimensions, which tips the
geo-proximity-weighted donor mode on three components by single-target amounts:
total_window_area 3.7184 -> 3.7484, door_count 0.3333 -> 0.3737, has_pv floor
0.8929 -> 0.8571 (one pair 25/28 -> 24/28).
This is a ground-truth-method change, not a prediction-logic loosening: the fix
is spec-correct, corpus MAE improved (0.571 -> 0.570) and the real-cert accuracy
pin held. Per the gate convention the affected floors/ceilings are re-baselined
to the measured values with the attribution recorded; tighten-only resumes here.
Bisected: #1666/#1667 leave the gates green; #1669 introduces all three shifts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The previous commit staged these two files at `git mv` time but never
staged the edits that followed, so they landed with their pre-refactor
content: `class PashubService` and `from backend.pashub_fetcher...`
imports that no longer resolve. Local runs passed because the working
tree was correct; CI collected the committed tree and failed with
ModuleNotFoundError: No module named 'backend.pashub_fetcher'.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The service was the last one living wholly under backend/. It now follows
the same layering as abri and the other newer services:
domain/pashub_fetcher/ core file classification, subfolders
infrastructure/pashub_fetcher/ PasHub client, token getter, wire DTOs
orchestration/ PashubFetcherOrchestrator (was PashubService)
applications/pashub_fetcher/ lambda handler, trigger request, dev tooling
core_files.py is split along the layer boundary: the domain module keeps the
filename/evidence-category classification rules and no longer imports
infrastructure.postgres, while the CoreFiles -> FileTypeEnum translation moves
to infrastructure/pashub_fetcher/core_file_types.py.
Tests move into the tests/ tree by layer. Note this puts them in the only
suite CI currently runs (unit_tests.yml is disabled), so these 73 tests now
execute on PRs for the first time; they were previously reachable only via
the legacy pytest.ini testpaths.
sharepoint_renamer's image now copies just domain/pashub_fetcher/ rather than
the whole service, since SharepointSubfolders is all it needed.
Behaviour is unchanged. tests/ goes 9927 -> 10000 passed (+73, exactly the
tests that moved in); the legacy suite keeps its same 17 pre-existing failures
and 11 errors. tests/test_lambda_packaging.py confirms both changed
Dockerfiles still copy their handler's full import closure.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
