SapBuildingPart.wall_u_value (the gov-EPC lodged wall U, authoritative over the
derived default) had no column and dropped on save, so the reloaded dwelling fell
back to the age-band default U — -2.32 SAP on the 2 affected RdSAP corpus certs.
Surfaced by the new DB-round-trip gate. The prior _UNPERSISTED_ALLOWLIST
justification ("deliberate — full-SAP only; RdSAP re-model uses default U-values")
was stale: the cascade DOES honour the lodged U on the API path.
Add wall_u_value to EpcBuildingPartModel, write in from_domain, reconstruct in
_to_building_part, drop the allowlist entry. Verified: certs 39036600 / 38137278
round-trip 72.43 / 69.11 exactly. Needs an added FE column
(epc_building_part.wall_u_value, double precision NULL) on assessment-model#443.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SapRoomInRoof.detailed_surfaces (+ the gable/common-wall scalars) had no columns,
so 107 corpus certs lost their whole §3.9/§3.10 room-in-roof geometry on the DB
round-trip and fell back to the Simplified all-elements age-band default —
worst case -15.21 SAP (#1665 / #1664).
Add EpcRoomInRoofModel (0..1 per building part, unique FK) + EpcRoomInRoofSurface
Model (0..n, surface_index order). Save nests via RETURNING (building part -> RIR
-> surfaces); delete clears bottom-up (surfaces -> room_in_roof -> parts) since
the FE FKs are ON DELETE no action; both read paths group + reconstruct in a new
_to_room_in_roof / _to_rir_surface, replacing the two-field flat rebuild in
_to_building_part. The flat room_in_roof_* columns on epc_building_part are
superseded and dropped in a follow-up (#1664). Nullable is preserved on
insulation_thickness_mm / u_value (null vs 0 is the Table 17-vs-18 branch); area
is unconstrained (a §3.9.2 absent-gable adjustment is signed). Drop the 8
_UNPERSISTED_ALLOWLIST entries. Verified: worst RIR cert 10012119141 round-trip
-15.21 -> 0.0000 with detailed_surfaces at deep equality.
Companion FE migration: assessment-model#443 (epc_room_in_roof +
epc_room_in_roof_surface). Deploy gate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
EpcPropertyData.sap_roof_windows (List[SapRoofWindow]) had no table, so 55
corpus certs lost their rooflights on the DB round-trip and were re-scored
without them (§3 (27a) heat transmission + §6 solar gain). Add EpcRoofWindowModel
(mirrors EpcPhotovoltaicArrayModel) with a roof_window_index order column and the
resolved SapRoofWindow fields; wire save (batch insert), delete (both delete
paths), both read paths (get + get_many) and _compose reconstruction; drop the
two _UNPERSISTED_ALLOWLIST entries.
u_value_raw / g_perpendicular / frame_factor are stored resolved, not re-derived:
the Table 24 lookup is keyed on glazing_type AND glazing_gap, and glazing_gap is
not retained on SapRoofWindow, so they can't be reconstructed from the persisted
fields. window_location is JSONB (int on API / str on site-notes). Verified: a
roof-window cert round-trips sap_roof_windows to deep equality.
Companion FE migration: assessment-model#443 (epc_roof_window). Deploy gate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SapAlternativeWall.u_value (heat_transmission.py:1616), .wall_thickness_mm
(:1645) and .is_basement (:1618) were read by the calculator but had no column,
so they dropped on save. Latent today (0 gov-API certs lodge them — live only on
the Elmhurst/site-notes path), but is_basement is the nastiest shape: it does not
null out, it FLIPS meaning (True -> None -> is_basement_wall reads False),
silently switching off the RdSAP §5.17 / Table 23 basement-wall U path.
Add alt_wall_{1,2}_{u_value,thickness_mm,is_basement} to EpcBuildingPartModel,
write in from_domain, reconstruct in _to_alt_wall, and drop the three
_UNPERSISTED_ALLOWLIST entries so the structural guard enforces them. is_basement
is nullable (None "not stated" != False "not a basement"; a NOT NULL DEFAULT
false re-enables the code-6 heuristic — same reasoning as #1661's wall_is_basement).
Companion assessment-model migration: six nullable columns on epc_building_part
(deploy gate). Round-trip + field-coverage guard green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
EpcPropertyModel.uprn was `Optional[int]`, which SQLModel maps to INTEGER, so a
real 11/12-digit UPRN (e.g. 10000452538) overflowed int32 and hard-failed the
insert with NumericValueOutOfRange — a failure on EVERY real-cert save, masked
because the round-trip fixtures use small fake UPRNs. The FE `property.uprn` is
already bigint; pin the column to BigInteger so ours matches (backend-only, no
migration). Verified: uprn 10000452538 saves + reloads.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SapHeating.cylinder_heat_loss is read by the calculator (cert_to_inputs §4
cylinder loss) but had no column, so it was silently dropped on save: a dwelling
reloaded from Postgres lost its lodged cylinder loss and fell back to the Table
2b age-band default — worst case -15.53 SAP over the corpus round-trip (#1665).
Add heating_cylinder_heat_loss (nullable double precision) to EpcPropertyModel,
write it in from_domain and reconstruct it in the SapHeating compose path,
mirroring heating_cylinder_volume_measured_l. Remove the now-false
_UNPERSISTED_ALLOWLIST entry ("dormant — not read by the calculator") so the
ADR structural guard enforces it. Verified: worst cylinder cert 10091630692
round-trip -15.53 -> 0.0000.
Companion assessment-model migration (heating_cylinder_heat_loss column) must
land + be applied per-environment before the backend deploys (deploy gate).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
RdSAP 10 §5.17 / Table 23. `wall_is_basement` is not a lodged U-value — it is a
disambiguation flag selecting WHICH RdSAP default applies, so unlike
wall/roof/floor_u_value it must survive the round-trip.
Code 6 is canonically SYSTEM-BUILT and the gov-API path infers "basement" from a
code-6 heuristic; the Elmhurst site-notes mapper sets wall_is_basement=False
precisely to defeat it. Dropping the field inverted the answer — a system-built
wall came back as a basement wall, dragging the whole ground floor onto the
Table 23 basement-floor U-value via `has_basement`.
Declared on no RdSAP schema, so it never arrives via the gov API; the earlier
corpus scan showing 0 occurrences measured the wrong population. The Elmhurst
site-notes path sets it and those EPCs are persisted
(scripts/wchg_elmhurst_ingest.py).
Nullable is load-bearing: None = "not stated, use the heuristic", False =
"explicitly system-built, do not".
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add `rafter_insulation_thickness` to `EpcBuildingPartModel` (JSONB, matching the
sibling thickness columns so Union[str, int] survives) and wire it through
`from_domain` / `_to_building_part`.
Property 749719 lodges "250mm" -> Table 16 col (2) U 0.23. Dropped, the roof
billed at U 2.30 — a 10x error worth -9.8 SAP (lodged 73, calculated 63.2) that
moved the dwelling a whole band, C -> D, and drew a 9.63-point loft-insulation
recommendation on a roof rated 4/5 by the assessor.
The sibling wall/roof/floor_u_value and wall_is_basement fields have the same
drop but are deliberately NOT persisted: they are full-SAP artefacts and we
re-model as RdSAP, which uses default U-values. Their allow-list entries are
re-justified rather than removed — the previous "not read by the calculator" was
stale and is what hid this drop.
Requires the Drizzle migration in Hestia-Homes/assessment-model to land FIRST;
without the column, EPC reads fail outright.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per PR review: the FE writes the selection config
({portfolio_id, property_ids?, select_all?}) into the FE-owned task.inputs and
passes only task_id, so a large hand-picked selection never travels in an HTTP
body. The route reads task.inputs, resolves to landlord_property_ids, caps, and
pins the recipe onto sub_task.inputs as before. Declares the FE-owned inputs
column on the TaskRow mirror so the backend can read it and the test schema
builds it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ADR-0056 amended: override and EPC own type/form facts; a property with
neither is Unknown whatever the legacy columns say. Mirror columns removed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The four lodged_* columns are nullable as a unit; from_domain writes them all
NULL when a predicted Property has no Lodged Performance, and to_domain
reconstructs lodged=None (lodged_sap_score is the read discriminator, mirroring
the bill block). The production lodged_* columns are FE-owned (Drizzle) and need
a companion ALTER ... DROP NOT NULL migration before the backend writes NULL.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire community heating fuel + CHP fraction (epc_main_heating_detail),
alt-wall is_sheltered + wall insulation thermal conductivity
(epc_building_part), and pv_diverter_present / measured cylinder volume /
AP50 air permeability (epc_property) through save + _compose/_to_*. All
deep-equal round-trip; coverage guard now enforces their reconstruction.
Columns live (FE migration applied).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add EpcPhotovoltaicArrayModel (epc_photovoltaic_array child table) and wire
save / delete / read so sap_energy_source.photovoltaic_arrays survives
load->save->load in order. Threaded through both the single get() and the
bulk _for_properties() paths via _compose -> _to_energy_source. Column
names match the FE migration (feature/epc-pv-and-floor-heatloss-schema).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add is_exposed_floor / is_above_partially_heated_space to
EpcFloorDimensionModel and wire from_domain + _to_floor_dimension. Column
names match the FE schema (feature/epc-pv-and-floor-heatloss-schema).
Live DB migration is run post-merge (drizzle-kit generate picks them up).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Persist SapConservatory as five nullable conservatory_* columns on epc_property
(1:1 with the dwelling) and rebuild it in _compose, so the §6.1 fold survives
save -> reload -> score. Without this the scored (re-hydrated) EPC silently
dropped the conservatory (persist != score) — a latent gap shared with the
21.0.1 path. Adds a deep-equality round-trip test. ADR-0036.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Caught live writing property_overrides on portfolio 796: the Python
override_component SAEnum lagged the DB enum, so reading a new-component row
back threw LookupError. Guard it with a consistency test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The SQLModel had drifted to a `bill_` prefix on the Bill Derivation block, but
the FE-owned Drizzle table uses unprefixed names (`heating_kwh`, `hot_water_kwh`
… `total_annual_bill_gbp`) plus a nullable `fuel_rates_period`. INSERTs failed
with UndefinedColumn. Rename the columns to mirror the live table column-for-
column (the prefix's anti-clash purpose is moot: `heating_kwh` != the recorded
`space_heating_kwh`), and add the `fuel_rates_period` column — left None until
Bill Derivation threads the snapshot period through.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The new pipeline left no per-Property record of a run (the old engine set
property.has_recommendations and populated property_details_epc). Restore the
marker: PropertyRepository.mark_modelled sets has_recommendations (true when the
Plan carries measures, mirroring the old engine) and bumps updated_at, so a
first-run under the new process is identifiable as updated_at >= 2026-06-01.
ModellingOrchestrator marks each Property after its Scenarios (true if any
Scenario yielded a measure); run_modelling_e2e's --persist path marks it too
(its compute runs on in-memory fakes, so the DB UoW sets it directly). Adds the
has_recommendations/updated_at columns to the PropertyRow mirror.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The measures a run considers should come from the Scenario, not a CLI flag.
The live scenario table persists exclusions only (no inclusions column), as a
Postgres text-array of exact MeasureType values.
- Scenario gains `exclusions: frozenset[MeasureType]` + `considered_measures()`
(all measures minus the excluded ones, or None when none are excluded).
- ScenarioModel.to_domain parses the `{a,b,c}` exclusions array into
MeasureTypes, raising on a token that is not an exact MeasureType value
(no high-level category expansion), per the strict-enum convention.
- ModellingOrchestrator._plan_for derives the allowlist from the Scenario's
exclusions, combined (intersection) with any explicit considered_measures
override via the new `combine_considered_measures`.
- run_modelling_e2e sources the allowlist from the Scenario; --measures /
--exclude-measures become optional overlays (e.g. the technical
secondary_heating_removal exclusion the catalogue cannot yet stock).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a `source` discriminator (lodged | predicted) to the EPC store so a Property
holds a lodged EPC and a predicted one (EPC Prediction gap-fill) at once
(ADR-0031). EpcRepository.save gains source="lodged"; idempotent delete is now
per-source (a predicted save no longer wipes lodged, and vice versa);
get_for_property/get_for_properties filter lodged; new get_predicted_for_property
/ get_predicted_for_properties read predicted. PropertyPostgresRepository.get +
get_many hydrate Property.predicted_epc, so the predicted picture reaches the
modelling read (both load via get_many). FakeEpcRepo mirrors the dual slot.
EpcPropertyModel gains `source` (default "lodged"); the test DB builds from the
SQLModel mirror so this is exercised without the prod migration. The matching
Drizzle change (column + per-(property_id,source) uniqueness) is the team's to
action before merge — docs/MIGRATION_NOTE_predicted_epc_source.md.
3 store tests (coexist, idempotent predicted re-save leaves lodged, lodged-only
has no predicted) + property-repo wiring; 85 pass across affected suites; new
code pyright-clean (2 pre-existing wwhrs errors in epc_property_table untouched).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pulls in 42 commits of calculator/mapper accuracy fixes from the per-cert
mapper-validation and floor/roof/heating fronts.
Conflict resolutions:
- mapper `_is_elmhurst_roof_window`: main dropped the branch's "wall location →
vertical" guard (it broke cert 000516's rooflight), but that re-broke cert
001431's two External-wall U>3.0 windows (which must stay vertical). The two
certs lodge a BYTE-IDENTICAL §11 row, so neither location nor U separates
them — the real discriminator is the room-in-roof context. Replaced the
unconditional U>3.0 backstop with one gated on the BP having a room-in-roof
(`_elmhurst_bp_has_room_in_roof`): 000516's Main BP has a "Room in roof type
1" (→ rooflight), 001431's does not (→ vertical). Validated against BOTH —
full Elmhurst worksheet suite 1038 pass + the 001431 window-extraction pin.
- property_postgres_repository: kept main's `ids_by_uprn` method + the branch's
`_restrictions_of` helper.
- sap_fuel.py: the branch relocated it to domain/billing/ (already carrying
main's to_table_32_code normalization), so kept the old path deleted.
Fallout from main's fabric fixes (validated by the boiler-3 real-cert pin which
still reproduces at delta 0):
- re-pinned the boiler-1 + boiler-instant-hw ASHP snapshot scores;
- main's §14.2 gas-boiler main-fuel derivation resolved the BGB/102 baseline
gap, so `test_gas_boiler_instant_hw_before_baselines` is now a passing test
(was an xfail tripwire).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
