The deterministic calculator reads sap_ventilation.extract_fans_count (which
already round-trips); the top-level epc.extract_fans_count is its mirror (the
mapper sets both from one source). Reconstruct it from the same column so
EpcPropertyData round-trips complete, dropping the allow-list exception.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
community heating fuel + CHP fraction, alt-wall is_sheltered, wall
insulation thermal conductivity, pv_diverter_present, measured cylinder
volume, AP50 air permeability — all calculator-read, all silently dropped on
save. FE columns now live; assert deep-equal round-trip and drop their
coverage-guard allow-list entries so the guard enforces reconstruction.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The ADR-0036 guard only inspected EpcPropertyData's top-level fields, so a
dropped field on a NESTED object (the PV-array list, the floor heat-loss
flags) slipped straight through. Generalise it to walk every domain
dataclass reachable from EpcPropertyData and check each field is
reconstructed by a _compose/_to_* mapper or allow-listed (per-field or
whole-class), keyed by Class.field.
Surfaced 14 pre-existing nested gaps the old guard was blind to: 7 are
calculator-read with no FE column (scoring-relevant silent-drop, same class
as the PV bug — tracked follow-up), the rest dormant or awaiting FE tables.
Each is now explicit and justified.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
sap_energy_source.photovoltaic_arrays has no table, so every array is
dropped on save — worth ~12 SAP points on an electrically-heated dwelling
(persist != score). Inject two ordered arrays onto a PV-free fixture.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
is_exposed_floor / is_above_partially_heated_space have no
epc_floor_dimension column, so a True flag round-trips back to the False
default and silently flips the floor's heat-loss path (persist != score).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fail if any EpcPropertyData field is neither reconstructed by _compose nor on a
documented allow-list, turning latent persistence gaps into explicit decisions
(would have caught the conservatory and roof-window drops). ADR-0036.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Persist SapConservatory as five nullable conservatory_* columns on epc_property
(1:1 with the dwelling) and rebuild it in _compose, so the §6.1 fold survives
save -> reload -> score. Without this the scored (re-hydrated) EPC silently
dropped the conservatory (persist != score) — a latent gap shared with the
21.0.1 path. Adds a deep-equality round-trip test. ADR-0036.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a `source` discriminator (lodged | predicted) to the EPC store so a Property
holds a lodged EPC and a predicted one (EPC Prediction gap-fill) at once
(ADR-0031). EpcRepository.save gains source="lodged"; idempotent delete is now
per-source (a predicted save no longer wipes lodged, and vice versa);
get_for_property/get_for_properties filter lodged; new get_predicted_for_property
/ get_predicted_for_properties read predicted. PropertyPostgresRepository.get +
get_many hydrate Property.predicted_epc, so the predicted picture reaches the
modelling read (both load via get_many). FakeEpcRepo mirrors the dual slot.
EpcPropertyModel gains `source` (default "lodged"); the test DB builds from the
SQLModel mirror so this is exercised without the prod migration. The matching
Drizzle change (column + per-(property_id,source) uniqueness) is the team's to
action before merge — docs/MIGRATION_NOTE_predicted_epc_source.md.
3 store tests (coexist, idempotent predicted re-save leaves lodged, lodged-only
has no predicted) + property-repo wiring; 85 pass across affected suites; new
code pyright-clean (2 pre-existing wwhrs errors in epc_property_table untouched).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
PR feedback (dancafc): the SQLModel column was Optional[str], but the
domain `SapBuildingPart.wall_insulation_thickness` is Optional[Union[str,
int]] — `_api_resolve_wall_insulation_thickness` returns an int mm when the
API lodges `wall_insulation_thickness == "measured"` (SAP 10.2 §5.7 /
Table 8). The plain str column round-trips that int back as the string
"100", corrupting the Table 8 insulated-wall U-value lookup.
This column was missed in the round-trip-fidelity §1 JSONB sweep
(#1129) — its `Union[str, int]` sibling `roof_insulation_thickness` was
converted, but `wall_insulation_thickness` was not, and no 21.0.0/21.0.1
fixture lodges "measured" so the gap stayed latent. Convert to JSONB
(matching `roof_insulation_thickness` / `flat_roof_insulation_thickness`),
align the column type to Optional[Union[str, int]] (also removes a pyright
type-mismatch), record it in the migration doc §1, and add a round-trip
guard test asserting an int survives as an int (fails as '100' == 100 on
the old str column).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Final slice of ADR-0012: collapse the per-property read round-trips a batch
made (Baseline hydrated ~8 queries x 30 properties one at a time) into a
handful of per-table IN queries.
- EpcPostgresRepository: extracted a shared `_compose(rows)` from `get` (the
windows + floor-dim fetches are now passed in, not fetched inline), so both
`get` and the new `get_for_properties(property_ids)` build EpcPropertyData
from pre-fetched rows. `get_for_properties` fetches each child table once
(`WHERE epc_property_id IN ...`), groups in memory, and composes — load-whole
per ADR-0002.
- PropertyRepository.get_many(property_ids) -> Properties: one query for the
property rows + one bulk EPC hydration, composed in input order.
- BaselineOrchestrator / IngestionOrchestrator read the batch via get_many
instead of N x get.
- Ports + fakes gain the bulk methods.
The #1129 round-trip fidelity test stays green (the compose extraction is
behaviour-preserving). New tests: bulk hydration correctness + round-trips are
constant w.r.t. batch size (one-per-table, proven by query count). 123 pass;
pyright strict clean; AAA.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Re-runs of a First Run batch re-save a property's data; that must replace,
not duplicate (ADR-0012 idempotent batch writes).
- `EpcPostgresRepository.save` deletes the property's existing EPC graph
(parent + all child tables, floor-dims via their building parts) before
inserting, when a `property_id` is given. Anonymous saves still insert.
- `BaselinePostgresRepository.save` deletes the existing row for the
`property_id` before inserting — no more unique-constraint violation on
re-save; also what the re-score-on-override path needs.
- Solar already upserts, so it's unchanged.
The #1129 round-trip fidelity test stays green (delete-first is a no-op on
a first save). 2 new tests (re-save replaces, not duplicates). pyright
strict clean; AAA.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add epc_renewable_heat_incentive table (space_heating_kwh, water_heating_kwh +
the three insulation-impact kWh fields), wired into EpcPostgresRepository
save/get. This is the P0 gap: RenewableHeatIncentive carries the baseline
space-heating/hot-water kWh that EPC Energy Derivation consumes.
The round-trip test now asserts full deep-equality (dropped the
renewable_heat_incentive exclusion) and passes for RdSAP 21.0.0 + 21.0.1.
DB migration for the new table documented in
docs/migrations/epc-property-round-trip-fidelity.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Relocate EpcPropertyModel + child tables from the dying backend/ tree to
infrastructure/postgres/epc_property_table.py (re-export shim keeps
documents_parser working). Add EpcRepository port + EpcPostgresRepository with
a full reverse mapper (epc_property tables -> EpcPropertyData).
Round-trip test surfaced two fidelity gaps:
1. Union[int,str] SAP code fields were str()-coerced on save, losing the int
(API) vs str (Site Notes) distinction. Now stored as JSONB (type-preserving).
2. The schema was a partial projection. Closed the cheap gaps on the model
(heating shower/bath counts, roof_construction_type, curtain_wall_age,
addendum, mechanical_vent_duct_insulation_level, SAP 10.2 §2 ventilation
fields + a ventilation_present flag). Structural gaps tracked as follow-ups;
renewable_heat_incentive (P0, #1137) excluded from the assertion until landed.
Round-trip passes for RdSAP-Schema-21.0.0 and 21.0.1; pyright strict clean.
Migration inventory for the DB: docs/migrations/epc-property-round-trip-fidelity.md
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
