Model/docs/adr/0040-nested-field-epc-persistence-fidelity.md

EPC persistence must round-trip nested scoring fields, guarded recursively

Context

The modelling pipeline scores the DB-stored EpcPropertyData (ADR-0003: the DB is the read model). ADR-0036 added a round-trip-fidelity guarantee — an EpcPropertyData saved to the epc_property tables, reloaded and mapped back must reconstruct the original — backed by a deep-equality round-trip test and a structural coverage guard.

Both had a blind spot: they only covered top-level EpcPropertyData fields. The round-trip test only catches a dropped field when a fixture populates it, and the chosen fixtures (RdSAP-21.0.0/21.0.1) happened not to exercise certain fields; the structural guard inspected only dataclasses.fields(EpcPropertyData), never recursing into nested objects. So fields on nested sub-objects were dropped silently, corrupting the scored baseline:

• sap_energy_source.photovoltaic_arrays — no table existed; a dwelling's solar PV was dropped on save, −12 SAP points on a real ASHP+solar property (725634: calc(live) 74.6 → calc(reloaded) 62.1).
• SapFloorDimension.is_exposed_floor / is_above_partially_heated_space — no columns; a True flag reloaded as the False default, flipping the floor's heat-loss path (+8 SAP points on 721534).
• Recursing the guard then surfaced seven more calculator-read nested fields with no column (community-heating fuel + CHP fraction, alt-wall is_sheltered, wall insulation thermal conductivity, pv_diverter_present, measured cylinder volume, AP50 air permeability) — the same silent-drop class.

A lossy read model is corrosive: the handler models off the faithful in-memory live cert (so the plan is right) but the baseline re-reads the lossy DB projection, so the two disagree for reasons unrelated to any override — compounding the ADR-0039 baseline confusion.

Decision

Make the epc_property projection faithful for every scoring field, and make the guard able to see nested gaps:

1. New storage (FE/Drizzle migrations, applied additively): epc_photovoltaic_array child table (one row per array, ordered by array_index); epc_floor_dimension.{is_exposed_floor, is_above_partially_heated_space}; and columns for the seven calculator-read nested fields. Python SQLModel + save/_compose mappers wire each, with deep-equality round-trip tests.
2. Recursive structural guard: the ADR-0036 coverage guard now walks every domain dataclass reachable from EpcPropertyData (through Optionals/lists) and asserts each field is either reconstructed by a _compose/_to_* mapper or on an allow-list keyed by Class.field. Adding a field at any depth now forces a conscious persist-or-justify decision.

Fields the calculator does not read stay allow-listed as dormant (e.g. SapBuildingPart.{floor_u_value, roof_u_value, wall_u_value}); a few structural gaps (SapRoofWindow, room-in-roof detail, alt-wall u_value/wall_thickness_mm) remain FE-column-pending and are explicitly allow-listed, not silent. The top-level extract_fans_count (a mirror of the persisted sap_ventilation copy the calculator actually reads) is reconstructed from that column so the object round-trips complete with no exception.

Consequences

• The DB EPC round-trips exactly for every scoring field — verified against the live schema: 725634 and 721534 now reload with SAP delta +0.00 (was −12.5 / +8.2). Residual non-equality is limited to allow-listed dormant fields and a sub-millimetre real-column float artifact, none scoring-relevant.
• This is a prerequisite for ADR-0039: storing calc(EPC + overrides) as the Effective baseline is only correct if the stored EPC is faithful. Until a property is re-modelled (handler re-fetches + re-saves), its existing DB row stays lossy, so the live baseline finalises only after deploy + re-model.
• A class of silent persistence drops can no longer hide: the recursive guard fails CI on any unjustified nested gap.
• New EPC scoring fields now carry a persistence cost (column + mapper + round-trip assertion) by construction — the intended friction.