Model/docs/adr/0017-plan-persistence-evolve-live-tables.md

Plan persistence — evolve the live tables, no Plan Phase

Status: Accepted. Decided in a /grill-with-docs session (2026-06-03) scoping the #1157 Plan-persistence schema. Builds on ADR-0011 / ADR-0012 (stage orchestrators, one Unit of Work per batch), ADR-0016 (the three scoring roles), and ADR-0005 (multi-phase deferred).

Context

The Modelling stage must persist a Plan per Property per Scenario. Unlike the rest of the rebuild, the output tables already exist in the live product: plan, recommendation, plan_recommendations (an m2m join), and scenario — SQLAlchemy Base models in backend/app/db/models/recommendations.py, which the live FE reads. This is schema evolution on a running product, not greenfield. Wholesale table changes are expensive and risky.

The rebuild's persistence convention is SQLModel table=True rows in infrastructure/postgres/, written through repos bound to a UnitOfWork, with the ephemeral-Postgres tests building the schema via SQLModel.metadata.create_all. The established way it already touches live tables is a SQLModel mirror pointing at the same physical table (task_table.py → tasks, product_table.py → material, property_table.py → property); the legacy Base model stays for the live app and the physical table is the shared contract.

Decision

• Reuse the live plan and recommendation tables via SQLModel mirrors in infrastructure/postgres/, written through a new PlanRepository on the Unit of Work. No new parallel tables. The legacy SQLAlchemy models remain for the live app's reads.
• Add recommendation.plan_id (FK → plan.id, ON DELETE CASCADE). New writes link each measure to its Plan directly; the plan_recommendations m2m is retired for new writes (its many-to-many made deletes pathologically slow). The m2m table is left in place until the last legacy reader is cut over.
• A persisted recommendation row is a Plan Measure — the one selected Measure Option with its role-3 (final-package cascade) attributed impact and its Cost. A Recommendation (the candidate, multi-Option, no stored impact) is never persisted as output. (See CONTEXT.md: Plan Measure vs Recommendation.)
• Post-retrofit figures stay flat on plan (the legacy columns). No plan_phase table and no phase column — multi-phase is deferred (ADR-0005).
• Idempotent replace per (property_id, scenario_id) (ADR-0012): a re-run deletes the matching plan rows — cascading to their recommendation rows via plan_id — then inserts fresh. One batch commit, never per-property.
• plan.is_default derives from scenario.is_default so exactly one default plan exists per Property even across many Scenarios. recommendation.default = True for every persisted Plan Measure (only selected measures are persisted today).
• Units match the live column contract: the calculator emits CO₂ in kg; the live co2_equivalent_savings / post_co2_emissions columns are tonnes, so divide by 1000 on the way in. The CO₂ baseline for the saving comes from the same calculator (PackageScorer.score(epc, [])), keeping baseline and post self-consistent.

Considered and rejected

• Greenfield clean tables for Plans — rejected: the live FE already reads plan/recommendation, and there is live data. A parallel table would fork the read model.
• Keep the plan_recommendations m2m — rejected: the join's cascade delete is the known performance killer this change exists to remove.
• JSONB blob for the package — rejected: the FE queries per-measure columns; flat typed columns are the existing contract.

Amendment (2026-06-03) — retire plan_recommendations, consolidate the models

The original decision retired the m2m for new writes but left it in place. This amendment drops it and consolidates the model definitions, decided in a /grill-with-docs session:

• plan_recommendations is dropped. All readers (portfolio_functions, Outputs, export/property_scenarios) and writers are cut onto recommendation.plan_id. The m2m is one-to-many in practice (a measure is never shared across Plans), so a single FK models it faithfully. The cross-repo expand/contract sequence (add → backfill → dual-write → cut reads → drop) is specified in docs/migrations/recommendation-plan-id.md; the two load-bearing rules are backfill before any read cuts over and dual-write the m2m until all reads are off it (the FE reads via Drizzle directly, so the repos cannot deploy atomically).
• One model per physical table, in infrastructure/postgres/modelling/. The drift hazard the original ADR accepted (two ORM definitions of plan/recommendation) is resolved by consolidating the whole backend/app/db/models/recommendations.py cluster (plan, recommendation, recommendation_materials, scenario, installed_measure, the PlanType/MeasureType enums) into single SQLModel …Row definitions in a new infrastructure/postgres/modelling/ subpackage, carrying full legacy column parity plus recommendation.plan_id. backend/app/db/models/recommendations.py becomes a re-export shim (the established epc_property.py pattern), aliasing the legacy names to the canonical …Row classes so backend/ callers keep working. The rebuild's partial PlanRow/RecommendationRow/ScenarioRow mirrors are absorbed into these.
• Scope: backend/ + the rebuild only. The etl/ and sfr/ reporting scripts that read the m2m are deferred to a later pass.

Consequences

• Two ORM definitions of plan/recommendation coexist (legacy SQLAlchemy + new SQLModel mirror), a drift hazard — mitigated by this being the established mirror pattern and the physical table being the single contract. Retiring the legacy models is later, separate work.
• The FE owns the Drizzle migration adding recommendation.plan_id (+ index) and, eventually, dropping plan_recommendations. Documented in docs/migrations/recommendation-plan-id.md.
• Unselected alternatives (the "swap-in" UX) will later be persisted as recommendation rows with default = False linked via plan_id — this schema is forward-compatible. The open question is what impact figure such a row carries: it cannot hold a role-3 attribution (it is not in the package), and ADR-0016 forbids surfacing the role-1 independent signal as truth. Deferred as an ADR-0016 question.
• Energy / bill columns (plan.post_energy_consumption, plan.energy_consumption_savings, plan.post_energy_bill, plan.energy_bill_savings, recommendation.kwh_savings, recommendation.energy_cost_savings) are delivered/billed kWh, not the calculator's primary energy. They are populated by a later Bill Derivation slice that re-runs bills on the post-package EPC; NULL until then.
• The #1157 tracer persists only SAP (post_sap_points, recommendation.sap_points), CO₂ in tonnes (post_co2_emissions, co2_savings, recommendation.co2_equivalent_savings), cost (estimated_cost, cost_of_works, contingency_cost), and the derived post_epc_rating. Valuation, plan_type, U-values, heat demand, labour, and the energy/bill cluster are left NULL for later slices.