Model/docs/adr/0017-plan-persistence-evolve-live-tables.md

# Plan persistence — evolve the live tables, no Plan Phase

**Status: Accepted.** Decided in a `/grill-with-docs` session (2026-06-03) scoping the #1157 Plan-persistence schema. Builds on [ADR-0011](0011-composable-stage-orchestrators.md) / [ADR-0012](0012-unit-of-work-per-stage-batch-transaction.md) (stage orchestrators, one Unit of Work per batch), [ADR-0016](0016-package-rescore-over-warm-start-optimisation.md) (the three scoring roles), and [ADR-0005](0005-multi-phase-scenarios-per-phase-recompute.md) (multi-phase deferred).

## Context

The Modelling stage must persist a **Plan** per Property per **Scenario**. Unlike the rest of the rebuild, the output tables already exist in the **live product**: `plan`, `recommendation`, `plan_recommendations` (an m2m join), and `scenario` — SQLAlchemy `Base` models in `backend/app/db/models/recommendations.py`, which the live FE reads. This is **schema evolution on a running product**, not greenfield. Wholesale table changes are expensive and risky.

The rebuild's persistence convention is SQLModel `table=True` rows in `infrastructure/postgres/`, written through repos bound to a `UnitOfWork`, with the ephemeral-Postgres tests building the schema via `SQLModel.metadata.create_all`. The established way it already touches live tables is a **SQLModel mirror pointing at the same physical table** (`task_table.py` → `tasks`, `product_table.py` → `material`, `property_table.py` → `property`); the legacy `Base` model stays for the live app and the physical table is the shared contract.

## Decision

- **Reuse the live `plan` and `recommendation` tables** via SQLModel mirrors in `infrastructure/postgres/`, written through a new `PlanRepository` on the Unit of Work. No new parallel tables. The legacy SQLAlchemy models remain for the live app's reads.
- **Add `recommendation.plan_id`** (FK → `plan.id`, `ON DELETE CASCADE`). New writes link each measure to its Plan directly; the **`plan_recommendations` m2m is retired for new writes** (its many-to-many made deletes pathologically slow). The m2m table is left in place until the last legacy reader is cut over.
- **A persisted `recommendation` row is a Plan Measure** — the one selected **Measure Option** with its **role-3 (final-package cascade) attributed impact** and its **Cost**. A **Recommendation** (the candidate, multi-Option, no stored impact) is never persisted as output. (See `CONTEXT.md`: Plan Measure vs Recommendation.)
- **Post-retrofit figures stay flat on `plan`** (the legacy columns). **No `plan_phase` table and no `phase` column** — multi-phase is deferred (ADR-0005).
- **Idempotent replace per `(property_id, scenario_id)`** (ADR-0012): a re-run deletes the matching `plan` rows — cascading to their `recommendation` rows via `plan_id` — then inserts fresh. One batch commit, never per-property.
- **`plan.is_default` derives from `scenario.is_default`** so exactly one default plan exists per Property even across many Scenarios. **`recommendation.default = True`** for every persisted Plan Measure (only selected measures are persisted today).
- **Units match the live column contract:** the calculator emits CO₂ in **kg**; the live `co2_equivalent_savings` / `post_co2_emissions` columns are **tonnes**, so divide by 1000 on the way in. The CO₂ baseline for the saving comes from the **same calculator** (`PackageScorer.score(epc, [])`), keeping baseline and post self-consistent.

## Considered and rejected

- **Greenfield clean tables for Plans** — rejected: the live FE already reads `plan`/`recommendation`, and there is live data. A parallel table would fork the read model.
- **Keep the `plan_recommendations` m2m** — rejected: the join's cascade delete is the known performance killer this change exists to remove.
- **JSONB blob for the package** — rejected: the FE queries per-measure columns; flat typed columns are the existing contract.

## Amendment (2026-06-03) — retire `plan_recommendations`, consolidate the models

The original decision *retired the m2m for new writes* but left it in place. This amendment **drops it** and **consolidates the model definitions**, decided in a `/grill-with-docs` session:

- **`plan_recommendations` is dropped.** All readers (`portfolio_functions`, `Outputs`, `export/property_scenarios`) and writers are cut onto `recommendation.plan_id`. The m2m is one-to-many in practice (a measure is never shared across Plans), so a single FK models it faithfully. The cross-repo expand/contract sequence (add → backfill → dual-write → cut reads → drop) is specified in [docs/migrations/recommendation-plan-id.md](../migrations/recommendation-plan-id.md); the two load-bearing rules are **backfill before any read cuts over** and **dual-write the m2m until all reads are off it** (the FE reads via Drizzle directly, so the repos cannot deploy atomically).
- **One model per physical table, in `infrastructure/postgres/modelling/`.** The drift hazard the original ADR accepted (two ORM definitions of `plan`/`recommendation`) is resolved by **consolidating the whole `backend/app/db/models/recommendations.py` cluster** (`plan`, `recommendation`, `recommendation_materials`, `scenario`, `installed_measure`, the `PlanType`/`MeasureType` enums) into single **SQLModel** `…Row` definitions in a new `infrastructure/postgres/modelling/` subpackage, carrying full legacy column parity plus `recommendation.plan_id`. `backend/app/db/models/recommendations.py` becomes a **re-export shim** (the established `epc_property.py` pattern), aliasing the legacy names to the canonical `…Row` classes so `backend/` callers keep working. The rebuild's partial `PlanRow`/`RecommendationRow`/`ScenarioRow` mirrors are absorbed into these.
- **Scope:** `backend/` + the rebuild only. The `etl/` and `sfr/` reporting scripts that read the m2m are deferred to a later pass.

## Consequences

- **Two ORM definitions of `plan`/`recommendation`** coexist (legacy SQLAlchemy + new SQLModel mirror), a drift hazard — mitigated by this being the established mirror pattern and the physical table being the single contract. Retiring the legacy models is later, separate work.
- The **FE owns the Drizzle migration** adding `recommendation.plan_id` (+ index) and, eventually, dropping `plan_recommendations`. Documented in `docs/migrations/recommendation-plan-id.md`.
- **Unselected alternatives** (the "swap-in" UX) will later be persisted as `recommendation` rows with `default = False` linked via `plan_id` — this schema is forward-compatible. The open question is *what impact figure* such a row carries: it cannot hold a role-3 attribution (it is not in the package), and ADR-0016 forbids surfacing the role-1 independent signal as truth. **Deferred** as an ADR-0016 question.
- **Energy / bill columns** (`plan.post_energy_consumption`, `plan.energy_consumption_savings`, `plan.post_energy_bill`, `plan.energy_bill_savings`, `recommendation.kwh_savings`, `recommendation.energy_cost_savings`) are **delivered/billed kWh**, not the calculator's primary energy. They are populated by a later **Bill Derivation slice that re-runs bills on the post-package EPC**; NULL until then.
- The **#1157 tracer persists only** SAP (`post_sap_points`, `recommendation.sap_points`), CO₂ in tonnes (`post_co2_emissions`, `co2_savings`, `recommendation.co2_equivalent_savings`), cost (`estimated_cost`, `cost_of_works`, `contingency_cost`), and the derived `post_epc_rating`. Valuation, `plan_type`, U-values, heat demand, labour, and the energy/bill cluster are left NULL for later slices.