mirror of
https://github.com/Hestia-Homes/Model.git
synced 2026-07-19 17:03:02 +00:00
Amends ADR-0004's both-halves-always-populated invariant: EPC Prediction borrows a neighbour's recorded figures, so a predicted Property has no Lodged Performance — its lodged half is absent (NULL), only the Effective half is persisted. Sharpens the CONTEXT.md glossary for Lodged / Baseline Performance. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
96 lines
8.3 KiB
Markdown
96 lines
8.3 KiB
Markdown
# `PropertyBaselinePerformance` stores both lodged and effective values
|
||
|
||
A Property's current performance has two states we care about: the rating that was lodged on the government register (the "lodged" SAP / band / carbon / heat) and the rating produced by the modelling pipeline against the current Effective EPC (the "effective" values, which may have been rebaselined by ML when the EPC was pre-SAP10 or when Landlord Overrides / Site Notes changed physical state). We considered storing a single set of values — the rebaselined-if-needed-otherwise-lodged figures — and rejected that. Both are stored as a pair on every `PropertyBaselinePerformance`, equal when no rebaselining trigger fires.
|
||
|
||
The pair lets the FE show "this is what the gov register says vs this is the SAP10-equivalent we modelled against" side by side without a second query, and keeps the audit trail clean: a user looking at a property's plan can see exactly which figure drove the recommendation pipeline. Storing only one set forces a downstream consumer to recompute the missing one from raw EPC fields when it needs both, which is the kind of derivation creep we want to keep out of the FE.
|
||
|
||
The cost is a wider row + the discipline that **every** `PropertyBaselinePerformance` populates both halves, even when they're equal. Annual kWh, fuel split and bills are not paired — they are always derived deterministically by `EpcEnergyDerivationService` against the Effective state, because the EPC's recorded cost fields use fuel rates pinned to the inspection date and the UCL correction depends on the modelled band.
|
||
|
||
## Consequences
|
||
|
||
- Reversing this means rewriting every consumer that has learned to read both values. Hard to roll back once the FE depends on the pair.
|
||
- The rebaseline trigger has two reasons (`pre_sap10`, `physical_state_changed`, or `both`) — store the reason alongside so we know *why* a property was rebaselined when debugging.
|
||
|
||
### Amendment (2026-05-30, #1135): standalone `property_baseline_performance` table
|
||
|
||
The original consequence read *"`property_details_epc` (or its successor) carries 8 fields
|
||
instead of 4 for the SAP-equivalent block"* — i.e. the pair as columns on the EPC-details table.
|
||
That is superseded. `property_details_epc` is being **retired**: it is too tightly coupled to the
|
||
schema of the legacy EPC API, which the Ara rebuild is moving off. So the pair has no home there.
|
||
|
||
`PropertyBaselinePerformance` instead persists as its **own standalone `property_baseline_performance` table, one
|
||
row per Property**, behind a dedicated `PropertyBaselineRepository` port (`save` / `get_for_property`),
|
||
mirroring the EPC slice's repo shape. This is the cleaner model regardless of the retirement:
|
||
`PropertyBaselinePerformance` is its own aggregate (a Property's current performance), not a detail of any
|
||
single EPC.
|
||
|
||
The row is **flat typed columns**, not a JSONB blob, because the FE both surfaces the block and
|
||
queries the lodged-vs-effective pair: `lodged_{sap_score, epc_band, co2_emissions,
|
||
primary_energy_intensity}`, the four `effective_*` mirrors, `rebaseline_reason`, and (for the part
|
||
of the energy block that needs no derivation) `space_heating_kwh` / `water_heating_kwh`. The
|
||
fourth paired quantity is **Primary Energy Intensity**, not "heat demand" — see CONTEXT.md
|
||
(the prose above predates that term being sharpened).
|
||
|
||
Fuel split and bills — the rest of the EPC Energy Derivation block — are **deferred to a
|
||
follow-up**: bills require a current Fuel Rates source (Ofgem-cap ETL) that does not yet exist, and
|
||
fuel split is produced by the same `EpcEnergyDerivationService`, so the two land together rather
|
||
than churning the table twice.
|
||
|
||
The SQLModel row is defined in `infrastructure/postgres/` so the ephemeral-Postgres tests build it
|
||
via `create_all`; the production migration is FE-owned (Drizzle ORM) and tracked in
|
||
`docs/migrations/`.
|
||
|
||
### Amendment (2026-06-30, #1361 Class B): the Lodged half is absent for a predicted Property
|
||
|
||
The original invariant — **every** `PropertyBaselinePerformance` populates **both** halves, even when
|
||
equal — assumed every Property has a record *of its own* performance to lodge: a public EPC cert (the
|
||
`epc_with_overlay` path) or a Site Notes survey. It does **not** hold for the **EPC Prediction** path
|
||
(ADR-0029/0031). A predicted Property has no record of itself — its `EpcPropertyData` is deterministic
|
||
neighbour synthesis that copies a representative comparable's structure wholesale, so the
|
||
`energy_rating_current` / band / CO2 / Primary Energy Intensity on the synthesised picture are a
|
||
*different dwelling's* lodged figures. Reading **Lodged Performance** off it manufactures a **phantom**:
|
||
a borrowed neighbour's SAP presented as this Property's government-register figure. This affected
|
||
**every** predicted Property (~12,236 rows estate-wide), not the 8 the `effective-lodged-divergence`
|
||
audit surfaced — that audit only fires when the borrowed figure lands ≥15 SAP from the Effective, so it
|
||
under-counts the phantom by three orders of magnitude.
|
||
|
||
**Decision.** The Lodged half is **optional**. When `source_path == "predicted"` there is no Lodged
|
||
Performance: `PropertyBaselinePerformance.lodged` is `None` and the four `lodged_*` columns are `NULL`.
|
||
The **Effective half is unchanged and still persisted** — a predicted Property is a first-class modelled
|
||
output (it flows through Rebaselining, Bill Derivation, and Modelling like any other; its Effective
|
||
Performance and bill block are correct and load-bearing for the FE and the plan-vs-effective audit
|
||
checks). `rebaseline_reason` stays `physical_state_changed` / `both`, which already records that the
|
||
Effective figure was scored from a changed picture.
|
||
|
||
**Principle (the boundary).** Lodged Performance requires a record **of this Property** — a lodged cert
|
||
*or* a Site Notes survey (the as-surveyed values are a real observation of this dwelling). EPC
|
||
Prediction borrows a neighbour's, so it has none. The branch is therefore `source_path == "predicted"` —
|
||
**not** `physical_state_changed` (true for Site Notes and Landlord Overrides too) and **not** "no public
|
||
EPC" (Site Notes has none yet keeps a legitimate Lodged Performance). Whether the Site Notes as-surveyed
|
||
values are truly "lodged" is a separate question, deferred.
|
||
|
||
**Rejected — a sentinel (`lodged = 0`).** Considered, to make "no record" visibly present rather than an
|
||
empty cell. Rejected: `0` is a valid-looking SAP score that (a) re-trips `effective-lodged-divergence`
|
||
for every predicted Property (`|effective − 0| ≥ 15`), (b) poisons every `AVG(lodged_*)` aggregate that
|
||
`NULL` is correctly excluded from, and (c) has no coherent `lodged_epc_band` enum member. The "no lodged
|
||
record" signal belongs in `NULL` (honest absence) plus the **structural provenance** already carried by
|
||
the distinct predicted-EPC slot — which the FE renders as a predicted badge — not overloaded onto the
|
||
score column.
|
||
|
||
**Consequences.**
|
||
- The four `lodged_*` columns become **nullable**. The production table is **FE-owned (Drizzle)**, so the
|
||
migration (`ALTER … DROP NOT NULL`) lands in the FE repo and **must precede** any backend write of a
|
||
`NULL` lodged, or the predicted-Property INSERT violates the constraint and aborts the batch
|
||
(ADR-0012). The SQLModel mirror in `infrastructure/postgres/` is updated to `Optional` so the
|
||
ephemeral-Postgres tests build the nullable shape.
|
||
- The fix lives in `PropertyBaselineOrchestrator.run()` — the single chokepoint both the First Run
|
||
pipeline and `applications/modelling_e2e/handler.py` call — so one change repairs both entry points.
|
||
The Rebaseliner port takes `Optional[Performance]`; for a predicted Property `physical_state_changed`
|
||
is always true, so `CalculatorRebaseliner` adopts the calculator output and never reads the absent
|
||
lodged half (the divergence-log path is the pristine-cert case only, where lodged is non-null).
|
||
- A one-time backfill (`scripts/`, dry-run default + `--apply`, idempotent) NULLs the four `lodged_*`
|
||
columns on existing predicted-source rows (~12,236). It corrects only the Lodged half; Effective, the
|
||
bill block, and `rebaseline_reason` are left intact.
|
||
- `effective-lodged-divergence` already short-circuits on `lodged_sap IS NULL`, so it goes green for
|
||
predicted Properties once backfilled. Class C of #1361 (a floor that ignores implausibly-low lodged
|
||
scores on *real* certs) is an additive guard, independent of this change.
|