docs(epc-prediction): ADR-0031 production wiring + CONTEXT gating rule

Resolve the slice-5 design tree (grill-with-docs): estimation runs in Ingestion
(refines ADR-0029 dec-3; drops the #1227 "shift to Modelling" — no surviving
rationale, and stages communicate only via persisted state); predicted EPC is
persisted in a DISTINCT slot (EPC table + source discriminator) so lodged +
predicted coexist (enables EPC Anomaly Flags); provenance is structural (the
slot), not a field on EpcPropertyData; effective_epc/source_path gain a
"predicted" branch; slice-5 is gap-fill only; property_type is a REQUIRED input
(hard cohort filter) from Landlord Overrides, and Properties with unknown type
are gated out (no national defaults). OS postcode_search as a broader type
source is a noted follow-on. CONTEXT EPC Prediction entry gains the gating rule.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Khalim Conn-Kowlessar 2026-06-16 03:23:12 +00:00
parent 58d5b17145
commit d1227fd0c6
2 changed files with 88 additions and 1 deletions

View file

@ -66,7 +66,7 @@ The reference cohort matched to a target Property, used by **EPC Prediction** fo
_Avoid_: neighbours, similar properties, peer set _Avoid_: neighbours, similar properties, peer set
**EPC Prediction**: **EPC Prediction**:
Producing a Property's `EpcPropertyData` picture from its **Comparable Properties** when it has no EPC (~30% of UK homes, typically long-tenure). **Deterministic** neighbour synthesis (k-NN-style — *not* ML; no trained model): take the cohort **mode** for the homogeneous categoricals (wall / roof / floor construction + insulation, construction age band), copy a single representative comparable's **structure** wholesale (building parts, per-window dimensions + orientations, floor dimensions) so the picture stays internally consistent for the calculator, then apply **Landlord Overrides** and the known inputs on top. The result is scored through **SAP10 Calculation** like any other **Effective EPC**, so a predicted Property flows through Rebaselining, Bill Derivation, and Modelling unchanged — marked as predicted so the UI can flag it. The same cohort machinery also produces **EPC Anomaly Flags** for Properties that *do* have an EPC. A future learned-weighting refinement is possible but separate, as with the calculator's ML residual head. Producing a Property's `EpcPropertyData` picture from its **Comparable Properties** when it has no EPC (~30% of UK homes, typically long-tenure). **Deterministic** neighbour synthesis (k-NN-style — *not* ML; no trained model): take the cohort **mode** for the homogeneous categoricals (wall / roof / floor construction + insulation, construction age band), copy a single representative comparable's **structure** wholesale (building parts, per-window dimensions + orientations, floor dimensions) so the picture stays internally consistent for the calculator, then apply **Landlord Overrides** and the known inputs on top. The result is scored through **SAP10 Calculation** like any other **Effective EPC**, so a predicted Property flows through Rebaselining, Bill Derivation, and Modelling unchanged — held in a **distinct predicted-EPC slot** that coexists with any lodged EPC (so provenance is structural and the UI can flag it; see ADR-0031). A **known property type is required** — the hard cohort filter (a flat is never sized from houses) — supplied by a **Landlord Override** (or, later, an Ordnance Survey lookup); a Property whose property type is genuinely unknown is **gated out**, never predicted from a mixed-type cohort and never given a national default. The same cohort machinery also produces **EPC Anomaly Flags** for Properties that *do* have an EPC. A future learned-weighting refinement is possible but separate, as with the calculator's ML residual head.
_Avoid_: EpcPredictionService (no "service" suffix — name the operation), ML prediction (it is deterministic), EPC estimation _Avoid_: EpcPredictionService (no "service" suffix — name the operation), ML prediction (it is deterministic), EPC estimation
### Survey documents ### Survey documents

View file

@ -0,0 +1,87 @@
# EPC Prediction production wiring
ADR-0029 settled *how* EPC Prediction synthesises a Property's `EpcPropertyData`
from its **Comparable Properties**; this records *how it wires into the running
pipeline* — where estimation runs, how a predicted EPC is stored and told apart
from a lodged one, and which Properties are eligible. Resolved in a
grill-with-docs session after the algorithm + validation harness were built and
the accuracy backlog (#12221228) closed.
## Status
Accepted (design). Refines ADR-0029 decision 3. Implementation pending (slice 5).
## Decisions
### 1. Estimation runs in Ingestion — the #1227 "shift to Modelling" is dropped
The cohort fetch + predict happens in `IngestionOrchestrator`, when
`epc_fetcher.get_by_uprn` returns `None` — upholding ADR-0029 decision 3. A
design note from issue #1227 had proposed moving estimation (and its distance
calcs) into `ModellingOrchestrator`; that is reversed here. Ingestion is already
the EPC-acquisition phase and *already resolves the Property's coordinates*
(`spatial`), so it can run the geo-weighted prediction with no new IO surface;
the First Run stages communicate **only through persisted state** (the pipeline
threads just `property_ids`, each stage reloads the `Property`), so a prediction
produced in Modelling would either have to be persisted there anyway or
recomputed every run. No rationale for the Modelling shift survived review.
Baseline and Modelling stay untouched — they read a populated `effective_epc`.
### 2. The predicted EPC is persisted in a distinct slot, never overwriting the lodged one
Because stages communicate via persisted state, the prediction **must be saved**
for Modelling to see it — in-memory-only would never reach stage 3. It is stored
as a **distinct predicted-EPC slot** on the Property (the EPC table reused with a
`source` discriminator — `lodged` / `predicted`), so a lodged EPC and a predicted
EPC can **coexist** on one Property. Coexistence is load-bearing: it is what lets
the same cohort machinery produce **EPC Anomaly Flags** for Properties that *do*
have an EPC (the dual-use named in ADR-0029), and it means a later real-EPC fetch
fills its own slot without the predicted one muddying provenance. Rejected: a
single EPC slot with an `is_predicted` flag — it cannot hold both, so it forecloses
anomaly detection and makes "lodged later arrives" ambiguous.
### 3. Provenance is structural — on the Property, not on `EpcPropertyData`
`EpcPropertyData` gains no `predicted` / `source` field. Which slot the picture
came from *is* its provenance. `Property.effective_epc` / `source_path` gain a
`"predicted"` branch, used only when there is no lodged EPC **and** no Site Notes
(the existing precedence is unchanged; a real source always wins). The
**Validation Cohort** then excludes any Property whose `effective_epc` resolves
via the predicted slot — it has no same-spec lodged ground truth — and the UI
flags it as predicted. Keeping `EpcPropertyData` clean means every downstream
consumer (calculator, generators, bill derivation) is unchanged and oblivious to
how the picture was sourced, exactly as for Landlord-Override and Reduced-Field
pictures.
### 4. Slice 5 is gap-fill only; always-predict (anomaly) is a follow-on
Prediction runs only when `epc is None`. Predicting for *every* Property to
compare against its lodged EPC (EPC Anomaly Flags) is real and the slot model
supports it, but it triples the ingestion cohort IO and needs its own
comparison + divergence-threshold + UI surface — so it does not ride in on the
wiring slice. The predicted-EPC slot and the `ComparableProperties` repository
this slice introduces are exactly what the anomaly capability reuses.
### 5. A known property type is required — eligibility is gated, never defaulted
A `PredictionTarget` needs `postcode` (from `PropertyIdentity`), `coordinates`
(geospatial), and `property_type` + `built_form` + `wall_construction` from
**Landlord Overrides**. `property_type` is the **hard** cohort filter (a flat
must not be sized from houses), so it is a **required input**: a Property whose
property type is genuinely unknown is **gated out** before prediction — flagged
un-predictable, never predicted from a mixed-type cohort and never given a
national default. (An Ordnance Survey `postcode_search` source can supply
property type more broadly than landlord input does; wiring it is a later
enhancement that widens the eligible population — out of scope here.) The
`ComparableProperties` repository port deferred by ADR-0029 is built in this
slice: it owns the cohort IO (postcode search → per-cert fetch → UPRN→coordinate
resolution) and returns candidate `Comparable`s for the domain
`select_comparables` to filter.
## Consequences
- A schema change: the EPC store gains a `source` discriminator (or equivalent),
and the Property repository a `get_predicted_for_property` read.
- Slice 5's reach is bounded to EPC-less Properties with a landlord-supplied
property type. The `postcode_search` integration is the lever to broaden it.
- `EpcPropertyData` stays unchanged, so no downstream consumer is touched.