mirror of
https://github.com/Hestia-Homes/Model.git
synced 2026-07-12 13:29:04 +00:00
35a7c07812
Records the grilling-session decisions amending ADR-0029's validation:
- Source cohort keeps all cert vintages (components are agnostic of the SAP
methodology that rated them); only the held-out validation TARGET is
restricted to SAP 10.2. Amends ADR-0029 decision 5 ("pre-SAP10 dropped").
- Component Accuracy (predicted vs API actual components) is the primary,
calculator-independent signal. calc(predicted) vs calc(actual) rejected
(circular ground truth, hides calculator error); neighbour-mean-lodged-SAP
baseline rejected (mixes SAP versions). calc(predicted) vs API-lodged
SAP/carbon/PE kept as a secondary, calculator-floored guard.
- Two tiers: committed anonymized fixture (ratcheting CI gate) + bulk-export
national battle-test on harness/epc_bulk.py + harness/cohort.py, emitting
accuracy + a failure taxonomy, re-baselining the gate floors.
CONTEXT.md: Comparable Properties corrected to all-vintage source; new
Component Accuracy term. ADR-0029 Validation section marked superseded.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
50 lines
6.4 KiB
Markdown
50 lines
6.4 KiB
Markdown
# EPC Prediction from Comparable Properties
|
||
|
||
~30% of UK homes (typically long-tenure) have no EPC. **EPC Prediction** produces a Property's `EpcPropertyData` picture from its **Comparable Properties** so an EPC-less Property flows through the rest of the pipeline (Rebaselining, Bill Derivation, Modelling) unchanged. This records the load-bearing design decisions taken in a grill-with-docs session.
|
||
|
||
## Status
|
||
|
||
Accepted (design). Implementation pending.
|
||
|
||
## Decisions
|
||
|
||
### 1. Predict the physical picture, score it with our calculator — never a SAP scalar
|
||
|
||
Prediction outputs a structured `EpcPropertyData` (building parts, windows, floor dimensions, construction + insulation, age band); SAP / CO2 / PEI / per-end-use kWh come from running `Sap10Calculator` on it. This is the same "assemble a picture, score once" mechanic as every other **Effective EPC** path (Landlord Overrides, Reduced-Field Synthesis), so a predicted Property is fully usable downstream (bills, measures, optimiser) — a directly-aggregated SAP scalar (legacy `SearchEpc`) would be a dead-end number. It also makes the component-classification accuracy metrics meaningful and keeps errors traceable to a wrong component rather than an opaque regression.
|
||
|
||
### 2. Deterministic neighbour synthesis, not ML
|
||
|
||
No trained model, no learned weights, no fit pipeline: filter a cohort, take categorical modes, copy a representative template, apply overrides. CONTEXT's prior "ML mechanism" framing is corrected — calling it ML invited the wrong architecture (training data, model artifacts, retraining). A future *learned-weighting* refinement is possible but separate, mirroring the calculator's flagged-future ML residual head. The domain class is `EpcPrediction` (no "Service" suffix, per the `BillDerivation` convention).
|
||
|
||
### 3. Fetch-phase fallback, behind a domain service + a cohort repository port
|
||
|
||
A pure **`EpcPrediction`** domain service (cohort of comparable `EpcPropertyData` in → predicted `EpcPropertyData` out) sits behind a **`ComparableProperties`** repository port that owns the cohort IO (postcode search → per-cert fetch, cached). It wires into `IngestionOrchestrator._fetch`: when `epc_fetcher.get_by_uprn` returns `None`, fetch the cohort and predict, persisting the picture **marked as predicted** (so the UI flags it and the Validation Cohort excludes it). Baseline and Modelling are untouched. Chosen over a fetcher-decorator (hides a heavy cohort fetch behind `get_by_uprn`) and a dedicated stage (a whole stage for "fill the gap when absent", duplicating IO ingestion already does). The heavy cohort IO stays visible in the no-unit IO phase.
|
||
|
||
### 4. Hybrid synthesis: cohort-mode categoricals + a coherent structural template
|
||
|
||
You cannot average a list of windows (counts differ; a mean orientation is meaningless) or building parts. So:
|
||
- **Homogeneous categoricals** (wall / roof / floor construction + insulation, age band) → cohort **mode** (robust to one oddball; drives the classification-rate metrics).
|
||
- **Structure & geometry** (building parts, per-window dimensions + orientations, floor dimensions) → copied wholesale from a **single representative comparable** chosen to be consistent with those modes and closest on geo + size (internally consistent for the calculator; drives the window-area / building-parts / floor-area residual metrics).
|
||
- **Landlord Overrides** and the known inputs are applied **on top**.
|
||
|
||
Rejected: field-by-field aggregation (legacy — incoherent, may not score sensibly) and single-nearest-neighbour copied wholesale (one atypical neighbour sets the categoricals → weaker classification).
|
||
|
||
### 5. Cohort selection: filter-then-relax ladder, weighted by geo × recency × similarity
|
||
|
||
Selection hard-filters on identity (property type, built form) and any **known Landlord Override** (e.g. solid brick — the mixed-street border case) **while ≥ k comparables remain**, widening the geographic scope (postcode → postcode-prefix) or demoting a known to a strong weight when sparse. Survivors are weighted by **geographic proximity** (true coordinates via `GeospatialRepository`, not the legacy house-number proxy) **× recency** (newer EPCs are higher quality) **× physical similarity**; ~~pre-SAP10 / very old certs are dropped~~ (amended by [ADR-0030](0030-epc-prediction-validation-is-sap-version-aware-and-component-first.md): all vintages are kept — components are methodology-agnostic — with recency as a graduated weight; only the *validation target* must be SAP 10.2). So a known field acts twice: upstream on cohort selection, and again as an override on the final picture.
|
||
|
||
### 6. Dual use: gap-fill (no EPC) and anomaly flags (has EPC)
|
||
|
||
The same cohort + comparison machinery produces **EPC Anomaly Flags** for Properties that *do* have an EPC (e.g. "all neighbours are 1930s; this lodges 1950 — correct?") — advisory, surfaced for user review. The no-EPC gap-fill lands first; the always-on anomaly-flag wiring is a follow-on increment.
|
||
|
||
## Validation
|
||
|
||
> **Superseded by [ADR-0030](0030-epc-prediction-validation-is-sap-version-aware-and-component-first.md).** The SAP-version mixing in the cohort makes the lodged-SAP comparisons below (esp. the neighbour-mean baseline) invalid; validation is now component-first over SAP-10.2-only targets. The frozen-corpus leave-one-out shape stands.
|
||
|
||
A **frozen postcode-clustered corpus** (a one-off fetch caches N postcodes × all their certs as `EpcPropertyData`) backs an offline, deterministic, repeatable **leave-one-out** harness over thousands of properties: drop a property with an EPC from its own cohort, predict it, compare predicted vs actual. Metrics: **classification rate** on wall / roof / floor construction + insulation and construction age band; **residuals** on SAP, total window area + window count, building-parts count, total floor area. SAP is reported three ways to attribute error — predicted-then-calculated vs (a) lodged SAP (end-to-end), (b) calculator-on-actual-components (isolates prediction error), (c) a direct neighbour-mean SAP baseline (proves predict-then-calculate beats naïve averaging).
|
||
|
||
## Open (implementation-level)
|
||
|
||
- **Provenance marker** on the picture (predicted vs real) — exact representation TBD; needed for the UI flag and Validation Cohort exclusion.
|
||
- **No-cohort fallback** when zero comparables survive even after relaxing (low-confidence national property-type + age defaults, or skip-with-flag).
|
||
- **Confidence signal** (cohort size + agreement) carried for the UI and anomaly thresholds.
|