Model/docs/HANDOVER_EPC_PREDICTION.md

EPC Prediction — handover

Branch feature/epc-prediction @ d8f015fb (37 ahead of origin/main; local-only, not pushed). Tree clean. All ranked backlog (#1222–1228) closed.

What this is

Deterministic neighbour synthesis that predicts a structured EpcPropertyData for an EPC-less UK home from its postcode-cohort of neighbours, so it flows through the modelling pipeline. NOT ML. Validation methodology + harness are built; the work is a measurable accuracy backlog.

READ FIRST (hold the full state)

• Memory project_epc_prediction — the spine: design, every commit, metrics, the open fronts, gotchas. Read it first.
• docs/adr/0029-… (design, 6 forks) and docs/adr/0030-…component-first.md (validation methodology — internalise: predict components, SAP/carbon/PE are a calculator-floored secondary guard).
• Memory feedback_per_component_best_method — THE load-bearing principle this session established (see below).
• Convention memories: feedback_aaa_test_convention, feedback_abs_diff_over_pytest_approx, feedback_commit_per_slice, feedback_bigger_slices_for_uniform_work.

The methodology (ADR-0030)

• Component Accuracy is the PRIMARY signal — predicted vs API-actual components, calculator-free. SAP/CO₂/PE vs lodged is SECONDARY and calculator-floored.
• Source cohort keeps ALL cert vintages; only held-out validation TARGETS are SAP 10.2 (sap_version == 10.2).
• The committed Tier-1 gate (tests/domain/epc_prediction/test_component_accuracy_gate.py) runs the calculator-free scorer over the frozen anonymised fixture (tests/fixtures/epc_prediction/, 36 SAP-10.2 targets) and asserts per-component ratchet floors. Deterministic → exact. Tighten-only: when you improve a component, bump its floor in the same commit. A mapper or fixture change re-baselines floors (not a regression) — document it.

THE PRINCIPLE that drove this session

Give each component its own best-fit synthesis method; never force one global mechanism on all of them. Validated head-to-head on the harness:

• Permanent fabric categoricals (wall, age) → physical-similarity-weighted mode (size×age toward cohort centre).
• Time-varying components (roof insulation, glazing) → recency-weighted mode.
• Coherence-coupled cluster (heating) → coherent whole-cluster donor, NEVER field-moded.
• Point-estimate scalar (floor area) → cohort median (MAD-minimising).
• Geo-varying components (age, wall, floor, glazing) → additionally geo-proximity weighted; roof showed no geo signal → excluded. All live in domain/epc_prediction/epc_prediction.py as composable weight vectors (_similarity_weights × _recency_weights × _geo_weights, combined via _combine, fed to _weighted_mode).

Closed this session (#1222 was done before; #1223–1228 this session)

• #1226 per-prediction confidence (PredictionConfidence, compute-only; agreement strongly predicts correctness, r=0.582).
• #1224 physical-similarity-weighted categorical mode (wall_insul/roof/floor +1–3pp).
• #1223 per-component, NOT a global recency template: floor-area→cohort median + glazing→recency mode. (A global recency template was rejected — it disturbed the coherence-coupled heating cluster.)
• #1225 coherent heating donor (modal signature = fuel+category+cylinder, recency tie-break). Biggest SAP lever: control 66→74%, SAP MAE 7.08→6.00 pre-merge.
• #1228 PEI investigation — DISPROVED the unit-bug hypothesis (calc/lodged ratio 1.06); reframed as calc floor + prediction-sensitivity. Report now surfaces CO₂/PEI calc floors. (Open calc-branch remnant; largely closed by the main merge — see below.)
• #1227 geo-proximity weighting — grilled, signal-checked (STRONG GO, esp. age), built per-component. Batch GeospatialRepository.coordinates_for_uprns, coords threaded onto Comparable/PredictionTarget, haversine kernel (_GEO_SCALE_KM=0.1, gate-safe optimum). Intra-postcode lift modest (cohort = 1 postcode); the bigger prize is cross-postcode expansion (deferred, needs dense corpus).
• Corpus grown 40→150 postcodes (6e9f8312); roof-insulation ±1 reporting.
• Merged origin/main (96 commits of calculator/mapper gap fixes, 0b2827e9).

Current metrics (post-merge, 150-pc corpus, 514 SAP-10.2 targets)

Component Accuracy (calculator-free): wall 91.2, wall_insul 79.0, age 57.2 (±1 84.7), roof_construction 78.2, floor_construction 79.6, heating_fuel 96.9, heating_category 95.7, heating_control 73.9, water_fuel 96.3, water_code 95.3, has_cylinder 89.7, cylinder_insul 52.4, secondary 42.0, roof_insul 49.3 (±1 53.7), floor_insul 94.7, room_in_roof 96.5, glazing 67.3, pv 98.8, solar 99.8.

Floor area: MAE 10.48 m² / MAPE 13.2% / typical (median actual) 61 m² (cohort median, unweighted).

End-to-end vs lodged (SECONDARY, calculator-floored): SAP pred MAE 6.25 / calc floor 0.95 (was 1.57 pre-merge, orig 3.25 — the calc fixes nearly validated the calculator, so the gap is now almost all prediction); CO₂ 0.61 / floor 0.18; PEI 39.6 / floor 13.7.

Key files

• domain/epc_prediction/epc_prediction.py — EpcPrediction.predict: median floor area + per-component weighted modes + glazing + heating donor + overrides.
• domain/epc_prediction/comparable_properties.py — select_comparables ladder; Comparable/PredictionTarget (carry coordinates).
• domain/epc_prediction/prediction_comparison.py — compare_prediction (25 signals).
• domain/epc_prediction/validation.py — iter_predictions + evaluate_component_accuracy (one scorer, calculator-free).
• harness/epc_prediction_corpus.py — load_corpus (+ _coordinates.json sidecar), load_coordinates, anonymise_payload.
• repositories/geospatial/ — GeospatialRepository.coordinates_for_uprns (batch).
• scripts/validate_epc_prediction.py (full report), build_epc_prediction_fixture.py, fetch_epc_prediction_corpus.py, fetch_corpus_coordinates.py.

Open fronts (ranked)

1. Geo-weighted floor-area median — measured quick win: MAE 10.48→9.77, MAPE 13.2→12.2%. Swap _median_floor_area for a geo-weighted median (reuse _geo_weights); gate-check + ratchet the floor_area ceiling. Smallest next slice.
2. Cross-postcode geo expansion — the real geo payoff (distance-weighted cohort beyond the single postcode). Needs a densely-sampled corpus (current 150 are scattered, so a target's true geo-neighbours aren't in-corpus). Design grilled; build a dense corpus first.
3. Slice-5 production wiring — ComparableProperties repo + the ModellingOrchestrator owning the EPC estimation + distance calcs (a deliberate shift from ADR-0029, which put the fallback in Ingestion). WRITE AN ADR when this lands (it reverses where the fallback lives). Add a provenance marker (EpcPropertyData has no predicted/source field yet).
4. Weak components with headroom only via NEW signals: age 57% / roof_insul 49% (method-exhausted — confirmed recency/similarity/plain all tie-or-worse); cylinder_insul / secondary are tiny-n.

How to run

• Token + S3 creds: set -a; . backend/.env; set +a (AWS creds mounted at ~/.aws).
• Tests: PYTHONPATH=. python -m pytest tests/domain/epc_prediction tests/harness/test_epc_prediction_corpus.py tests/repositories/geospatial -o addopts="" -p no:cacheprovider -q
• Full report: PYTHONPATH=. python scripts/validate_epc_prediction.py (corpus /tmp/epc_prediction_corpus).
• Gate is just a pytest test (deterministic, calculator-free).
• pyright strict, zero new errors, on every touched file.

In-flight / gotchas

• Corpus lives in /tmp/epc_prediction_corpus (gitignored; 150 pc / 3719 certs + _coordinates.json). Backed up to /workspaces/home/epc_prediction_corpus_backup (persistent host mount — survives container rebuild; /tmp does NOT). Coords backup at /workspaces/home/epc_prediction_corpus_coords_backup.json. If /tmp is wiped, restore from the backup before running the full report.
• Coordinates: OS Open-UPRN parquet is DATA_BUCKET/spatial/ (boto3 — s3fs NOT installed; read via get_object→BytesIO; boto3.client needs # pyright: ignore[reportUnknownMemberType, reportUnknownVariableType]). The cert payload carries uprn (the join key). The committed fixture ships _coordinates.json (OGL OS OpenData) so the gate exercises geo without S3.
• NEVER commit the API token, /tmp corpus, or the coords cache. The tests/fixtures/epc_prediction one is anonymised + intentional.
• Conventions: AAA test headers; abs(x-y) <= tol not pytest.approx; commit per slice (stage by name, watch untracked); ADR-cite in commit messages; class is EpcPrediction (no "Service").
• Per-item workflow: implement TDD red→green on this branch → run the harness → record before/after → ratchet gate floors → gh issue comment impact → close.
• The merge is local, not pushed — push only if asked.
• Update memory project_epc_prediction as state changes.