Model/docs/HANDOVER_MODELLING.md

# HANDOVER — Modelling stage rebuild

**Branch:** `feature/bill-derivation` (worktree `/workspaces/home/hestia-worktrees/model-assemble-new-backend`). **HEAD:** `4c104050`.
**PRD:** GitHub `Hestia-Homes/Model#1152`, sliced into #1153–#1161.

## Issue status

| Issue | What | State |
|---|---|---|
| #1153 | Overlay Applicator + `EpcSimulation` | ✅ closed (`350f4c8e`) |
| #1154 | Package Scorer | code done (`7a478cff`); **Elmhurst cascade pin pending parser** |
| #1155 | wall Recommendation Generator | ✅ closed (`bb2c0068`) |
| #1156 | score Options + attribution | ✅ closed (`13dd5fe8`) |
| #1157 | persist a Plan via `ModellingOrchestrator` | **not started — HITL (persistence-schema review)** |
| #1158 | roof (loft) generator | generator done (`3c87be8e`); end-to-end + pin pending (#1157 + parser) |
| #1159 | floor generator | generator done (`4c104050`); end-to-end + pin pending |
| #1160 | Optimiser (knapsack + greedy repair) | not started (blocked by #1157/#1158/#1159) |
| #1161 | Measure Dependency (ventilation) | not started (blocked by #1160) |

## Design (already recorded — read these)

- **CONTEXT.md** terms: Recommendation (a *target surface*; Recommendations **partition** the modifiable EPD surface so overlays never collide), Measure Option (bundle-capable; deduped by overlay), **Simulation Overlay** (`EpcSimulation`), Product, Cost, Contingency, Measure Dependency. Targeting: building parts by `BuildingPartIdentifier`; **windows by index**; systems direct.
- **ADR-0016**: the three scoring roles (per-Option signal → whole-package re-score → final-package marginal cascade attribution) + warm-start MILP → dependency injection → package re-score → greedy repair. Resolves ADR-0005 §14.
- Governing: **ADR-0005** (multi-phase scenarios, per-phase recompute vs rolling Effective EPC), **ADR-0011** (composable stage orchestrators), **ADR-0012** (one Unit of Work per stage, commit once).

## What's built

All in `domain/modelling/`, `domain/building_geometry.py`, `repositories/product/`, `infrastructure/postgres/product_table.py`. **25 tests green, pyright strict clean, purely additive.**

- `simulation.py` — `EpcSimulation(building_parts: Mapping[BuildingPartIdentifier, BuildingPartOverlay])`; `BuildingPartOverlay` (all-optional: `wall_insulation_type`, `roof_insulation_thickness`, `floor_insulation_thickness`).
- `overlay_applicator.py` — `apply_simulations(baseline, simulations) -> EpcPropertyData`. **Generic field-fold** (adding overlay fields needs NO change here — proven by roof/floor), sequential (later overlay wins), deep-copies (baseline never mutated), targets parts by identifier, writes the `sap_*` fields. Returns a throwaway EPD.
- `recommendation.py` — `Recommendation(surface, options)`, `MeasureOption(measure_type, description, overlay, cost)`, `Cost(total, contingency_rate)`.
- `product.py` / `contingencies.py` — `Product(measure_type, unit_cost_per_m2, contingency_rate)`; per-type contingency (cavity 0.10, loft 0.10, suspended floor 0.20, solid floor 0.26).
- `package_scorer.py` — `PackageScorer(calculator: SapCalculator).score(baseline, simulations) -> Score(sap_continuous, co2_kg_per_yr, primary_energy_kwh_per_yr)`. The reusable scoring primitive (role 2).
- `scoring.py` — `marginal_impacts(scorer, baseline, overlays) -> list[MeasureImpact]` (telescoping cascade, role 3); `independent_option_impacts(scorer, baseline, options)` (role 1, scores each *distinct* overlay once). `MeasureImpact(sap_points, co2_savings_kg_per_yr, energy_savings_kwh_per_yr)`.
- `wall_recommendation.py` — `recommend_cavity_wall(epc, products)`: detect cavity (`wall_construction==4`) + uninsulated (`wall_insulation_type==4`) → overlay sets `wall_insulation_type=2` (Table 6 "Filled cavity").
- `roof_recommendation.py` — `recommend_loft_insulation(epc, products)`: detect `roof_insulation_thickness==0` → overlay `roof_insulation_thickness=270`.
- `floor_recommendation.py` — `recommend_floor_insulation(epc, products)`: detect uninsulated ground floor + construction (`floor_construction_type` "Suspended"/"Solid") → overlay `floor_insulation_thickness=100`.
- `building_geometry.py` — `gross_heat_loss_wall_area`, `roof_area`, `ground_floor_area` (per part, by identifier; party walls excluded; areas are heat-loss/§3.8 quantities, not totals).
- `repositories/product/` — `ProductRepository` (ABC port, `get(measure_type)->Product`); `ProductPostgresRepository` reads the externally-owned `material` table (defensive SQLModel view `MaterialRow`; `total_cost → unit_cost_per_m2`; joins contingency). A `ProductJsonRepository` (file source, for ETL-gap costs) is intended behind the same port — **the one remaining parser-independent AFK task**.

## Key facts / gotchas

- **Hand-built baseline fixture** (no PDF): `tests/domain/sap10_calculator/worksheet/_elmhurst_worksheet_000490.build_epc()`. Its MAIN is an uninsulated cavity wall + uninsulated suspended ground floor + 300 mm (insulated) loft. Used as the baseline in every generator/scorer test. MAIN gross heat-loss wall area = **45.93 m²**, roof area = **14.85 m²**, ground floor = **14.85 m²**.
- **Calculator entry:** `Sap10Calculator().calculate(epc) -> SapResult` (`sap_score_continuous`, `co2_kg_per_yr`, `primary_energy_kwh_per_yr`). Depend on the **`SapCalculator`** abstraction. Filled-cavity wall code = **2** (`domain/sap10_ml/rdsap_uvalues.py::u_wall`). Calculator reads wall/roof/floor from `SapBuildingPart` structured fields, NOT `EnergyElement` descriptions (those are detection-only).
- **Worktree vs main import trap:** `python /tmp/foo.py` imports the repo from `/workspaces/model` (editable install), NOT this worktree. Run with `PYTHONPATH=<worktree>` or via `pytest` (rootdir handles it). `pytest` already uses worktree code.
- **Running tests:** `python -m pytest <path> -q`. Do NOT pass `-p no:cov` (pytest.ini injects `--cov` args that then error). DB repo tests spin up ephemeral Postgres via the `db_engine` fixture (`tests/conftest.py`) — slower; SQLModel tables auto-register on import.
- **Conventions:** commit per TDD slice; conventional-commit message ending `Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>`; stay on `feature/bill-derivation` (user's choice). Tests use literal `# Arrange / # Act / # Assert`; assert with `abs(x - y) <= tol` (not `pytest.approx`); pyright strict, zero errors; annotate call-return locals.

## The two gates

1. **Parser fix (in flight — `feature/per-cert-mapper-validation` agent → main).** Once cert **001431** parses, wire the **Elmhurst before/after cascade pins**:
   - Files (main checkout): `/workspaces/model/sap worksheets/Recommendations Elmhurst Files/<measure>/{before,after}/Summary_*.pdf` — `cavity_wall_insulation - main wall` (001431), `loft_insulation - main building`, `solid_floor`/`suspended_floor - main building`, etc.
   - Pipeline: `from backend.documents_parser.parser import parse_site_notes_pdf; epd = parse_site_notes_pdf(path)`.
   - Pin: parse `before` → apply the measure's overlay (the matching `recommend_*` Option's `overlay`) → `PackageScorer.score` → compare to `after` (either the `after` worksheet's SAP/kWh/carbon, or `Sap10Calculator().calculate(parse(after))`). `/tmp/spike_diff.py` diffs before/after EPDs to derive/validate the overlay empirically.
   - The parser bug being fixed: `_extract_windows` reads `location`/`orientation`/`data_source` as single tokens, but 001431 lodges multi-token values ("External wall", "North West") with a blank Glazing Gap, so `'Manufacturer'` lands on the `u_value` float. (`ec9ef0e8` fixed a *different* symptom.)
   - Closing these → #1154 done; #1158/#1159 end-to-end once #1157 exists.
2. **#1157 persist a Plan (HITL).** Design-review the Plan / Plan Phase / Recommendation persistence schema + `ScenarioRepository` method shapes, then build `ModellingOrchestrator.run(property_ids, scenario_ids)` per ADR-0011/0012 (one UoW, commit once, thread only IDs, read via repos). Template: `orchestration/property_baseline_orchestrator.py`. Then roof/floor end-to-end + #1160 optimiser + #1161 ventilation dependency.

## Relevant memories (auto-loaded)

- `project_openos_conservation_data_gap` — EWI eligibility needs listed/conservation status, not ingested; blocks the solid-wall EWI slice (later), NOT the fabric tracers.
- `project_calculator_geometry_extraction` — the calculator holds reusable geometry; `building_geometry.py` is the start; DRY the calculator onto it later (coordinate with the calculator branch); **don't edit `heat_transmission.py` now**.