diff --git a/docs/HANDOVER_MODELLING.md b/docs/HANDOVER_MODELLING.md new file mode 100644 index 00000000..8f538e29 --- /dev/null +++ b/docs/HANDOVER_MODELLING.md @@ -0,0 +1,63 @@ +# HANDOVER — Modelling stage rebuild + +**Branch:** `feature/bill-derivation` (worktree `/workspaces/home/hestia-worktrees/model-assemble-new-backend`). **HEAD:** `4c104050`. +**PRD:** GitHub `Hestia-Homes/Model#1152`, sliced into #1153–#1161. + +## Issue status + +| Issue | What | State | +|---|---|---| +| #1153 | Overlay Applicator + `EpcSimulation` | ✅ closed (`350f4c8e`) | +| #1154 | Package Scorer | code done (`7a478cff`); **Elmhurst cascade pin pending parser** | +| #1155 | wall Recommendation Generator | ✅ closed (`bb2c0068`) | +| #1156 | score Options + attribution | ✅ closed (`13dd5fe8`) | +| #1157 | persist a Plan via `ModellingOrchestrator` | **not started — HITL (persistence-schema review)** | +| #1158 | roof (loft) generator | generator done (`3c87be8e`); end-to-end + pin pending (#1157 + parser) | +| #1159 | floor generator | generator done (`4c104050`); end-to-end + pin pending | +| #1160 | Optimiser (knapsack + greedy repair) | not started (blocked by #1157/#1158/#1159) | +| #1161 | Measure Dependency (ventilation) | not started (blocked by #1160) | + +## Design (already recorded — read these) + +- **CONTEXT.md** terms: Recommendation (a *target surface*; Recommendations **partition** the modifiable EPD surface so overlays never collide), Measure Option (bundle-capable; deduped by overlay), **Simulation Overlay** (`EpcSimulation`), Product, Cost, Contingency, Measure Dependency. Targeting: building parts by `BuildingPartIdentifier`; **windows by index**; systems direct. +- **ADR-0016**: the three scoring roles (per-Option signal → whole-package re-score → final-package marginal cascade attribution) + warm-start MILP → dependency injection → package re-score → greedy repair. Resolves ADR-0005 §14. +- Governing: **ADR-0005** (multi-phase scenarios, per-phase recompute vs rolling Effective EPC), **ADR-0011** (composable stage orchestrators), **ADR-0012** (one Unit of Work per stage, commit once). + +## What's built + +All in `domain/modelling/`, `domain/building_geometry.py`, `repositories/product/`, `infrastructure/postgres/product_table.py`. **25 tests green, pyright strict clean, purely additive.** + +- `simulation.py` — `EpcSimulation(building_parts: Mapping[BuildingPartIdentifier, BuildingPartOverlay])`; `BuildingPartOverlay` (all-optional: `wall_insulation_type`, `roof_insulation_thickness`, `floor_insulation_thickness`). +- `overlay_applicator.py` — `apply_simulations(baseline, simulations) -> EpcPropertyData`. **Generic field-fold** (adding overlay fields needs NO change here — proven by roof/floor), sequential (later overlay wins), deep-copies (baseline never mutated), targets parts by identifier, writes the `sap_*` fields. Returns a throwaway EPD. +- `recommendation.py` — `Recommendation(surface, options)`, `MeasureOption(measure_type, description, overlay, cost)`, `Cost(total, contingency_rate)`. +- `product.py` / `contingencies.py` — `Product(measure_type, unit_cost_per_m2, contingency_rate)`; per-type contingency (cavity 0.10, loft 0.10, suspended floor 0.20, solid floor 0.26). +- `package_scorer.py` — `PackageScorer(calculator: SapCalculator).score(baseline, simulations) -> Score(sap_continuous, co2_kg_per_yr, primary_energy_kwh_per_yr)`. The reusable scoring primitive (role 2). +- `scoring.py` — `marginal_impacts(scorer, baseline, overlays) -> list[MeasureImpact]` (telescoping cascade, role 3); `independent_option_impacts(scorer, baseline, options)` (role 1, scores each *distinct* overlay once). `MeasureImpact(sap_points, co2_savings_kg_per_yr, energy_savings_kwh_per_yr)`. +- `wall_recommendation.py` — `recommend_cavity_wall(epc, products)`: detect cavity (`wall_construction==4`) + uninsulated (`wall_insulation_type==4`) → overlay sets `wall_insulation_type=2` (Table 6 "Filled cavity"). +- `roof_recommendation.py` — `recommend_loft_insulation(epc, products)`: detect `roof_insulation_thickness==0` → overlay `roof_insulation_thickness=270`. +- `floor_recommendation.py` — `recommend_floor_insulation(epc, products)`: detect uninsulated ground floor + construction (`floor_construction_type` "Suspended"/"Solid") → overlay `floor_insulation_thickness=100`. +- `building_geometry.py` — `gross_heat_loss_wall_area`, `roof_area`, `ground_floor_area` (per part, by identifier; party walls excluded; areas are heat-loss/§3.8 quantities, not totals). +- `repositories/product/` — `ProductRepository` (ABC port, `get(measure_type)->Product`); `ProductPostgresRepository` reads the externally-owned `material` table (defensive SQLModel view `MaterialRow`; `total_cost → unit_cost_per_m2`; joins contingency). A `ProductJsonRepository` (file source, for ETL-gap costs) is intended behind the same port — **the one remaining parser-independent AFK task**. + +## Key facts / gotchas + +- **Hand-built baseline fixture** (no PDF): `tests/domain/sap10_calculator/worksheet/_elmhurst_worksheet_000490.build_epc()`. Its MAIN is an uninsulated cavity wall + uninsulated suspended ground floor + 300 mm (insulated) loft. Used as the baseline in every generator/scorer test. MAIN gross heat-loss wall area = **45.93 m²**, roof area = **14.85 m²**, ground floor = **14.85 m²**. +- **Calculator entry:** `Sap10Calculator().calculate(epc) -> SapResult` (`sap_score_continuous`, `co2_kg_per_yr`, `primary_energy_kwh_per_yr`). Depend on the **`SapCalculator`** abstraction. Filled-cavity wall code = **2** (`domain/sap10_ml/rdsap_uvalues.py::u_wall`). Calculator reads wall/roof/floor from `SapBuildingPart` structured fields, NOT `EnergyElement` descriptions (those are detection-only). +- **Worktree vs main import trap:** `python /tmp/foo.py` imports the repo from `/workspaces/model` (editable install), NOT this worktree. Run with `PYTHONPATH=` or via `pytest` (rootdir handles it). `pytest` already uses worktree code. +- **Running tests:** `python -m pytest -q`. Do NOT pass `-p no:cov` (pytest.ini injects `--cov` args that then error). DB repo tests spin up ephemeral Postgres via the `db_engine` fixture (`tests/conftest.py`) — slower; SQLModel tables auto-register on import. +- **Conventions:** commit per TDD slice; conventional-commit message ending `Co-Authored-By: Claude Opus 4.8 `; stay on `feature/bill-derivation` (user's choice). Tests use literal `# Arrange / # Act / # Assert`; assert with `abs(x - y) <= tol` (not `pytest.approx`); pyright strict, zero errors; annotate call-return locals. + +## The two gates + +1. **Parser fix (in flight — `feature/per-cert-mapper-validation` agent → main).** Once cert **001431** parses, wire the **Elmhurst before/after cascade pins**: + - Files (main checkout): `/workspaces/model/sap worksheets/Recommendations Elmhurst Files//{before,after}/Summary_*.pdf` — `cavity_wall_insulation - main wall` (001431), `loft_insulation - main building`, `solid_floor`/`suspended_floor - main building`, etc. + - Pipeline: `from backend.documents_parser.parser import parse_site_notes_pdf; epd = parse_site_notes_pdf(path)`. + - Pin: parse `before` → apply the measure's overlay (the matching `recommend_*` Option's `overlay`) → `PackageScorer.score` → compare to `after` (either the `after` worksheet's SAP/kWh/carbon, or `Sap10Calculator().calculate(parse(after))`). `/tmp/spike_diff.py` diffs before/after EPDs to derive/validate the overlay empirically. + - The parser bug being fixed: `_extract_windows` reads `location`/`orientation`/`data_source` as single tokens, but 001431 lodges multi-token values ("External wall", "North West") with a blank Glazing Gap, so `'Manufacturer'` lands on the `u_value` float. (`ec9ef0e8` fixed a *different* symptom.) + - Closing these → #1154 done; #1158/#1159 end-to-end once #1157 exists. +2. **#1157 persist a Plan (HITL).** Design-review the Plan / Plan Phase / Recommendation persistence schema + `ScenarioRepository` method shapes, then build `ModellingOrchestrator.run(property_ids, scenario_ids)` per ADR-0011/0012 (one UoW, commit once, thread only IDs, read via repos). Template: `orchestration/property_baseline_orchestrator.py`. Then roof/floor end-to-end + #1160 optimiser + #1161 ventilation dependency. + +## Relevant memories (auto-loaded) + +- `project_openos_conservation_data_gap` — EWI eligibility needs listed/conservation status, not ingested; blocks the solid-wall EWI slice (later), NOT the fabric tracers. +- `project_calculator_geometry_extraction` — the calculator holds reusable geometry; `building_geometry.py` is the start; DRY the calculator onto it later (coordinate with the calculator branch); **don't edit `heat_transmission.py` now**.