diff --git a/domain/sap10_calculator/docs/HANDOVER_POST_S0380_76.md b/domain/sap10_calculator/docs/HANDOVER_POST_S0380_76.md new file mode 100644 index 00000000..bdd4c486 --- /dev/null +++ b/domain/sap10_calculator/docs/HANDOVER_POST_S0380_76.md @@ -0,0 +1,267 @@ +# Handover — post S0380.74..76 + Appendix H integration + +Branch: `feature/per-cert-mapper-validation`. **HEAD `a532f75d`**. +Predecessor: [`HANDOVER_POST_S0380_73_APPENDIX_H_BLOCKED.md`](HANDOVER_POST_S0380_73_APPENDIX_H_BLOCKED.md). + +## Slices committed this session (S0380.74..76) + +Long-standing Appendix H 1.81× over-count CLOSED, orchestrator +wired into the HW cascade. + +| Slice | Commit | What | +|---|---|---| +| **S0380.74** | `3bf728ce` | Appendix H (H7) U3.3 monthly-integrated convention closes 1.81× over-count. SAP 10.2 internal ambiguity for (H7)m between p.75 (W/m² flux) and p.76's "from U3.3 in Appendix U" (kWh/m²/month integrated). Elmhurst follows U3.3; cascade was using U3.2. Fix: convert flux × hours/1000 inside (H9). **47/48 month-observations pin at <1e-4 kWh** across 4 fixtures (000565 + new A/B/C at `sap worksheets/Solar PV tests/`). | +| **S0380.75** | `a9143d09` | Wire Appendix H orchestrator into water-heating cascade. New `solar_water_heating_monthly_kwh_override` param on `water_heating_from_cert`; helper `_solar_hw_monthly_override` in `cert_to_inputs.py` calls orchestrator with RdSAP 10 §10.11 Table 29 defaults + cert-lodged collector orientation/pitch/overshading from Elmhurst Summary §16.0. Extended `Renewables` + `EpcPropertyData` + extractor + mapper. **Cert 000565 HW pin: +271.84 → −68.96 kWh/yr (4× closer).** | +| **S0380.76** | `a532f75d` | Combined-cylinder H12/H13 routing. Empirical pattern across 4 worksheets (cert 000565 H12=53 ≈ 160/3, cert A/B/C H12=37 ≈ 110/3) → combined-cylinder default H12 = ⅓ × cylinder volume per f-chart pre-heat-zone convention. Derives H12/H13 from `epc.has_hot_water_cylinder + sap_heating.cylinder_size`. **Cert 000565 solar Q_s: 268 → 283 kWh/yr (worksheet 281.35, Δ +1.73 = 0.6% error).** | + +**Test baseline at HEAD `a532f75d`:** 547 pass + 9 expected +`test_sap_result_pin[000565-*]` cascade-gap fails. +Pyright net-zero on every touched file. + +## Appendix H closure narrative (the trap that closed) + +The cascade vs worksheet ratio for cert 000565 was 1.81× — handover +[`HANDOVER_POST_S0380_73_APPENDIX_H_BLOCKED.md`](HANDOVER_POST_S0380_73_APPENDIX_H_BLOCKED.md) +concluded this was blocked on BS EN 15316-4-3:2017 access. That +framing was wrong. The answer lives in the SAP 10.2 spec itself, in +the cross-reference between (H7) and Appendix U §U3.3 that page 76 +makes verbatim. + +The diagnostic that closed the trap: + +1. **3 new solar-HW worksheets** generated by the user at + `sap worksheets/Solar PV tests/` (A-baseline-south-modest, + B-highY, C-lowY) — pooled with cert 000565 → 48 + month-observations. +2. **Empirical fit attempts** (Klein 6-coef refit, 9-coef extended + with XY interactions, multiplicative Y/X correction) all rejected + — none matched the worksheet's polynomial output without sign + flips or overfitting. +3. **ChatGPT-mediated documentary research** ruled out hidden BRE + errata, SBEM-style Method 2 corrections, EN-style Im definitions + in W/m². Identified SAP 10.2 internal p.75 vs p.77 inconsistency + over the H8 overshading factor (real but wrong direction). +4. **Back-solving the polynomial** at fixed X for Y_eff across 24 + worksheet-positive observations revealed Y_eff/Y_cascade took + ONLY two distinct values: **0.7200 (exact)** for 30-day months, + **0.7440 (exact)** for 31-day months — i.e., `days × 24 / 1000` + exactly. No utilizability function, no missing constant — a + per-month unit-conversion factor. +5. **Spec text resolution**: SAP 10.2 p.76 reads (H7)m as "Monthly + solar radiation per m² from U3.3 in Appendix U". §U3.3 (p.130) + defines the conversion `S_monthly = 0.024 × n_m × S(orient,p,m)` + — i.e. kWh/m²/month, NOT W/m². The cascade's `surface_solar_ + flux_w_per_m2` returns the §U3.2 24h-avg flux in W/m² (verified + bit-exact against worksheet line 295: SE 90° Jan region 0 = + 36.7938 W/m²). Equation H1 expected the U3.3 monthly integrated + value. The page-77 (H23) formula's `× hours / 1000` term + double-converts when (H7) is W/m² instead of kWh/m²/month. + +Full diagnostic in +[`BRIEF_APPENDIX_H_EN_15316_RESEARCH.md`](BRIEF_APPENDIX_H_EN_15316_RESEARCH.md) +§"Closure — 4-cert empirical investigation (2026-05-29)". + +## Cert 000565 state (HEAD `a532f75d`) + +| Pin | Cascade | Worksheet | Δ | Root cause | +|---|---:|---:|---:|---| +| **sap_score (int)** | **29** | **29** | **0 ✓ EXACT** | unchanged | +| sap_score_continuous | 29.2905 | 28.5087 | +0.7818 | downstream of HW + space_heating | +| ecf | 5.3073 | 5.3866 | −0.0793 | downstream | +| total_fuel_cost_gbp | 4611.14 | 4680.26 | −69.12 | downstream | +| co2_kg_per_yr | 6352.61 | 6447.63 | −95.02 | downstream | +| **space_heating_kwh** | **59274.46** | **59008.35** | **+266.11** | **RR fold-in (RdSAP §3.10 detailed-RR geometry)** | +| main_heating_fuel | 34867.33 | 34710.79 | +156.53 | follows space_heating (Δ × 1/COP) | +| **hot_water_kwh** | **3668.54** | **3755.03** | **−86.49** | **demand cascade gaps (see below)** | +| lighting | 1387.02 | 1384.84 | +2.19 | sub-spec | +| pumps_fans | 255.00 | 252.52 | +2.48 | MEV PCDB record missing | + +### Appendix H solar Q_s — DONE (spec-pinned) + +| Quantity | Cascade | Worksheet | Δ | +|---|---:|---:|---:| +| Solar Q_s annual | 283.08 | 281.35 | **+1.73 kWh (0.6%)** | +| Solar Q_s monthly | various | various | **<1e-4 kWh per month** for 47/48 observations across 4 fixtures | + +The orchestrator is now spec-pinned. Remaining HW pin gap is in the +demand cascade, not in solar. + +## Open thread #1 — Cert 000565 HW demand cascade gaps (3 bugs) + +The −86 kWh HW residual is the net of three independent +demand-cascade bugs that were previously masked by the +357 kWh "no +solar credit" over-count: + +### A) Primary loss (59)m missing — biggest single fix (+1175 kWh) + +For HP main + HW cylinder routing, the cascade leaves +`primary_loss_monthly_kwh = 0`. Worksheet line (59)m for cert 000565 +sums to **1174.79 kWh/yr**. SAP 10.2 §4 line 7700 + Table 3 (PDF +p.159) defines primary loss for indirect cylinders. The current +`_primary_loss_override` helper at +[`cert_to_inputs.py:3322`](../rdsap/cert_to_inputs.py#L3322) +gates on a `_primary_loss_applies(main, cylinder_present, hp_record)` +check that returns False for cert 000565 — likely because the HP +main has integral vessel info in PCDB but cert 000565's HP route is +to an EXTERNAL cylinder (not integral). Audit `_primary_loss_applies` +against SAP 10.2 §4 line 7700 specifically for the HP + external +cylinder case. + +Closing this would shift cert 000565 HW pin from −86 → −86+1175 = +**+1089** (over-shoot, but then the next two fixes bring it back). + +### B) (45)m energy_content over by 903 kWh + +| Component | Cascade | Worksheet | +|---|---:|---:| +| (45)m sum | 2189 | 1286 | +| (62)m sum | 3181 | 3060 | + +Cascade `energy_content_monthly_kwh` over-counts by 903 kWh/yr. +Likely candidates: +- Occupancy formula difference (cert 000565 TFA 319.91 m², worksheet + uses occupancy from line 42) +- Hot water demand per occupant — cert lodges "showers are both + electric and non-electric" per RdSAP Table 29 default for solar + systems, which may reduce the non-electric shower demand + +Audit `assumed_occupancy()` and the (42)-(45) cascade vs worksheet +lines for cert 000565 specifically. + +### C) Storage loss (56) over by 98 kWh + missing (57)m solar adjustment + +Cascade (56) = 992 vs worksheet (56) = 894 (Δ +98). + +Worksheet additionally computes (57)m = (56) × (H13−H12)/H13 — the +solar-adjusted storage loss. For cert 000565: (57) = 596.91 (vs +(56) = 893.95). When solar HW is present, (62)m uses (57) NOT (56). +Cascade currently passes (56) as `solar_storage_monthly_kwh_override` +without the (H13−H12)/H13 reduction. Fix: +- Audit cascade's storage loss formula at + [`cert_to_inputs.py:3289`](../rdsap/cert_to_inputs.py#L3289) + `_cylinder_storage_loss_override` — what's adding the extra 98? +- Add (57)m solar adjustment when solar HW is present. The + multiplier is `(cylinder_volume_l - dedicated_solar_storage_l) / + cylinder_volume_l` derivable from the existing + `_hot_water_cylinder_volume_l(epc)` helper and the same H12 = + cylinder_volume / 3 rule used in `_solar_hw_monthly_override`. + +### Total impact estimate + +If all three close: HW pin = −86 + 1175 (A) − 903 (B) − 396 (C: 98 ++ 298) = **−210** before re-pinning solar Q_s under the corrected +(62)m. The Q_s itself would shift slightly since (H17)m changes. +Expect HW pin to land within ±50 kWh of zero after all three + +re-equilibrium. + +## Open thread #2 — Cert 000565 RR fold-in (space_heating +266) + +Unchanged. RdSAP §3.10 detailed-RR geometry / area formula not in +repo. Largest energy residual after HW closes. Was already noted in +predecessor handover. + +## Open thread #3 — Cert 000565 MEV pumps_fans (+2.48) + +Unchanged. PCDB MEV record not in repo (cert lodges PCDF 500755). +Acquiring the PCDB MEV table is the gating step. + +## Open thread #4 — 12 gas-combi PV certs at +0.5..+1.6 PE + +Unchanged. S0380.73 cooking fix surfaced these — no worksheets +available for these certs. Re-pinned at current residuals. + +## Open thread #5 — 5 SAP-integer-residual certs + +Unchanged. All API-only (no worksheets). User has agreed not to +chase these without worksheet ground truth. + +## How to run the baseline + +```bash +PYTHONPATH=/workspaces/model python -m pytest \ + backend/documents_parser/tests/test_summary_pdf_mapper_chain.py \ + backend/documents_parser/tests/test_elmhurst_extractor.py \ + backend/documents_parser/tests/test_elmhurst_end_to_end.py \ + domain/sap10_calculator/worksheet/tests/test_e2e_elmhurst_sap_score.py \ + domain/sap10_calculator/worksheet/tests/test_appendix_h_solar.py \ + domain/sap10_calculator/rdsap/tests/test_cert_to_inputs.py \ + domain/sap10_calculator/rdsap/tests/test_golden_fixtures.py \ + --no-cov -q +``` + +Expected: **547 pass + 9 expected `test_sap_result_pin[000565-*]` +fails**. + +## Recommended next slice — primary_loss (59)m for HP + external cylinder + +Biggest single residual: +1175 kWh. Audit +`_primary_loss_applies(main, cylinder_present, hp_record)` at +[`cert_to_inputs.py`](../rdsap/cert_to_inputs.py) against SAP 10.2 +§4 line 7700. The HP + external cylinder case (cert 000565 shape: +HP main 1 + gas combi main 2 servicing DHW via WHC 914 + cylinder +present) likely falls outside the current gate. Test by adding the +HP-external-cylinder path and verifying the primary_loss matches +worksheet line (59)m for cert 000565. + +### What NOT to do + +- Don't re-investigate the Appendix H 1.81× over-count. Closed. +- Don't propose more Appendix H polynomial / utilizability fixes. + S0380.74's U3.3 fix is the correct answer. +- Don't try to close cert 000565's HW pin in one slice. The −86 is + the net of three independent demand-cascade bugs; each closes a + separate residual. +- Don't widen pin tolerances or xfail residual gaps + ([[feedback-zero-error-strict]]). The 9 cert 000565 fails are + the work queue. +- Don't reference SAP 10.3 ([[feedback-sap-10-2-only-never-10-3]]). + +## Standard workflow per slice + +1. Read SAP 10.2 spec page for the change — quote it in commit +2. Probe current cascade output, identify exact spec-vs-cascade gap +3. Write failing test FIRST (AAA structure) +4. Implement helper / change +5. Verify test passes +6. Run full handover suite (command above) +7. Check pyright on touched files — net-zero from baseline +8. Commit with spec citation +9. Update relevant memory if state changed + +## Files touched this session + +| File | Slice | Change | +|---|---|---| +| `domain/sap10_calculator/worksheet/appendix_h_solar.py` | S0380.74 | Rename `monthly_solar_energy_available_h9_w` → `_h9_kwh_per_month`, add `hours_in_month` param, apply U3.3 conversion | +| `domain/sap10_calculator/worksheet/tests/test_appendix_h_solar.py` | S0380.74 | Cert 000565 (H24)m magnitude pin at abs < 1e-3 kWh; H9 + Y23 unit tests updated for kWh/month units | +| `domain/sap10_calculator/docs/BRIEF_APPENDIX_H_EN_15316_RESEARCH.md` | S0380.74 | New "Closure" section with empirical evidence + root cause | +| `domain/sap10_calculator/docs/HANDOVER_POST_4_CERT_EMPIRICAL.md` | S0380.74 | NEW — closure handover (points to brief) | +| `datatypes/epc/surveys/elmhurst_site_notes.py` | S0380.75 | `Renewables` gains `solar_hw_collector_orientation` / `_pitch_deg` / `_overshading` | +| `datatypes/epc/domain/epc_property_data.py` | S0380.75 | Same three fields added | +| `datatypes/epc/domain/mapper.py` | S0380.75 | `from_elmhurst_site_notes` propagates the three new fields | +| `backend/documents_parser/elmhurst_extractor.py` | S0380.75 | §16.0 section parsing | +| `domain/sap10_calculator/worksheet/water_heating.py` | S0380.75 | `solar_water_heating_monthly_kwh_override` param on `water_heating_from_cert` | +| `domain/sap10_calculator/rdsap/cert_to_inputs.py` | S0380.75, S0380.76 | Table 29 constants + `_solar_hw_monthly_override` + `_orientation_from_summary_string` + `_hot_water_cylinder_volume_l` + combined-cylinder H12/H13 derivation; demand_pass intermediate call | + +## Spec source quick-reference + +- **SAP 10.2 full specification**: `domain/sap10_calculator/docs/specs/sap-10-2-full-specification-2025-03-14.pdf` + - Appendix H (solar thermal): p.74-78 + - Equation H1 + Y/X definitions: p.75 + - (H7) "from U3.3" + (H9) formula: p.76 + - (H22)-(H24) worksheet: p.77 + - Table H1 (collector params), Table H2 (overshading), Table H3 (coefficients): p.78 + - Appendix U §U3.2 (W/m² flux polynomial): p.128 + - Appendix U §U3.3 (kWh/m²/month integrated): p.130 + - §4 line 7700 + Table 3 (primary loss): p.159 + - Table 12d/12e (monthly electric factors): p.195-196 +- **RdSAP 10 specification**: `domain/sap10_calculator/docs/specs/rdsap-10-specification.pdf` + - §10.5 Table 28 (cylinder size codes → litres): p.[lookup] + - §10.11 Table 29 (solar panel defaults): p.58 +- **S10TP-04** (BRE Appendix H change note): `domain/sap10_calculator/docs/specs/sap10 technical papers/S10TP-04 - Change to Appendix H to include solar space heating - V1_3.pdf` +- **SAP 10.3** at `domain/sap10_calculator/docs/specs/sap-10-3-full-specification-2026-01-13.pdf`: **DO NOT reference** (project tracks 10.2 only per [[feedback-sap-10-2-only-never-10-3]]) + +## Memory updated this session + +- `project_cert_000565_recovery_state` — Appendix H closure + S0380.75/76 outcomes +- `MEMORY.md` — index entry refreshed diff --git a/domain/sap10_calculator/docs/NEXT_AGENT_PROMPT_POST_S0380_76.md b/domain/sap10_calculator/docs/NEXT_AGENT_PROMPT_POST_S0380_76.md new file mode 100644 index 00000000..29d71ac5 --- /dev/null +++ b/domain/sap10_calculator/docs/NEXT_AGENT_PROMPT_POST_S0380_76.md @@ -0,0 +1,153 @@ +# Next-agent prompt — post S0380.76 + +Branch: `feature/per-cert-mapper-validation`. +HEAD: `a532f75d`. + +Read these in order before any tool call: + +1. [`HANDOVER_POST_S0380_76.md`](HANDOVER_POST_S0380_76.md) (full state) +2. [`BRIEF_APPENDIX_H_EN_15316_RESEARCH.md`](BRIEF_APPENDIX_H_EN_15316_RESEARCH.md) §"Closure" (Appendix H lesson — for reference, not action) + +Also load these memories before starting: + +- `project_cert_000565_recovery_state` — cert 000565 slice history + current pin state +- `project_golden_coverage_state` — cohort state +- `feedback_sap_10_2_only_never_10_3` — **CRITICAL** — never reference SAP 10.3 spec +- `feedback_verify_handover_claims` — verify spec citations before implementing +- `feedback_spec_floor_skepticism` — "spec-precision floor" framing usually hides a real spec bug +- `feedback_zero_error_strict` — pyright net-zero per touched file +- `feedback_commit_per_slice` — one slice = one commit +- `feedback_spec_citation_in_commits` — quote spec text + page in commit messages +- `feedback_aaa_test_convention` — every new test uses `# Arrange / # Act / # Assert` +- `feedback_e2e_validation_philosophy` — component pins at <1e-3; SAP integer delta=0; no adaptive ceilings + +## State summary + +**Cumulative session result:** Long-standing Appendix H 1.81× +over-count CLOSED. Orchestrator wired into HW cascade. Test +baseline preserved at **547 pass + 9 expected +`test_sap_result_pin[000565-*]` cascade-gap fails**. + +Cert 000565 SAP integer rating is **EXACT (29)**. The 9 sub-pins +are now visible work-queue items rather than blocked-on-external- +data: + +| Pin | Δ kWh/yr | Root cause | +|---|---:|---| +| sap_score (int) | **0 ✓ EXACT** | unchanged | +| hot_water_kwh_per_yr | **−86.49** | Three demand-cascade bugs (see below). Solar Q_s itself is spec-pinned at Δ +1.73 of worksheet. | +| space_heating_kwh | +266.11 | RR fold-in (RdSAP §3.10 detailed-RR geometry) | +| main_heating_fuel | +156.53 | Follows space_heating via 1/COP | +| co2 | −95.02 | Downstream of HW | +| total_fuel_cost | −69.12 | Downstream of HW | +| ecf | −0.0793 | Downstream | +| sap_score_continuous | +0.7818 | Downstream | +| lighting | +2.19 | sub-spec | +| pumps_fans | +2.48 | MEV PCDB record missing | + +## Recommended next slice — primary_loss (59)m for HP + external cylinder + +**Biggest single residual fix: +1175 kWh.** + +For cert 000565 (HP main 1 + gas combi main 2 servicing DHW via +WHC 914 + cylinder present), the cascade leaves +`primary_loss_monthly_kwh = 0`. Worksheet line (59)m sums to +1174.79 kWh/yr. + +SAP 10.2 §4 line 7700 + Table 3 (PDF p.159) defines primary loss +for indirect cylinders. The current `_primary_loss_override` helper +at +[`cert_to_inputs.py`](../rdsap/cert_to_inputs.py) +(search for `def _primary_loss_override`) gates on +`_primary_loss_applies(main, cylinder_present, hp_record)` which +returns False for cert 000565 — likely because the HP main has +integral vessel info in PCDB but cert 000565's HP routes to an +EXTERNAL cylinder. + +### Audit steps + +1. Read SAP 10.2 §4 around line 7700 + Table 3 verbatim — what + determines primary loss for HP + external cylinder? +2. Probe cert 000565: what does `_primary_loss_applies` return and + why? Step through with `main = epc.sap_heating.main_heating_ + details[0]`, `cylinder_present = True`, `hp_record = heat_pump_ + record(main.main_heating_index_number)`. +3. Compare against cert worksheet line (59)m monthly values + (already extracted in handover): + ``` + (59)m = 128.38, 115.95, 128.38, 124.24, 128.38, 41.92, + 43.31, 43.31, 41.92, 128.38, 124.24, 128.38 + ``` + Sum = 1174.79. +4. Write failing test pinning `primary_loss_monthly_kwh` for cert + 000565 to worksheet line (59)m at abs < 1e-3 kWh. +5. Implement the HP-external-cylinder path in + `_primary_loss_applies` or its caller. +6. Verify cert 000565 HW pin moves from −86 → expected new value + (likely around +1089 before the next two demand fixes land). + +### Why this is one slice, not three + +The three demand-cascade bugs ((45)m over by 903, (59)m under by +1175, (56)/(57)m mismatch by ~396) are INDEPENDENT. Each closes a +separate spec rule: +- (45)m → audit `assumed_occupancy()` + (42)-(45) HW demand + cascade — separate slice, possibly the showers-electric-and-non- + electric Table 29 default +- (59)m → THIS SLICE, primary loss for HP + external cylinder +- (56)/(57)m → next-next slice: storage loss formula audit + (57)m + solar-adjusted routing + +Land them in priority order. Each will move the HW pin in a +predictable direction; recheck after each. + +## What NOT to do + +- **Don't re-investigate Appendix H 1.81× over-count.** CLOSED in + S0380.74. The U3.3 unit-convention fix is the correct answer. +- **Don't propose more polynomial / utilizability fixes.** +- **Don't try to close cert 000565 HW pin in one slice.** Three + independent demand-cascade bugs to fix; each is its own slice. +- **Don't widen pin tolerances or xfail residual gaps** per + [[feedback-zero-error-strict]]. The 9 cert 000565 fails are the + work queue. +- **Don't reference SAP 10.3** ([[feedback-sap-10-2-only-never-10-3]]). +- **Don't chase the 12 gas-combi PV certs or the 5 SAP-residual + certs without worksheets** — user has explicitly de-prioritised. + +## Standard workflow per slice + +1. Read SAP 10.2 spec page for the change — quote it in commit +2. Probe current cascade output, identify exact spec-vs-cascade gap +3. Write failing test FIRST (AAA structure) +4. Implement helper / change +5. Verify test passes +6. Run full handover suite (command below) +7. Check pyright on touched files — net-zero from baseline +8. Commit with spec citation +9. Update relevant memory if state changed + +## How to run the baseline + +```bash +PYTHONPATH=/workspaces/model python -m pytest \ + backend/documents_parser/tests/test_summary_pdf_mapper_chain.py \ + backend/documents_parser/tests/test_elmhurst_extractor.py \ + backend/documents_parser/tests/test_elmhurst_end_to_end.py \ + domain/sap10_calculator/worksheet/tests/test_e2e_elmhurst_sap_score.py \ + domain/sap10_calculator/worksheet/tests/test_appendix_h_solar.py \ + domain/sap10_calculator/rdsap/tests/test_cert_to_inputs.py \ + domain/sap10_calculator/rdsap/tests/test_golden_fixtures.py \ + --no-cov -q +``` + +Expected: **547 pass + 9 expected `test_sap_result_pin[000565-*]` +fails**. + +## Memory hygiene + +After the next slice, update: +- `project_cert_000565_recovery_state` — add primary_loss closure + state + which next demand-cascade bug to tackle. + +Good luck.