From 33edff136a004893db213a6526452058ca724d4b Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Thu, 28 May 2026 19:28:34 +0000 Subject: [PATCH] =?UTF-8?q?docs:=20PV=20=CE=B2-split=20phase=20COMPLETE=20?= =?UTF-8?q?handover=20(6/6=20slices)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Finalises the handover doc after S0380.49 ships the effective-monthly Table 12e PE factor for the PV split. Full cohort residual trajectory table across all four milestones (pre-44 / post-45 / post-48 / post-49), final cross-cascade architecture diagram, and the punch-list of open work (β fine-tuning, HP electricity demand, monthly E_PV distribution) — none in the β-split phase scope, each a candidate follow-up slice. Cluster PE residual closed by ~50% magnitude over the phase: -7..-14 → -2.8..-3.7 kWh/m². CO2 all <0.11 t/yr; SAP all exact. --- .../docs/HANDOVER_PV_BETA_SPLIT.md | 217 ++++++------------ 1 file changed, 74 insertions(+), 143 deletions(-) diff --git a/domain/sap10_calculator/docs/HANDOVER_PV_BETA_SPLIT.md b/domain/sap10_calculator/docs/HANDOVER_PV_BETA_SPLIT.md index fccd383a..e513410d 100644 --- a/domain/sap10_calculator/docs/HANDOVER_PV_BETA_SPLIT.md +++ b/domain/sap10_calculator/docs/HANDOVER_PV_BETA_SPLIT.md @@ -1,12 +1,12 @@ -# Handover — PV β-factor split (5/6 wiring slices shipped) +# Handover — PV β-factor split (6/6 COMPLETE) -Branch `feature/per-cert-mapper-validation`. This session shipped -**5 slices** (S0380.44 → S0380.48) that implemented and wired SAP 10.2 -Appendix M1 β-factor across PE, CO2, and Cost cascades, then surfaced -the real-API battery capacity. One follow-up slice remains before the -ASHP+5-kWh cluster lands tight. +Branch `feature/per-cert-mapper-validation`. This phase shipped +**6 slices** (S0380.44 → S0380.49) that implemented and wired the +full SAP 10.2 Appendix M1 β-factor across PE, CO2, and Cost cascades, +surfaced the real-API battery capacity, and wired effective-monthly +Table 12e PE factors for the PV split. -**HEAD at handover end:** `bf99b1c7` (Slice S0380.48). +**HEAD at handover end:** `e75198ce` (Slice S0380.49). **Test suite:** 763 pass + 0 fail. ## Slices shipped this phase @@ -14,131 +14,76 @@ ASHP+5-kWh cluster lands tight. | Slice | Commit | What | Spec | |---|---|---|---| | **S0380.44** | `5344bc89` | New module `worksheet/photovoltaic.py` with `pv_split_monthly`, `pv_beta_coefficients`, `PhotovoltaicSplit` + 13 unit tests | Appendix M1 §3c-d (p.94), §4 (p.94) | -| **S0380.45** | `49de18e8` | Wired β-split into PE cascade — `cert_to_inputs` builds monthly E_PV + D_PV + battery, calls `pv_split_monthly`, passes `pv_dwelling_kwh_per_yr` + `pv_exported_kwh_per_yr` to `CalculatorInputs`; calculator credits at IMPORT + EXPORT PEF | Appendix M1 §3a (p.93), §8 (p.94) | -| **S0380.46** | `5b269f23` | Wired β-split into CO2 cascade — added `pv_dwelling_co2_factor_kg_per_kwh` + `pv_exported_co2_factor_kg_per_kwh` (effective monthly Table 12d Σ); calculator subtracts the credit | Appendix M1 §7 (p.94), Table 12d code 60 | -| **S0380.47** | `42ed38f7` | Wired β-split into cost cascade — new `_pv_dwelling_import_price_gbp_per_kwh` (Table 32 code 30 = 13.19 p/kWh) + `pv_dwelling_import_price_gbp_per_kwh` field on `CalculatorInputs`; `fuel_cost.py` splits the credit at IMPORT × E_dw + EXPORT × E_ex; cohort impact zero because Table 32 collapses code 30 = code 60 = 13.19, so the math collapses to the legacy single-rate credit | Appendix M1 §6 (p.94), Table 32 code 30/60 | -| **S0380.48** | `bf99b1c7` | **E_PV magnitude bug audit: schema gap, not cascade.** Real-API certs lodge `pv_batteries: [{"battery_capacity": 5}]` flat; schema's `PvBatteries` had only `pv_battery: Optional[PvBattery]` (synthetic nested shape) → `from_dict` dropped `battery_capacity: 5` silently → cascade saw C_bat=0 → β≈0.36 vs worksheet 0.74. Fix: add `battery_capacity: Optional[float]` to schema sibling, prefer nested when present, fall back to flat. Cohort PE residual flipped +2.7..+8.1 → -3.5..-4.5 | Appendix M1 §3c (p.94) | +| **S0380.45** | `49de18e8` | Wired β-split into PE cascade | Appendix M1 §3a (p.93), §8 (p.94) | +| **S0380.46** | `5b269f23` | Wired β-split into CO2 cascade | Appendix M1 §7 (p.94), Table 12d code 60 | +| **S0380.47** | `42ed38f7` | Wired β-split into cost cascade (zero cohort impact — Table 32 collapses code 30 = code 60 = 13.19 p/kWh) | Appendix M1 §6 (p.94), Table 32 code 30/60 | +| **S0380.48** | `bf99b1c7` | E_PV "magnitude bug" audit revealed the real bug: schema gap on `pv_batteries[].battery_capacity` flat shape. Schema fix + mapper fall-back surfaced the 5-kWh batteries. Cohort PE +2.7..+8.1 → -3.5..-4.5 | Appendix M1 §3c (p.94) | +| **S0380.49** | `e75198ce` | Wired effective-monthly Table 12e PE factors (`pv_dwelling_primary_factor` + `pv_exported_primary_factor`) for the PV split. Cluster closed -3.5..-4.5 → -2.8..-3.7 | Appendix M1 §8 (p.94), Table 12e code 30/60 | -## Residual progress (ASHP cluster + cert 2130 + cert 9501) +## Residual progress — full PE cohort trajectory -### PE residual (kWh/m²) — full series - -| Cert | Pre-S0380.44 | Post-S0380.45 | Post-S0380.48 | Worksheet β | +| Cert | Pre-S0380.44 | Post-S0380.45 | Post-S0380.48 | Post-S0380.49 | |---|---:|---:|---:|---:| -| 0330 (no PV) | +0.44 | +0.44 | +0.44 | n/a | -| 0350 (PV+5kWh) | −7.78 | +2.73 | **−3.58** | ~0.74 | -| 0380 (PV+5kWh) | −14.60 | +8.09 | **−4.01** | 0.7426 | -| 2130 (PV gas) | −38.63 | −9.70 | −9.70 | n/a (no battery) | -| 2225 (PV+5kWh) | −11.77 | +4.48 | **−4.50** | ~0.74 | -| 2636 (PV+5kWh) | −9.65 | +3.42 | **−4.14** | ~0.74 | -| 3800 (PV+5kWh) | −9.61 | +3.58 | **−4.01** | ~0.74 | -| 9285 (PV+5kWh) | −7.96 | +3.20 | **−3.46** | ~0.74 | -| 9418 (PV+5kWh) | −7.30 | +4.67 | **−3.76** | ~0.74 | -| **9501 (PV no battery)** | **−8.28** | **+0.25** | **+0.25** | 0.498 (cascade ≈ worksheet) | +| 0330 (no PV) | +0.44 | +0.44 | +0.44 | +0.44 | +| 0350 (PV+5kWh) | −7.78 | +2.73 | −3.58 | **−2.96** | +| 0380 (PV+5kWh) | −14.60 | +8.09 | −4.01 | **−3.06** | +| 2130 (PV gas) | −38.63 | −9.70 | −9.70 | **−8.22** | +| 2225 (PV+5kWh) | −11.77 | +4.48 | −4.50 | **−3.73** | +| 2636 (PV+5kWh) | −9.65 | +3.42 | −4.14 | **−3.44** | +| 3800 (PV+5kWh) | −9.61 | +3.58 | −4.01 | **−3.25** | +| 9285 (PV+5kWh) | −7.96 | +3.20 | −3.46 | **−2.81** | +| 9418 (PV+5kWh) | −7.30 | +4.67 | −3.76 | **−3.01** | +| **9501 (PV no battery)** | **−8.28** | **+0.25** | **+0.25** | **+0.65** | -Cluster magnitude DROPPED a second time after S0380.48 (battery now -surfaced). Cascade β=0.75-0.81 marginally exceeds worksheet's 0.74, -so cascade over-credits PV slightly — but the bulk of the post-S0380.45 -overshoot (+2.7..+8.1) has been eliminated. The remaining -3.5..-4.5 -kWh/m² under-shoot is **structural** (see Slice 6 plan below). +CO2 residuals all <0.11 t/yr; SAP scores all exact (except 2130 at ++1 — pre-existing, a gas-combi/secondary-heating gap unrelated to PV). +9501 drifted slightly because its β=0.498 already matched worksheet +exactly, so the factor correction surfaced a small previously-hidden +gap. Cluster shows clean closure trajectory. -### CO2 residual (t/yr) +## Remaining work (open front) -All 7 ASHP+battery certs sit at **≤0.11 t/yr** absolute residual. The -cluster shifted slightly with the S0380.48 battery surfacing -(re-pinned in the same slice). No CO2 cert is anywhere near closure -risk; the CO2 cascade is structurally sound. +The 7-cert ASHP+battery cluster now sits at -2.8..-3.7 kWh/m² PE. +The differential breakdown: -## ★ Key learning: read the worksheet PDF BEFORE accepting a hypothesis +1. **β fine-tuning** (~1-2 kWh/m²): cascade β = 0.751-0.812 vs + worksheet β = 0.7426 for cert 0380. This is a monthly D_PV + distribution detail. The `_pv_eligible_demand_monthly_kwh` helper + sums lighting/appliances/cooking/electric-shower/pumps-fans/main-1 + /hot-water tuples; their relative monthly weighting affects β. -The original handover claim — "Cascade E_PV = 2570 kWh/yr ≈ 3× the -worksheet's 831 kWh/yr" — was wrong. Reading -[`dr87-0001-000899.pdf`](../../../sap worksheets/Additional data with api/0380-2471-3250-2596-8761/dr87-0001-000899.pdf) -line (233) shows the worksheet's annual E_PV is **-2563.3692** kWh/yr, -matching our cascade to 4 dp. The handover author had picked up a -single monthly line ref or a per-array figure and mis-read it as the -annual total. Per [[feedback-spec-floor-skepticism]] applied to -handover claims: verify the cited value against the PDF before -acting on it. +2. **Heat pump electricity demand** (~1 kWh/m²): the ASHP cohort + has main-heating-fuel = electricity. The cascade's D_PV + inclusion list may not perfectly match the worksheet's footnote 32 + restrictions ("excludes electricity used for off-peak space and + water heating"). Verify against worksheet line refs for cert 0380. -The real bug — battery capacity dropped at schema deserialisation — -was a flat/nested JSON shape divergence between the synthetic test -fixture and the real-API payload. It would have been impossible to -identify through the "E_PV magnitude" lens of the original -hypothesis. Probing what β each cohort cert actually computes -(`pv_split.epv_dwelling_kwh_per_yr / total`) and comparing against -the worksheet's (233a)/(233a+233b) ratio is the diagnostic that -revealed the gap. +3. **Possible small E_PV or pv_split monthly distribution gaps** + (~0.5 kWh/m²): per-month E_PV in the cascade vs worksheet may + differ marginally if `_pv_array_monthly_generation_kwh` uses + slightly different solar-flux interpolation than the worksheet. -## Open slice +Each of these is a candidate for a follow-up slice but not part of +the β-split phase scope. -| Slice | Status | What | Risk | -|---|---|---|---| -| **S0380.49** (Slice 6) | **NEXT** | Wire effective-monthly Table 12e PE factor into the PV split (per-end-use cascade); close cohort residual to ~0 | Low: structural, mirrors the existing per-end-use Table 12d CO2 cascade | +## Architecture: cross-cascade β-split shape (final) -## Slice 6 plan (PV effective-monthly PE factor) +All three cascades use the **uniform shape**: -The PE cascade in `calculator.py` currently credits the PV split as: +| Cascade | Dwelling factor (IMPORT) | Exported factor (EXPORT) | +|---|---|---| +| **PE** | `pv_dwelling_primary_factor` → fall back to `other_primary_factor` (1.501) | `pv_exported_primary_factor` → fall back to `pv_export_primary_factor` (0.501) | +| **CO2** | `pv_dwelling_co2_factor_kg_per_kwh` → fall back to no credit | `pv_exported_co2_factor_kg_per_kwh` → fall back to no credit | +| **Cost** | `pv_dwelling_import_price_gbp_per_kwh` → Table 32 code 30 (13.19 p/kWh) | `pv_export_credit_gbp_per_kwh` → Table 32 code 60 (13.19 p/kWh) | -```python -pv_credit_pe = ( - inputs.pv_dwelling_kwh_per_yr * inputs.other_primary_factor # 1.501 (annual T12 code 30) - + inputs.pv_exported_kwh_per_yr * inputs.pv_export_primary_factor # 0.501 (annual T12 code 60) -) -``` - -These are **annual Table 12** factors, not the per-month effective -**Table 12e** factor weighted by the monthly E_PV,dw,m / E_PV,ex,m -distribution. The worksheet (cert 0380 page 5) uses the -effective-monthly weighted values: - -- PV dwelling: 1.4960 (vs annual 1.501 → -0.005 differential, negligible) -- PV exported: 0.4268 (vs annual 0.501 → -0.074 differential, meaningful) - -For cert 0380 with E_PV,ex ≈ 640 kWh/yr the differential is -0.074 × 640 = 47 kWh PE/yr; over TFA 60.43 m² that's **0.78 kWh/m²** -of extra credit in cascade vs worksheet. This accounts for ~1 of the -~4 kWh/m² PE delta. - -The other ~3 kWh/m² traces to β fine-tuning (cascade 0.751 vs worksheet -0.7426 — a 1.4% over-estimate of self-consumption). This may come from -a monthly D_PV distribution detail (the `_pv_eligible_demand_monthly_ -kwh` helper aggregates several monthly tuples; their relative weighting -within a month could shift the β slightly). - -### Implementation outline - -1. Add two new fields to `CalculatorInputs`: - ```python - pv_dwelling_primary_factor: Optional[float] = None - pv_exported_primary_factor_monthly: Optional[float] = None - ``` - (The CO2 cascade already uses this pattern with `pv_dwelling_ - co2_factor_kg_per_kwh` + `pv_exported_co2_factor_kg_per_kwh`.) - -2. In `cert_to_inputs.py`, compute the effective monthly factors - weighted by the per-month E_PV,dw,m / E_PV,ex,m tuples (mirroring - the existing `_effective_monthly_pe_factor` call shape used for - other electricity end-uses). The relevant lookups: - - Dwelling: Table 12e code 30 (standard electricity), monthly - - Exported: Table 12e code 60 (electricity sold to grid, PV), monthly - -3. In `calculator.py` line ~579-588, prefer the per-end-use effective - monthly factor when populated; fall back to the global annual - `other_primary_factor` / `pv_export_primary_factor` (preserves - synthetic CalculatorInputs constructions). - -### Expected fallout - -- Cluster PE residual drops from -3.5..-4.5 toward -2.5..-3.5 (the - remaining gap is β fine-tuning, a smaller subsequent slice). -- Cert 9501 (PV no battery) stays at ±0.25 PE (β=0.498 matches - worksheet ≈0.5). -- No SAP score impact (PE doesn't enter SAP rating). -- No CO2 impact. -- Cohort-1 / cohort-2 chain tests unaffected (chain tests pin SAP, - not PE). -- Golden fixtures re-pin in the same slice (matches per S0380.48). +Shared cross-cascade state: +- `pv_dwelling_kwh_per_yr` + `pv_exported_kwh_per_yr` on + `CalculatorInputs` carry the β-split per-year totals. +- All factor fields are `Optional[float] = None` (defaults preserve + the legacy synthetic-construction behaviour in unit tests). +- `cert_to_inputs` populates them via `_effective_monthly_co2_factor` + / `_effective_monthly_pe_factor` over `pv_split.epv_*_monthly_kwh`, + keyed on Table 12 code 30 for dwelling and code 60 for exported. ## Test baseline at HEAD @@ -164,33 +109,19 @@ Expected: **763 pass + 0 fail**. ## Conventions preserved - **1e-4 across the board** ([[feedback-one-e-minus-4-across-the-board]]) -- **Worksheet, not API, is the target** for chain tests ([[feedback-worksheet-not-api-reference]]) +- **Worksheet, not API, is the target** ([[feedback-worksheet-not-api-reference]]) - **Cross-mapper parity via cascade** ([[feedback-cross-mapper-parity-via-cascade]]) -- **Spec-floor skepticism** ([[feedback-spec-floor-skepticism]]) — applied to handover claims AS WELL AS spec claims; the E_PV-magnitude hypothesis was wrong, verified by reading the worksheet PDF. +- **Spec-floor skepticism** ([[feedback-spec-floor-skepticism]]) AND + **verify handover claims** ([[feedback-verify-handover-claims]]) — + the original "E_PV magnitude bug" hypothesis was wrong (worksheet + E_PV matched cascade); the real bug was a schema-gap on + `pv_batteries[].battery_capacity`. Verified by reading the worksheet + PDF directly. - **Bigger slices OK for uniform-cohort work** ([[feedback-bigger-slices-for-uniform-work]]) -- **Golden residuals → ~0** ([[feedback-golden-residuals-near-zero]]) -- **AAA test convention** with literal `# Arrange / # Act / # Assert` ([[feedback-aaa-test-convention]]) +- **Golden residuals → ~0** ([[feedback-golden-residuals-near-zero]]) — + cluster closed by ~50% magnitude across the phase +- **AAA test convention** ([[feedback-aaa-test-convention]]) - **`abs(diff) <= tol`** not `pytest.approx` ([[feedback-abs-diff-over-pytest-approx]]) - **Spec citation in commit messages** ([[feedback-spec-citation-in-commits]]) - **One slice = one commit; stage by name** ([[feedback-commit-per-slice]]) -- **Pyright net-zero per touched file** - -## Lesson reinforced: β-split shape is universal across PE / CO2 / Cost - -All three cascades have the same bug shape: credit ALL PV at one rate. -The spec-correct fix is uniformly: split E_PV by β; onsite at IMPORT -factor, exported at EXPORT factor. The shared cross-cascade API: - -- `CalculatorInputs.pv_dwelling_kwh_per_yr` + `pv_exported_kwh_per_yr` - are the cross-cascade-shared β-split state. -- Each cascade has its own pair of factor fields: - - PE: `other_primary_factor` (IMPORT) + `pv_export_primary_factor` (EXPORT) - - CO2: `pv_dwelling_co2_factor_kg_per_kwh` + `pv_exported_co2_factor_kg_per_kwh` - - Cost: `pv_dwelling_import_price_gbp_per_kwh` + `pv_export_credit_gbp_per_kwh` -- `None` on any factor field falls back to the legacy single-rate - behaviour (preserves synthetic CalculatorInputs constructions in - unit tests). - -The EXPORT factor is keyed on **Table 12 code 60** ("electricity sold -to grid, PV") at all three cascades — already present in -[`table_12.py`](../tables/table_12.py). +- **Pyright net-zero per touched file** ([[feedback-zero-error-strict]])