From 377962f8bdb9e8e420e3b55a80081b552dad5502 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Tue, 19 May 2026 07:35:19 +0000 Subject: [PATCH] =?UTF-8?q?docs:=20strengthen=20handover=20with=20=C2=A77b?= =?UTF-8?q?=20outstanding=20findings=20+=20PCDB=20roadmap?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit §7b "Outstanding findings to pick up during the systematic pass" collects spec-correct fixes that were reverted because they regressed SAP MAE against the corpus — but the spec basis is unambiguous and they WILL be the right answer once cert-calibration is re-derived. Treat as TODOs, not dead-ends. Documents: Finding 1 — HW cylinder zero-loss for combi (PE MAE -6.64 measured) Finding 2 — Standing charges Table 12 note (a) Finding 3 — Cat=10 room-heater Table 12a fractional blending Finding 4 — Lighting Appendix L proper (L1-L12 cascade) Finding 5 — Internal-gains Table 5 water-heating + losses rows Finding 6 — Storage-loss-factor table values 3× off spec Finding 7 — Heat-pump fallback (needs PCDB) Finding 8 — Smaller gaps carried forward Each documents the spec section/page reference, the current code bug, empirical impact where measured, and when to pick up during the section-by-section sweep. PCDB section strengthened from "deferred to Session C" to an explicit roadmap: data source URL, lookup key (main_heating_index_number), fields needed, recommended sequencing (after spec sweep so cert-cal is re-derivable), and why-not-now (cert-cal currently masks PCDB gaps). Co-Authored-By: Claude Opus 4.7 --- docs/sap-spec/HANDOVER_SYSTEMATIC_REVIEW.md | 230 +++++++++++++++++++- 1 file changed, 225 insertions(+), 5 deletions(-) diff --git a/docs/sap-spec/HANDOVER_SYSTEMATIC_REVIEW.md b/docs/sap-spec/HANDOVER_SYSTEMATIC_REVIEW.md index bf90df2a..eb001368 100644 --- a/docs/sap-spec/HANDOVER_SYSTEMATIC_REVIEW.md +++ b/docs/sap-spec/HANDOVER_SYSTEMATIC_REVIEW.md @@ -122,11 +122,41 @@ re-derive the cert-calibration once at the end. until the RdSAP 10 base case matches Elmhurst.** S-B24 / S-B29 attempted partial handling; those slices can stay or be reverted at your discretion when you reach §§4-7 of RdSAP and §3 of SAP 10.2. -- **PCDB (Product Characteristics Database).** ADR-0009 says Session C. - Heat pumps (cat=4) have catastrophic per-cert MAE because we use - Table 4a fallback efficiency 2.30 instead of PCDB SCOP. There's a - `NoOpPcdbLookup` stub seam ready in Session A; data fetch + parser - is its own milestone. +- **PCDB (Product Characteristics Database).** ADR-0009 deferred this + to Session C. **This is a real future task, not a permanent + exclusion.** Heat pumps (cat=4) have catastrophic per-cert MAE (19 + SAP points) because we use Table 4a fallback efficiency 2.30 + instead of PCDB SCOP (typically 2.80-3.50). Gas boilers with + `main_heating_data_source=1` (78% of corpus boiler certs) fall back + to a category-default 0.80 vs typical PCDB-listed condensing-boiler + efficiencies of 0.88-0.94 — that's most of the per-cert SAP residual + variance on gas certs. + + A `NoOpPcdbLookup` stub seam exists in Session A (per ADR-0009 grill + outcome #1). The fetch+parse work is non-trivial: + - **Data source**: BRE PCDB at https://www.ncm-pcdb.org.uk — + boilers + heat pumps are downloadable CSVs (thousands of rows + each). + - **Lookup key**: cert lodges `main_heating_index_number` which is + the PCDB product ID. Match by that. + - **Per-product fields needed**: seasonal efficiency, secondary + efficiency, output kW, flow-temperature curve (for HPs). + - **Effort**: ~half-day for the lookup + tests; ongoing maintenance + when BRE publishes new PCDB revisions. + + **Recommended sequencing**: complete the systematic RdSAP spec + sweep first. Once the spec-correct engine is built and cert-cal + re-derived, PCDB integration should drop heat-pump residuals from + 19 SAP points to ~1, and tighten the gas-boiler residual variance. + At that point heat pumps (cat=4) and PCDB-listed boilers + (`main_heating_data_source=1`) become accessible. + + **Why not now**: the cert-calibration prices currently absorb the + missing PCDB efficiency (HP costs at off-peak rate compensates for + too-low SCOP). Fixing PCDB without re-deriving cert-cal would push + HP certs in the wrong direction. Same lesson as the other reverted + fixes in §7b — fix the spec layer first, the calibration layer + later. - **SAP 10.3** (13-01-2026). The corpus is SAP 10.2. SAP 10.3 has identical Table 12 codes (only values shift). Don't update spec references to 10.3 until the corpus migrates. @@ -365,6 +395,196 @@ of state A, not entangled with it. --- +## 7b. Outstanding findings to pick up during the systematic pass + +The prior session identified several spec-correct fixes that were +**reverted because they made SAP MAE worse against the corpus, but the +spec basis is unambiguous and the fixes WILL be the right answer once +the cert-calibration is re-derived against a clean engine.** Treat +these as TODOs the systematic pass should encounter when it reaches +the relevant section. They're listed here so the work isn't lost. + +### Finding 1 — HW cylinder zero-loss rule for combi boilers +**Status**: spec-correct fix exists in working-tree-only form +(uncommitted). Reverted at end of last session. + +**Spec basis**: +- **SAP 10.2 Table 2 footer (page 158)**: "In the case of a + combination boiler: a) the storage loss factor is zero if the + efficiency is taken from Table 4b" +- **SAP 10.2 Table 3 footer (page 160)**: "Primary loss is set to + zero for the following: Electric immersion heater, Combi boiler + (including when it is part of a combined heat pump and boiler + package and provides all the hot water), CPSU (including electric + CPSU), Boiler and thermal store within a single casing, Separate + boiler and thermal store connected by no more than 1.5 m of + insulated pipework, Direct-acting electric boiler, Heat pump (not + combined heat pump and boiler package with a non-combi boiler) + from PCDB with hot water vessel integral to package" + +**The bug**: our calculator currently adds storage loss (~135 kWh) +and primary loss (~245 kWh) for ALL certs with an age band lodged, +ignoring whether the dwelling has a cylinder. **67% of corpus certs +explicitly lodge `has_hot_water_cylinder=False`** (the modal combi +boiler case) — we add 380 kWh of fictional HW losses for each. + +**The fix** (sketch, ~10 lines): +1. Add `has_cylinder: bool = True` keyword to + `predicted_hot_water_kwh` in `packages/domain/src/domain/ml/demand.py`. +2. When `has_cylinder=False`, set `storage_loss = 0` and `primary_loss = 0`. +3. In `cert_to_inputs.py` (around line 829), pass + `has_cylinder=epc.has_hot_water_cylinder and not is_instantaneous`. + +**Empirical impact** (measured on 300-cert probe): +- **PE MAE: 43.32 → 36.68 (−6.64) ← biggest single fix found this session** +- PE bias: 37.69 → 30.41 (−7.28) +- SAP MAE: 4.61 → 5.00 (+0.39, regression) +- 3 of 7 golden fixtures break + +**Why it was reverted**: the SAP regression + broken fixtures indicate +the fictional HW losses were partially compensating for OTHER bugs +(likely lighting over-prediction for LED-dominant homes). The right +ordering is: fix the spec-clear cases (HW cylinder, lighting per +Appendix L, etc.) together, then re-derive cert-cal. + +**When to pick up**: when you reach §4 / Appendix J during the +systematic pass. Pair with the lighting Appendix L fix to avoid +breaking the golden fixtures individually. + +### Finding 2 — Standing charges (Table 12 note (a)) +**Status**: spec-correct, never implemented. Empirically rejected by +4-mode probe. + +**Spec basis**: SAP 10.2 Table 12 note (a), page 190: +> "For calculations including regulated energy uses only (e.g. +> regulation compliance, energy ratings): +> - The standing charge for electricity standard tariff is omitted +> - The standing charge for off-peak electricity is added to space +> and water heating costs where either main heating or hot water +> uses off-peak electricity +> - The standing charge for gas fuels is added to space and water +> heating costs where the gas fuel is used for space heating +> (main or secondary) or for water heating" + +**The bug**: our calculator never adds standing charges. Per spec, a +gas-heated dwelling should have £92/yr added to the ECF numerator. + +**Empirical impact** (4-mode probe, 300 certs): +| Mode | All certs | Gas-only | +|---|---|---| +| cert-cal, no standing (current) | MAE 4.69, bias +0.98 | MAE 4.01, bias +0.80 | +| cert-cal + gas standing | MAE 4.94, bias **−2.62** | MAE 4.31, bias **−3.53** | + +Adding standing charges shifts SAP bias by ~3.5 points downward — +clearly the wrong direction. The cert-cal prices (3.48p gas vs spec +3.64p) implicitly absorb the standing-charge contribution. + +**When to pick up**: when you reach §12 / Table 12. Apply alongside +spec-correct unit prices (3.64p gas, 16.49p elec) and re-derive +cert-cal to match Elmhurst's residual deviation pattern. + +### Finding 3 — Cat=10 room heaters off-peak routing +**Status**: spec-correct, currently bills room heaters at off-peak +rate on E7 dwellings. Empirically rejected. + +**Spec basis**: SAP 10.2 Table 12a (page 191): +> "Other direct-acting electric heating (including electric secondary +> heating): 7-hour tariff 1.00 high rate; 10-hour tariff 0.50 high rate" + +**The bug**: our cert-calibration (`cert_calibration_e7_codes`) +extends the off-peak set to include codes 691-696 (room heaters). +That's the S-B14 empirical extension — the previous agent found it +helped some specific certs. Per Table 12a it's WRONG: room heaters +on E7 should bill 100% at HIGH rate, not at low rate. + +**Empirical impact**: switching from off-peak (5.50p cert-cal) to +standard rate (13.19p) — closer to spec but still not the high rate +(15.29p cert-cal) — inverted the bias from +5.88 to −6.00 without +improving MAE. + +**The real issue**: Table 12a defines FRACTIONAL blending (e.g. +"90% high, 10% low" for direct-acting electric boiler on 7-hour +tariff), not binary on/off-peak. Our calculator only supports binary. +A proper implementation needs per-system high-rate fractions. + +**When to pick up**: when you reach §12 / Table 12a. Implement +fractional blending for all the rows of Table 12a, not just cat=10. + +### Finding 4 — Lighting (Appendix L proper) +**Status**: gap. Current code uses a 9.3 kWh/m² heuristic with simple +LED/CFL reductions; spec is the L1-L12 cascade with daylight +correction, fixed-lighting capacity, top-up + portable shares, +monthly profile. + +**Spec basis**: SAP 10.2 Appendix L §L1 (pages 88-90), equations +L1-L12. + +**The bug**: for a 100 m² LED-dominant home (e.g. cert 7536-3827 with +51 LEDs), our heuristic returns 465 kWh/yr; spec returns ~94 kWh/yr. +Over-prediction by ~5× on LED-dominant homes (which is most modern +stock). + +**Empirical impact** (estimated): +- ~5-6 kWh/m² PEUI over-prediction for LED-dominant population +- Corpus-weighted: ~3-4 kWh/m² PEUI bias contribution + +**When to pick up**: when you reach Appendix L. Pair with the HW +cylinder fix (Finding 1) to avoid the SAP MAE regression. + +### Finding 5 — Internal-gains Table 5 missing rows +**Status**: gap. Spec Table 5 has 7 rows for internal gains; our +`worksheet/internal_gains.py` implements 4. + +**Spec basis**: SAP 10.2 Table 5 (page 177). + +**Missing rows**: +- **Water heating**: `1000 × (65)ₘ / (nₘ × 24)` W — the HW losses + (cylinder + distribution + primary) recycled as heated-space gains + via worksheet line (65). Reduces space heating demand. +- **Losses**: `−40 × N` W — heat to incoming cold water and + evaporation. Negative contribution. + +**Empirical impact** (estimated): +- For N=2.7: HW gains ≈+75 W, losses ≈−108 W, net ≈−33 W. Currently + we miss both → our gains are 33 W too high → space heating demand + too low → PE under-predicted by ~3 kWh/m² (rough). + +**When to pick up**: when you reach §5 / Table 5. Worksheet line (65) +also needs implementation — the HW losses already exist in our calc +(see `demand.py:_cylinder_storage_loss_kwh` etc.), they just need +piping into internal_gains. + +### Finding 6 — Storage-loss-factor table values are wrong +**Status**: gap. Affects only certs with `has_hot_water_cylinder=True` +(33% of corpus). + +**Spec basis**: SAP 10.2 Table 2 (page 158). + +**The bug**: `domain.ml.demand:_STORAGE_LOSS_FACTOR` values are ~3× +LOWER than spec. E.g. for 38mm foam our value is 0.0056, spec is +0.0181. Effect: we UNDER-predict cylinder storage loss by ~300 kWh +for storage systems, partly cancelling the over-prediction from +Finding 1. + +**When to pick up**: when you reach §4 / Table 2. Fix WITH Finding 1 +(combi zero-loss) so the cancellation doesn't dominate the +direction. + +### Finding 7 — Heat-pump fallback efficiency 2.30 +**Status**: gap that requires PCDB. See §8b. + +### Finding 8 — Other smaller gaps (carry forward) +- Boiler interlock −5% penalty (§9.2.1) — never applied +- Table 4c condensing boiler / HP emitter temperature adjustment — never applied +- Control-temperature adjustment from Table 4e — always 0 in code, spec varies +- Wall U-values for Scotland / Wales / NIR — only England fully transcribed +- Per-junction thermal bridging (Table R2) — global y approximation only +- Multi-main heating (`main_heating_fraction` ≠ 1) — first main only +- Cooling §10 — not implemented (rare in UK) +- FEE §11 — not implemented (new-build only) + +--- + ## 8. Don't repeat — known dead-ends - ❌ **Switching "NI" wall thickness to None alone** (S-B5 in history) —