# Handover — post Slices S0380.131..137 Branch: `feature/per-cert-mapper-validation`. **HEAD `3542186f`**. Predecessor: [`HANDOVER_POST_S0380_130.md`](HANDOVER_POST_S0380_130.md). ## TL;DR Seven slices landed on top of `c8486077`. The work spanned a fuel-price correction, a strict-raise on missing fuel that surfaced 26 corpus variants relying on a silent mains-gas default, a measurement-bug fix in the corpus's PE pin, and three slices closing per-cluster cascade gaps via SAP 10.2 Table 4a R-dispatch + a canonical electric-fuel classifier. | Slice | Commit | Scope | |---|---|---| | **S0380.131** | `14eee259` | Heating-oil price 7.64 → 5.44 (empirical, Elmhurst worksheet + cert 0240 back-solve) | | **S0380.132** | `0aa40b63` | `MissingMainFuelType` strict-raise on empty `main_fuel_type` (26 corpus variants moved to `_BLOCKED_BY_MISSING_MAIN_FUEL_TYPE`) | | **S0380.133** | `0d2d41ab` | Elmhurst §14.0 EES Code → Table 32 fuel code (BAF/BAI/RAM=anthracite, BCC=coal, BDI=dual, BKI=smokeless, BQI=wood chips, RPS=pellets bags, RUN=bulk, RWN=wood logs) — 10 solid-fuel variants unblocked | | **S0380.134** | `7530ed3f` | Corpus PE pin compared against `cert_to_demand_inputs` (EPC block) instead of rating mode (rating block has no Total PE row) | | **S0380.135** | `829a3318` | Table 4a R-dispatch in `_responsiveness` keyed on `sap_main_heating_code` (solid-fuel codes 151-161, 631-636) | | **S0380.136** | `4d004790` | `_is_electric_main` / `_is_electric_water` route via canonical T32-first normaliser (`table_32.is_electric_fuel_code`) — closes solid fuel 6 dual-fuel SAP −11.37 → +1.95 | | **S0380.137** | `3542186f` | Table 4a R-dispatch extended to electric storage / UFH / Electricaire / direct-acting / ceiling (codes 401-409, 421-425, 515, 691, 694, 701) | Extended handover suite at HEAD: **880 pass, 0 fail**. ## What changed ### Spec compliance (Table 4a + Table 32 + spec line 15271) S0380.135 + S0380.137 implement SAP 10.2 spec line 15271: > "R = responsiveness of main heating system (Table 4a or Table 4d)" Pre-slices the cascade only consulted Table 4d (emitter-based) — Table 4a's per-heating-system R (typically lower than 1.0 for non-modulating systems) was silently ignored. The new `_RESPONSIVENESS_BY_SAP_CODE` dispatch in [`cert_to_inputs.py`](../rdsap/cert_to_inputs.py) overrides the Table 4d fallback when the SAP code is in the dict (31 entries covering all solid-fuel + electric storage / UFH / direct-acting / ceiling codes from Table 4a p.169-170). S0380.131 corrected `tables/table_32.py` heating oil 7.64 → 5.44 (empirical, no spec citation possible — RdSAP 10 spec PDF p.95 is outlier vs Elmhurst worksheet + gov.uk register back-solve). ### Strict-raise + canonical normalisation pattern S0380.132 added `MissingMainFuelType(ValueError)` in [`exceptions.py`](../exceptions.py). `_main_fuel_code` raises when the mapper leaves `main_fuel_type` empty / non-int. This surfaced 26 of 41 corpus variants relying on the silent mains-gas default. S0380.136 promoted `table_32._is_electric_code` to public `is_electric_fuel_code` and routed `_is_electric_main` / `_is_electric_water` through it. Closed an API/Table-32 code-10 collision (API 10 = electricity, T32 10 = dual fuel) that re-routed solid fuel 6's cost through off-peak electric tariff. ### Mapper extraction extensions S0380.133 added `main_heating_ees: str` field to [`elmhurst_site_notes.py:MainHeating`](../../../datatypes/epc/surveys/elmhurst_site_notes.py) and extraction in [`elmhurst_extractor.py`](../../../backend/documents_parser/elmhurst_extractor.py), plus `_ELMHURST_MAIN_HEATING_EES_TO_FUEL_CODE` dict in [`mapper.py`](../../../datatypes/epc/domain/mapper.py) (10 entries keyed on 3-letter EES code). ### Corpus test structure [`test_heating_systems_corpus.py`](../../../backend/documents_parser/tests/test_heating_systems_corpus.py) now has three tiers: 1. `_EXPECTATIONS` (25 variants) — full residual-pin grid: SAP / cost / CO2 from `cert_to_inputs` (rating block), PE from `cert_to_demand_inputs` (EPC block). 2. `_BLOCKED_BY_MISSING_MAIN_FUEL_TYPE` (16 variants) — assert-on-raise tier driving `test_heating_systems_corpus_blocked_variant_raises_missing_main_fuel_type`. 3. Each variant covered exactly once across the two tiers (41 total). ## Current residual cluster at HEAD `3542186f` ### Solid fuel — 10/10 unblocked, tight cluster | variant | SAP code | R | ΔSAP | ΔPE | |---|---:|---:|---:|---:| | solid fuel 2 | 158 | 0.50 | +2.64 | -1211 | | solid fuel 3 | 160 | 0.50 | +1.32 | -935 | | solid fuel 4 | 633 | 0.50 | +1.59 | +151 | | solid fuel 5 | 153 | 0.75 | +1.70 | +160 | | solid fuel 6 | 160 | 0.50 | +1.95 | +87 | | solid fuel 7 | 160 | 0.50 | +2.04 | +44 | | solid fuel 8 | 160 | 0.50 | +1.81 | +88 | | solid fuel 9 | 636 | 0.75 | +1.71 | +155 | | solid fuel 10 | 634 | 0.50 | +1.75 | +120 | | solid fuel 11 | 634 | 0.50 | +1.62 | +171 | 7/10 PE residuals within ±220 kWh. SAP cluster all +1.32 to +2.64. solid fuel 2 (-1211 PE) + solid fuel 3 (-935 PE) are the remaining outliers — likely Table 4a efficiency variant or kWh-totals issue. ### Electric direct-acting — 6/7 unblocked, +5..+9 SAP cluster open | variant | SAP code | R | ΔSAP | Δcost | ΔPE | |---|---:|---:|---:|---:|---:| | electric 1 | 191 | 1.00 | +9.64 | −£222 | +165 | | electric 2 | 524 | 1.00 | +5.85 | −£135 | +971 | | electric 3 | 401 | 0.00 | +9.43 | −£217 | -1059 | | electric 5 | 402 | 0.20 | +6.76 | −£156 | -96 | | electric 6 | 404 | 0.40 | +7.82 | −£180 | -494 | | electric 7 | 408 | 0.60 | +7.58 | −£175 | -428 | | electric 8 | 409 | 0.80 | +5.84 | −£135 | +200 | | electric 9 | 421 | 0.00 | +6.77 | −£156 | +154 | **Shared pattern across all 7:** SAP +5.8..+9.6 with cost −£135..−£222. Consistent cost under-count strongly suggests a single Table 12a high/low-rate fraction handling bug OR a pumps/fans electric cascade gap. Same "one fix many variants" leverage pattern as previous slices. ### Other cascade-OK variants | variant | ΔSAP | ΔPE | notes | |---|---:|---:|---| | ashp | +5.67 | -12 | ✓ PE closed | | gshp | +5.16 | -455 | | | oil 1 | +2.66 | -1050 | | | oil pcdb 1/2 | +0.42 | -84 | ✓ basically closed | | oil pcdb 3 | +1.16 | -271 | | | pcdb 1 | +6.95 | -3135 | largest open PE | ### Blocked tier (16 variants in `_BLOCKED_BY_MISSING_MAIN_FUEL_TYPE`) | Category | Variants | SAP code(s) | EES code(s) | Likely fix | |---|---|---|---|---| | Community heating | 1, 2, 3, 4, 6 | 301-304 | COM (all share) | Derive fuel from §14.1 Community Heating block | | Electric storage | 11, 12, 13, 14 | 515, 691, 701 | WEA, REA, OEA | Extend `_ELMHURST_MAIN_HEATING_EES_TO_FUEL_CODE` to electric EES codes | | No system | (1) | 699 | NON | Spec assumed electric heaters | | Liquid-fuel non-oil | oil 2-6 | Table 4b 126-141 | BFD, BXE, BXF, BZC, B3C | Extend §15.0 fallback / mapper dict for HVO / FAME / B30K / bioethanol | | PCDB Bulk LPG | pcdb 3 | (PCDB) | (absent) | Add `"Bulk LPG"` → 2 to `_ELMHURST_MAIN_FUEL_TO_SAP10` | ## Test baseline at HEAD `3542186f` ```bash PYTHONPATH=/workspaces/model python -m pytest \ backend/documents_parser/tests/test_summary_pdf_mapper_chain.py \ backend/documents_parser/tests/test_heating_systems_corpus.py \ backend/documents_parser/tests/test_elmhurst_extractor.py \ backend/documents_parser/tests/test_elmhurst_end_to_end.py \ domain/sap10_calculator/worksheet/tests/test_e2e_elmhurst_sap_score.py \ domain/sap10_calculator/worksheet/tests/test_heat_transmission.py \ domain/sap10_calculator/worksheet/tests/test_internal_gains.py \ domain/sap10_calculator/worksheet/tests/test_solar_gains.py \ domain/sap10_calculator/worksheet/tests/test_dimensions.py \ domain/sap10_calculator/worksheet/tests/test_rating.py \ domain/sap10_calculator/worksheet/tests/test_ventilation.py \ domain/sap10_calculator/worksheet/tests/test_appendix_h_solar.py \ domain/sap10_calculator/worksheet/tests/test_mev.py \ domain/sap10_calculator/rdsap/tests/test_cert_to_inputs.py \ domain/sap10_calculator/rdsap/tests/test_golden_fixtures.py \ domain/sap10_calculator/tests/test_pcdb_table_322_lookup.py \ domain/sap10_calculator/tests/test_pcdb_table_329_lookup.py \ domain/sap10_calculator/tests/test_table_12a.py \ --no-cov -q ``` Expected: **880 pass, 0 fail**. ## Memories to load (in order) ``` project-heating-systems-corpus # full state at HEAD 3542186f feedback-sap-10-2-only-never-10-3 # CRITICAL — never reference SAP 10.3 feedback-worksheet-not-api-reference feedback-spec-citation-in-commits feedback-verify-handover-claims feedback-zero-error-strict feedback-commit-per-slice feedback-aaa-test-convention feedback-e2e-validation-philosophy feedback-abs-diff-over-pytest-approx feedback-spec-floor-skepticism feedback-golden-residuals-near-zero feedback-one-e-minus-4-across-the-board reference-unmapped-sap-code # updated S0380.135 + S0380.137 reference-unmapped-api-code project-oil-price-spec-divergence # S0380.131 detail ``` ## Next-slice candidates (in priority order) ### 1. Electric +5..+9 SAP cluster — highest leverage 7 electric corpus variants share +5.8..+9.6 SAP and −£135..−£222 cost under-count. Pattern strongly suggests one shared cascade gap. Likely candidates: - **Table 12a high/low-rate fraction** for electric main heating — the cascade applies tariff splits per `space_heating_high_rate_fraction`, but the worksheet may use a different fraction or skip the split. - **Pumps/fans kWh / cost** — cascade reports 130 kWh/yr; worksheet reports 41 kWh/yr. Cascade over-counts by 89 kWh × electric ~13 p/kWh = ~£12 — small, not the dominant cost gap. - **Cost factor cascading** — for electric main on 18-hour tariff, the cascade uses 5.50 p/kWh (off-peak low rate). The worksheet uses... need to probe. Probing one variant (electric 3, the worst at +9.43 SAP / -£217 cost) would identify the shared cause. If a single Table 12a / tariff fix closes most of the 7, that's a high-value slice. ### 2. Unblock community heating cluster 5 community heating variants all share `EES Code: COM` (no fuel info in the EES code). The fuel must be derived from the §14.1 Community Heating/Heat Network block which lodges the heat source type (gas boiler / CHP / heat pump / etc.). Each maps to a Table 32 heat-network code (51-58, 41-49). Implementation pattern: extend the extractor to capture §14.1 community heat source, add a SAP-code-301-304 → community-heating-fuel dispatch in the mapper. ### 3. Unblock electric storage variants (11, 12, 13, 14) 4 electric corpus variants blocked because mapper has no fuel. SAP codes 515 (Electricaire), 691 (Panel heaters), 701 (Electric ceiling) imply electric. Extend `_ELMHURST_MAIN_HEATING_EES_TO_FUEL_CODE`: | EES | Variant | Fuel | |---|---|---| | WEA | electric 11 (SAP 515) | 30 (standard electric) | | REA | electric 12 (SAP 691) | 30 | | OEA | electric 13/14 (SAP 701) | 30 | Or alternative: gate on `sap_main_heating_code in {191, 401-409, 421-425, 515, 691, 694, 701}` and infer electric — broader pattern. ### 4. solid fuel 2 / 3 PE residuals (-935 to -1211) After R-dispatch closed 7/10 solid-fuel PE residuals, 2 remain at ~-1000 PE. Both are anthracite (codes 158, 160). Same fuel and same R as other variants that closed. Possible: - Table 4a efficiency variant (winter/summer split) - Secondary heating fraction (Table 11) not applying ### 5. pcdb 1 PE residual −3135 Oil PCDB-listed boiler cert (no SAP code, PCDB index drives lookup). Largest open PE residual. Separate cause from R-dispatch. ### 6. Tariff-dependent R promotion Codes 402/403/405 have R=0.20/0.40 off-peak vs R=0.40/0.60 24-hour tariff per Table 4a. Current dict uses off-peak default (corpus is all off-peak). If a 24-hour cert ever surfaces, promote `_RESPONSIVENESS_BY_SAP_CODE` from `dict[int, float]` to `dict[(int, Tariff), float]` lookup. ### 7. Latent strict-raise opportunity `table_32.is_electric_fuel_code` / `_is_gas_code` silently return False for unmapped fuel codes. User raised this in S0380.136 discussion as a follow-up forcing-function pattern (same shape as `MissingMainFuelType`). Broad blast radius — defer until after the visible-residual closures. ## Spec source quick-reference All under `domain/sap10_calculator/docs/specs/`: - **SAP 10.2 full spec**: `sap-10-2-full-specification-2025-03-14.pdf` - **Spec line 15271** (R = responsiveness ... Table 4a or Table 4d) - **Table 4a** (p.163-170) — heating systems with R column - **Table 4b** (p.170-171) — gas / liquid fuel boilers - **Table 4d** (p.170) — heat emitter R - **Table 4e** (p.171-174) — control codes - **Table 9 / 9a / 9b** — heating duration + MIT formulas (where R enters the MIT adjustment) - **Table 12** (p.191) — SAP rating fuel prices (regulated tariff) - **Table 12a** — high/low-rate fraction by system × tariff - **RdSAP 10 spec**: `RdSAP 10 Specification 10-06-2025.pdf` - **§19 Table 32** (p.95) — RdSAP10 fuel prices / CO2 / PE - Heating oil price 7.64 in spec but 5.44 empirically (per S0380.131) - **BRE technical papers** at `sap10 technical papers/` — no Table 32 errata - **SAP 10.3** at `sap-10-3-full-specification-2026-01-13.pdf`: **DO NOT reference** (per [[feedback-sap-10-2-only-never-10-3]]) ## Workflow per slice 1. Read spec page + identify rule 2. Probe cascade vs worksheet line-by-line for one variant in the cluster; verify the diagnosis closes the residual via monkey-patch 3. Write failing AAA test (literal `# Arrange / # Act / # Assert`) 4. Implement helper / dispatch entry / mapper extension 5. Verify test passes 6. Probe full cluster impact + re-pin affected variants 7. Run extended handover suite (command above) 8. Pyright net-zero check on touched files (`git stash` → pyright → `git stash pop` → pyright) 9. Commit with spec citation + `Co-Authored-By: Claude Opus 4.7 ` 10. Update `project-heating-systems-corpus` + `MEMORY.md` index ## What NOT to do - **Don't reference SAP 10.3** — track 10.2 deliberately - **Don't widen pin tolerances** to make pins pass — re-pin smaller or find the spec gap - **Don't re-investigate closed work** (Slices .91..137 all settled) - **Don't add new helpers to `domain/sap10_ml/`** — on the deprecation path - **Don't conflate the R-dispatch with the cost cluster** — R closes PE (via demand), the +5..+9 SAP residual on electrics is the *cost-side* gap, a separate issue - **Don't accept "spec-precision floor" framing** without spec-citation work — verify against worksheet PDF + cross-cert empirical evidence ## User direction at end of session The conversation flowed: 1. Started on solid fuel 8 +0.87 ΔSAP — discovered it was a compensating- errors illusion (real CO2 Δ +3525) 2. User: "could we add an exception in the calculator that an empty fuel type can't be given?" → S0380.132 strict-raise 3. User: "I'm okay with breaking the tests if that means not debugging silent, incorrect fallbacks" 4. Suggested SAP-code → fuel derivation → S0380.133 solid-fuel EES dispatch 5. User asked for audit of remaining patterns → found PE measurement bug + per-cluster issues → S0380.134 (PE pin fix) → S0380.135 (R dispatch solid fuel) 6. User: "is this another opportunity to raise an exception?" during S0380.136 — answered: this is a *different bug class* (type ambiguity, not missing data); broader raise opportunity exists for `is_electric_fuel_code` / `_is_gas_code` silent-False on unmapped (catalogued as #7 above) 7. S0380.137 extended R-dispatch to electric The "find ONE fix that closes MULTIPLE variants" framing is the user's preferred approach. Each slice closed 6-10 variants via a single table-dispatch or convention-routing change. Good luck.