Model/domain/sap10_calculator/docs/HANDOVER_POST_S0380_137.md

Handover — post Slices S0380.131..137

Branch: feature/per-cert-mapper-validation. HEAD 3542186f. Predecessor: HANDOVER_POST_S0380_130.md.

TL;DR

Seven slices landed on top of c8486077. The work spanned a fuel-price correction, a strict-raise on missing fuel that surfaced 26 corpus variants relying on a silent mains-gas default, a measurement-bug fix in the corpus's PE pin, and three slices closing per-cluster cascade gaps via SAP 10.2 Table 4a R-dispatch + a canonical electric-fuel classifier.

Slice	Commit	Scope
S0380.131	14eee259	Heating-oil price 7.64 → 5.44 (empirical, Elmhurst worksheet + cert 0240 back-solve)
S0380.132	0aa40b63	MissingMainFuelType strict-raise on empty main_fuel_type (26 corpus variants moved to _BLOCKED_BY_MISSING_MAIN_FUEL_TYPE)
S0380.133	0d2d41ab	Elmhurst §14.0 EES Code → Table 32 fuel code (BAF/BAI/RAM=anthracite, BCC=coal, BDI=dual, BKI=smokeless, BQI=wood chips, RPS=pellets bags, RUN=bulk, RWN=wood logs) — 10 solid-fuel variants unblocked
S0380.134	7530ed3f	Corpus PE pin compared against cert_to_demand_inputs (EPC block) instead of rating mode (rating block has no Total PE row)
S0380.135	829a3318	Table 4a R-dispatch in _responsiveness keyed on sap_main_heating_code (solid-fuel codes 151-161, 631-636)
S0380.136	4d004790	_is_electric_main / _is_electric_water route via canonical T32-first normaliser (table_32.is_electric_fuel_code) — closes solid fuel 6 dual-fuel SAP −11.37 → +1.95
S0380.137	3542186f	Table 4a R-dispatch extended to electric storage / UFH / Electricaire / direct-acting / ceiling (codes 401-409, 421-425, 515, 691, 694, 701)

Extended handover suite at HEAD: 880 pass, 0 fail.

What changed

Spec compliance (Table 4a + Table 32 + spec line 15271)

S0380.135 + S0380.137 implement SAP 10.2 spec line 15271:

"R = responsiveness of main heating system (Table 4a or Table 4d)"

Pre-slices the cascade only consulted Table 4d (emitter-based) — Table 4a's per-heating-system R (typically lower than 1.0 for non-modulating systems) was silently ignored. The new _RESPONSIVENESS_BY_SAP_CODE dispatch in cert_to_inputs.py overrides the Table 4d fallback when the SAP code is in the dict (31 entries covering all solid-fuel + electric storage / UFH / direct-acting / ceiling codes from Table 4a p.169-170).

S0380.131 corrected tables/table_32.py heating oil 7.64 → 5.44 (empirical, no spec citation possible — RdSAP 10 spec PDF p.95 is outlier vs Elmhurst worksheet + gov.uk register back-solve).

Strict-raise + canonical normalisation pattern

S0380.132 added MissingMainFuelType(ValueError) in exceptions.py. _main_fuel_code raises when the mapper leaves main_fuel_type empty / non-int. This surfaced 26 of 41 corpus variants relying on the silent mains-gas default.

S0380.136 promoted table_32._is_electric_code to public is_electric_fuel_code and routed _is_electric_main / _is_electric_water through it. Closed an API/Table-32 code-10 collision (API 10 = electricity, T32 10 = dual fuel) that re-routed solid fuel 6's cost through off-peak electric tariff.

Mapper extraction extensions

S0380.133 added main_heating_ees: str field to elmhurst_site_notes.py:MainHeating and extraction in elmhurst_extractor.py, plus _ELMHURST_MAIN_HEATING_EES_TO_FUEL_CODE dict in mapper.py (10 entries keyed on 3-letter EES code).

Corpus test structure

test_heating_systems_corpus.py now has three tiers:

1. _EXPECTATIONS (25 variants) — full residual-pin grid: SAP / cost / CO2 from cert_to_inputs (rating block), PE from cert_to_demand_inputs (EPC block).
2. _BLOCKED_BY_MISSING_MAIN_FUEL_TYPE (16 variants) — assert-on-raise tier driving test_heating_systems_corpus_blocked_variant_raises_missing_main_fuel_type.
3. Each variant covered exactly once across the two tiers (41 total).

Current residual cluster at HEAD 3542186f

Solid fuel — 10/10 unblocked, tight cluster

variant	SAP code	R	ΔSAP	ΔPE
solid fuel 2	158	0.50	+2.64	-1211
solid fuel 3	160	0.50	+1.32	-935
solid fuel 4	633	0.50	+1.59	+151
solid fuel 5	153	0.75	+1.70	+160
solid fuel 6	160	0.50	+1.95	+87
solid fuel 7	160	0.50	+2.04	+44
solid fuel 8	160	0.50	+1.81	+88
solid fuel 9	636	0.75	+1.71	+155
solid fuel 10	634	0.50	+1.75	+120
solid fuel 11	634	0.50	+1.62	+171

7/10 PE residuals within ±220 kWh. SAP cluster all +1.32 to +2.64. solid fuel 2 (-1211 PE) + solid fuel 3 (-935 PE) are the remaining outliers — likely Table 4a efficiency variant or kWh-totals issue.

Electric direct-acting — 6/7 unblocked, +5..+9 SAP cluster open

variant	SAP code	R	ΔSAP	Δcost	ΔPE
electric 1	191	1.00	+9.64	−£222	+165
electric 2	524	1.00	+5.85	−£135	+971
electric 3	401	0.00	+9.43	−£217	-1059
electric 5	402	0.20	+6.76	−£156	-96
electric 6	404	0.40	+7.82	−£180	-494
electric 7	408	0.60	+7.58	−£175	-428
electric 8	409	0.80	+5.84	−£135	+200
electric 9	421	0.00	+6.77	−£156	+154

Shared pattern across all 7: SAP +5.8..+9.6 with cost −£135..−£222. Consistent cost under-count strongly suggests a single Table 12a high/low-rate fraction handling bug OR a pumps/fans electric cascade gap. Same "one fix many variants" leverage pattern as previous slices.

Other cascade-OK variants

variant	ΔSAP	ΔPE	notes
ashp	+5.67	-12	✓ PE closed
gshp	+5.16	-455
oil 1	+2.66	-1050
oil pcdb 1/2	+0.42	-84	✓ basically closed
oil pcdb 3	+1.16	-271
pcdb 1	+6.95	-3135	largest open PE

Blocked tier (16 variants in _BLOCKED_BY_MISSING_MAIN_FUEL_TYPE)

Category	Variants	SAP code(s)	EES code(s)	Likely fix
Community heating	1, 2, 3, 4, 6	301-304	COM (all share)	Derive fuel from §14.1 Community Heating block
Electric storage	11, 12, 13, 14	515, 691, 701	WEA, REA, OEA	Extend _ELMHURST_MAIN_HEATING_EES_TO_FUEL_CODE to electric EES codes
No system	(1)	699	NON	Spec assumed electric heaters
Liquid-fuel non-oil	oil 2-6	Table 4b 126-141	BFD, BXE, BXF, BZC, B3C	Extend §15.0 fallback / mapper dict for HVO / FAME / B30K / bioethanol
PCDB Bulk LPG	pcdb 3	(PCDB)	(absent)	Add "Bulk LPG" → 2 to _ELMHURST_MAIN_FUEL_TO_SAP10

Test baseline at HEAD 3542186f

PYTHONPATH=/workspaces/model python -m pytest \
    backend/documents_parser/tests/test_summary_pdf_mapper_chain.py \
    backend/documents_parser/tests/test_heating_systems_corpus.py \
    backend/documents_parser/tests/test_elmhurst_extractor.py \
    backend/documents_parser/tests/test_elmhurst_end_to_end.py \
    domain/sap10_calculator/worksheet/tests/test_e2e_elmhurst_sap_score.py \
    domain/sap10_calculator/worksheet/tests/test_heat_transmission.py \
    domain/sap10_calculator/worksheet/tests/test_internal_gains.py \
    domain/sap10_calculator/worksheet/tests/test_solar_gains.py \
    domain/sap10_calculator/worksheet/tests/test_dimensions.py \
    domain/sap10_calculator/worksheet/tests/test_rating.py \
    domain/sap10_calculator/worksheet/tests/test_ventilation.py \
    domain/sap10_calculator/worksheet/tests/test_appendix_h_solar.py \
    domain/sap10_calculator/worksheet/tests/test_mev.py \
    domain/sap10_calculator/rdsap/tests/test_cert_to_inputs.py \
    domain/sap10_calculator/rdsap/tests/test_golden_fixtures.py \
    domain/sap10_calculator/tests/test_pcdb_table_322_lookup.py \
    domain/sap10_calculator/tests/test_pcdb_table_329_lookup.py \
    domain/sap10_calculator/tests/test_table_12a.py \
    --no-cov -q

Expected: 880 pass, 0 fail.

Memories to load (in order)

project-heating-systems-corpus    # full state at HEAD 3542186f
feedback-sap-10-2-only-never-10-3 # CRITICAL — never reference SAP 10.3
feedback-worksheet-not-api-reference
feedback-spec-citation-in-commits
feedback-verify-handover-claims
feedback-zero-error-strict
feedback-commit-per-slice
feedback-aaa-test-convention
feedback-e2e-validation-philosophy
feedback-abs-diff-over-pytest-approx
feedback-spec-floor-skepticism
feedback-golden-residuals-near-zero
feedback-one-e-minus-4-across-the-board
reference-unmapped-sap-code       # updated S0380.135 + S0380.137
reference-unmapped-api-code
project-oil-price-spec-divergence # S0380.131 detail

Next-slice candidates (in priority order)

1. Electric +5..+9 SAP cluster — highest leverage

7 electric corpus variants share +5.8..+9.6 SAP and −£135..−£222 cost under-count. Pattern strongly suggests one shared cascade gap. Likely candidates:

• Table 12a high/low-rate fraction for electric main heating — the cascade applies tariff splits per space_heating_high_rate_fraction, but the worksheet may use a different fraction or skip the split.
• Pumps/fans kWh / cost — cascade reports 130 kWh/yr; worksheet reports 41 kWh/yr. Cascade over-counts by 89 kWh × electric ~13 p/kWh = ~£12 — small, not the dominant cost gap.
• Cost factor cascading — for electric main on 18-hour tariff, the cascade uses 5.50 p/kWh (off-peak low rate). The worksheet uses... need to probe.

Probing one variant (electric 3, the worst at +9.43 SAP / -£217 cost) would identify the shared cause. If a single Table 12a / tariff fix closes most of the 7, that's a high-value slice.

2. Unblock community heating cluster

5 community heating variants all share EES Code: COM (no fuel info in the EES code). The fuel must be derived from the §14.1 Community Heating/Heat Network block which lodges the heat source type (gas boiler / CHP / heat pump / etc.). Each maps to a Table 32 heat-network code (51-58, 41-49).

Implementation pattern: extend the extractor to capture §14.1 community heat source, add a SAP-code-301-304 → community-heating-fuel dispatch in the mapper.

3. Unblock electric storage variants (11, 12, 13, 14)

4 electric corpus variants blocked because mapper has no fuel. SAP codes 515 (Electricaire), 691 (Panel heaters), 701 (Electric ceiling) imply electric. Extend _ELMHURST_MAIN_HEATING_EES_TO_FUEL_CODE:

EES	Variant	Fuel
WEA	electric 11 (SAP 515)	30 (standard electric)
REA	electric 12 (SAP 691)	30
OEA	electric 13/14 (SAP 701)	30

Or alternative: gate on sap_main_heating_code in {191, 401-409, 421-425, 515, 691, 694, 701} and infer electric — broader pattern.

4. solid fuel 2 / 3 PE residuals (-935 to -1211)

After R-dispatch closed 7/10 solid-fuel PE residuals, 2 remain at ~-1000 PE. Both are anthracite (codes 158, 160). Same fuel and same R as other variants that closed. Possible:

• Table 4a efficiency variant (winter/summer split)
• Secondary heating fraction (Table 11) not applying

5. pcdb 1 PE residual −3135

Oil PCDB-listed boiler cert (no SAP code, PCDB index drives lookup). Largest open PE residual. Separate cause from R-dispatch.

6. Tariff-dependent R promotion

Codes 402/403/405 have R=0.20/0.40 off-peak vs R=0.40/0.60 24-hour tariff per Table 4a. Current dict uses off-peak default (corpus is all off-peak). If a 24-hour cert ever surfaces, promote _RESPONSIVENESS_BY_SAP_CODE from dict[int, float] to dict[(int, Tariff), float] lookup.

7. Latent strict-raise opportunity

table_32.is_electric_fuel_code / _is_gas_code silently return False for unmapped fuel codes. User raised this in S0380.136 discussion as a follow-up forcing-function pattern (same shape as MissingMainFuelType). Broad blast radius — defer until after the visible-residual closures.

Spec source quick-reference

All under domain/sap10_calculator/docs/specs/:

• SAP 10.2 full spec: sap-10-2-full-specification-2025-03-14.pdf
  • Spec line 15271 (R = responsiveness ... Table 4a or Table 4d)
  • Table 4a (p.163-170) — heating systems with R column
  • Table 4b (p.170-171) — gas / liquid fuel boilers
  • Table 4d (p.170) — heat emitter R
  • Table 4e (p.171-174) — control codes
  • Table 9 / 9a / 9b — heating duration + MIT formulas (where R enters the MIT adjustment)
  • Table 12 (p.191) — SAP rating fuel prices (regulated tariff)
  • Table 12a — high/low-rate fraction by system × tariff
• RdSAP 10 spec: RdSAP 10 Specification 10-06-2025.pdf
  • §19 Table 32 (p.95) — RdSAP10 fuel prices / CO2 / PE
  • Heating oil price 7.64 in spec but 5.44 empirically (per S0380.131)
• BRE technical papers at sap10 technical papers/ — no Table 32 errata
• SAP 10.3 at sap-10-3-full-specification-2026-01-13.pdf: DO NOT reference (per feedback-sap-10-2-only-never-10-3)

Workflow per slice

1. Read spec page + identify rule
2. Probe cascade vs worksheet line-by-line for one variant in the cluster; verify the diagnosis closes the residual via monkey-patch
3. Write failing AAA test (literal # Arrange / # Act / # Assert)
4. Implement helper / dispatch entry / mapper extension
5. Verify test passes
6. Probe full cluster impact + re-pin affected variants
7. Run extended handover suite (command above)
8. Pyright net-zero check on touched files (git stash → pyright → git stash pop → pyright)
9. Commit with spec citation + Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
10. Update project-heating-systems-corpus + MEMORY.md index

What NOT to do

• Don't reference SAP 10.3 — track 10.2 deliberately
• Don't widen pin tolerances to make pins pass — re-pin smaller or find the spec gap
• Don't re-investigate closed work (Slices .91..137 all settled)
• Don't add new helpers to domain/sap10_ml/ — on the deprecation path
• Don't conflate the R-dispatch with the cost cluster — R closes PE (via demand), the +5..+9 SAP residual on electrics is the cost-side gap, a separate issue
• Don't accept "spec-precision floor" framing without spec-citation work — verify against worksheet PDF + cross-cert empirical evidence

User direction at end of session

The conversation flowed:

1. Started on solid fuel 8 +0.87 ΔSAP — discovered it was a compensating- errors illusion (real CO2 Δ +3525)
2. User: "could we add an exception in the calculator that an empty fuel type can't be given?" → S0380.132 strict-raise
3. User: "I'm okay with breaking the tests if that means not debugging silent, incorrect fallbacks"
4. Suggested SAP-code → fuel derivation → S0380.133 solid-fuel EES dispatch
5. User asked for audit of remaining patterns → found PE measurement bug + per-cluster issues → S0380.134 (PE pin fix) → S0380.135 (R dispatch solid fuel)
6. User: "is this another opportunity to raise an exception?" during S0380.136 — answered: this is a different bug class (type ambiguity, not missing data); broader raise opportunity exists for is_electric_fuel_code / _is_gas_code silent-False on unmapped (catalogued as #7 above)
7. S0380.137 extended R-dispatch to electric

The "find ONE fix that closes MULTIPLE variants" framing is the user's preferred approach. Each slice closed 6-10 variants via a single table-dispatch or convention-routing change.

Good luck.