Model/domain/sap10_calculator/docs/HANDOVER_MAPPER_BUGS.md

Handover — golden-cert mapper/cascade bugs (post-0240 wall fix)

Point-in-time note. Start from AGENT_GUIDE.md for methodology, the 1e-4 bar, the per-line debugging loop, and the suite command. This records the state after closing the 0240 investigation and fixing the first of several API-mapper/cascade bugs the audit surfaced.

• Branch: feature/per-cert-mapper-validation
• HEAD: b9bbcecb (docs after S0380.213). Confirm with git rev-parse HEAD.
• Baseline: 2386 passed, 1 skipped, 0 failed (AGENT_GUIDE §4 suite command). ALSO run domain/sap10_ml/tests/ when touching rdsap_uvalues.py — 2 PRE-EXISTING stone-formula failures there, see Thread 1.
• Next slice number: S0380.214.

---

★ CURRENT PRIORITY — drive golden-fixture SAP residuals to ZERO

_EXPECTATIONS in test_golden_fixtures.py holds 53 pinned golden certs. The suite is GREEN. Only 3 have a non-zero SAP integer residual, and they split into two kinds:

cert	lodged	cont SAP	resid	nature (from the cert's notes:)
7536-3827-0600	68	69.071	+1	+0.57 over — multi-age bps (Main D / Ext1 L / Ext2 F); glazing U (S0380.97 set glazing_type=2 → Table 24 spec U=2.0, but the cert's lodged U "appears higher than the spec default")
2130-1033-4050	82	83.349	+1	+0.85 over — end-terrace + 1 ext, gas combi PCDB 17505, 2× PV arrays; SAP +1 came from the cohort PV-β cascade interaction (S0380.45/.49), not a pinpointed fabric line; PE residual −8.22 sits in gas-combi PE + secondary credit

These are TWO DIFFERENT root causes — not a shared one (an earlier audit label calling both "multi-part wall" was wrong; trust the notes: above).

• 7536 is the more tractable: a clear glazing-U hypothesis. S0380.97 forced glazing_type=2 to the Table 24 default U=2.0; the note suspects the cert's true per-bp glazing U is higher (multi-age D/L/F geometry). Walk §3 window (27) per-bp vs the lodged register window rating; the lever is likely the glazing U for one of the extensions. ASK the user for a simulated Elmhurst worksheet mirroring 7536's glazing (double-glazed, multi-age bps) to pin the true (27) U rather than guess.
• 2130 is harder: the SAP +1 is a PV-β / cohort cascade interaction, not a single fabric line. Its PE residual (−8.22) is a documented gas-combi-PE + secondary-credit deferral. Decompose which metric drives the integer flip (cost/EI vs PE) before touching anything; this one may need the PV/secondary path, not fabric.

Both certs are API-only (no worksheet) → bar is ±0.5 SAP vs lodged; the goal is the integer flip (69→68 / 83→82), i.e. shave ~0.57 / ~0.85 off the continuous SAP. Per the session methodology lesson, ASK for a worksheet rather than guess a U-value or factor.

0240 (−1) is NOT driveable from the JSON and the user has decided NOT to re-pin it — document the cause only. Continuous SAP is 72.462; the true SAP is 72. The lodged 73 requires a "2013 or later" circulation pump (41 kWh); 0240's open-data API lodges central_heating_pump_age=0 = Unknown → 115 kWh. The encoding was proven across 13 API+Summary pairs (0=Unknown, 2=2013+). The export did not preserve the pump age that produced the lodged 73, so 73 is unreachable without inventing data. Both fabric bugs that masked it are now fixed (wall S0380.209 + roof S0380.211 → cont 72.462). Leave the pin at actual_sap=73, expected_sap_resid=-1; the notes already record this. Driving it to zero would mean fudging the pump age — don't.

Latent (lower priority): 9390 (community, −2, unpinned/retired) ~7% demand over-count — see Thread 2.

---

What this session shipped

slice	what
S0380.208	Promoted simulated case 7 (combi swap of case 6) to an e2e fixture. PROVED the condensing-oil-combi (SAP code 130, no cylinder, combi instantaneous DHW, Eq D1, Table 3a keep-hot) path is exact at 1e-4 with zero calculator changes → exonerated the heating as the source of 0240's residual.
S0380.209	Fixed the API-path wall U "as built, insulated (assumed)" bug — routes to the as-built age-band row, not the 50 mm retrofit bucket. New _described_as_retrofit_insulated in heat_transmission.py. Worksheet-validated by case 9 (sandstone J → 0.35) + case 10 (solid brick J → 0.35). Re-pinned 0240 PE +1.8687 → +5.5044, CO2 +0.0907 → +0.2757 (SAP integer 72 unchanged).
S0380.210	CLOSED cert 0390 (Thread 3). Cavity wall "as built, partial insulation (assumed)" (type 4) was mis-routed to the Table 6 "Filled cavity" row (band F 0.40) → should be "Cavity as built" (band F 1.0). New _cavity_described_as_filled in rdsap_uvalues.py excludes "partial insulation" from the filled trigger (keeps "insulated (assumed)" → filled). SAP +7 → +0, PE −27.97 → +0.53, CO2 −2.71 → −0.12. Mirrors S0380.209 on the cavity path.
S0380.211	CLOSED Thread 1 (roof). 0240 Ext1 vaulted (code 5) NI roof returned 0.68 (§5.11.4 50 mm) → should be Table 18 col (1) age-band (band J 0.16), matching 33 cohort-2 ND vaulted roofs. New u_roof(is_sloping_ceiling=...) flag threaded from heat_transmission (codes 5/8). 0240 PE +5.50 → +1.52, CO2 +0.28 → +0.07 (SAP 72). Also corrected the S0380.210 cavity unit test in domain/sap10_ml/tests/ (suite-command gap — see Thread 1).
S0380.212	Thread 2 CO2/PE collision FIXED. EPC fuel 20 = "mains gas (community)" collided with Table-12 biomass code 20 → community CO2 6.4× low. New _heat_network_factor_fuel_code translates 20→51 for heat-network mains only (5 sites: SH+HW CO2/PE/price). 9390 CO2 0.44→3.03 t (lodged 2.8), PE 204→220. Case-14-validated ((367) 0.2100 / (467) 1.1300). Cost +4 tail open.
S0380.213	Thread 2 cost +4 FIXED via the heat-network standing charge. API community fuel 20 isn't a Table-32 gas code → _is_gas_code False → £0 standing (vs SAP 10.2 Table 12 note (l) £120; case 14 (351)=£120). New _heat_network_standing_charge_gbp (£120 full / £60 DHW-only, §C3.2) REPLACES the fuel standing for heat-network mains (no double-count; CH corpus stays £120). 9390 SAP +4 → -2 (exposes a ~7% demand over-count — follow-up).

Both also carry a memory: project_case7_combi_exonerated, project_as_built_insulated_assumed_bug.

---

The 0240 verdict — RESOLVED, not closable to 73

0240's true SAP is 72, proven three independent ways:

• Elmhurst case 8 (pump=Unknown, 0240's actual lodged value) → worksheet SAP 72.
• Elmhurst case 9 (correct sandstone wall 0.35 + sloping roof 0.25) → SAP 72.
• Our cascade with both the wall (done) and roof (pending) bugs fixed → SAP cont 72.31.

The register's 73 requires a "2013 or later" circulation pump (41 kWh) — Elmhurst case 7 with that pump = 73. But 0240's API lodges central_heating_pump_age=0 = Unknown → 115 kWh. The encoding was proven from 13 API+Summary pairs: 0="Unknown" (9 pairs), 2="2013 or later" (4 pairs). The open-data export did not preserve the pump age that produced the lodged 73. It is genuinely unreachable from the JSON. Do not chase 0240 to 73; re-pin to its correct 72 once the roof lands.

pump_age enum (verified): 0=Unknown→115, 1=Pre-2013→165, 2=2013+→41 (_TABLE_4F_CIRCULATION_PUMP_KWH_BY_AGE in cert_to_inputs.py).

---

THE METHODOLOGY LESSON FROM THIS SESSION (read this)

The 0240 baseline (cont 72.39) was two offsetting bugs: a wall U under-count (0.25 vs 0.35, less loss) masking an Ext1 roof U over-count (0.68 vs ~0.25, more loss). Fixing one alone moves the residual the "wrong" way — fix both (feedback_software_no_special_handling). And: Elmhurst is the arbiter, not the spec text — twice this session a confident spec/first-principles read was wrong and a generated worksheet settled it (NI=not-indicated not "none"; pump 0=Unknown). Generate a worksheet rather than guess a U-value or factor. The user generates Elmhurst worksheets readily (cases 7–10 done) — ask for one.

---

THREAD 1 — Roof fix — CLOSED (S0380.211)

0240's Ext1 (BP2) lodges roof_construction=5 (vaulted), NI thickness, "Pitched, insulated (assumed)", band J → the cascade hit u_roof's insulation_thickness_mm==0 and _described_as_insulated override → 0.68 (the §5.11.4 retrofit-50 mm joist row). A vaulted/sloping ceiling has no joist void, so per RdSAP 10 §5.11 Table 18 (p.45) it takes the column (1) age-band default (band J = 0.16), NOT 0.68.

The arbiter was the cohort, not case 11 — a methodology trap avoided. The handover above guessed the value was col-3 0.25 (→ cont 72.31), citing case 9. That was WRONG. The decisive evidence: 33 cohort-2 certs lodge ND (thickness None) vaulted roofs (roof_construction=5, band D) that already pin to their dr87 worksheets at 0.40 = Table 18 col (1). 0240's only difference is the NI sentinel (insulation present, unknown thickness), which uniquely hit the 0.68 override. So the spec-correct value is col (1) 0.16, and 0240 lands at cont 72.4617, integer 72 — NOT 72.31. A first broad attempt (route sloping → col-3 _FLAT_ROOF_BY_AGE) broke all 33 cohort certs (band D col-3 = 2.30 vs worksheet 0.40) — that failure is what revealed the col-1 answer. Lesson: when a U-value change moves worksheet-pinned cohort certs off their pins, the change is wrong; the cohort worksheets are ground truth.

Implementation: new u_roof(is_sloping_ceiling=...) flag, threaded from heat_transmission for roof_construction_type containing "sloping ceiling" (code 8) or "vaulted" (code 5). Fires only on the NI case (thickness 0 + "insulated (assumed)") → col (1); the ND/None path is untouched (already col 1) and a normal pitched-with-loft roof still takes the §5.11.4 50 mm row (flag defaults False). 0240 PE +5.5044 → +1.5181, CO2 +0.2757 → +0.0728 (SAP 72 unchanged). Re-pinned in test_golden_fixtures.py.

⚠ Suite-command gap discovered: the AGENT_GUIDE §4 suite command does NOT run domain/sap10_ml/tests/, where u_roof/u_wall unit tests live. S0380.210 shipped a broken test_u_wall_cavity_..._filled_cavity_row there unnoticed; S0380.211 corrected it (→ ..._as_built_row). When touching rdsap_uvalues.py, also run domain/sap10_ml/tests/. Two PRE-EXISTING failures live there (stone §5.6 thin-wall formula 3.7408 vs Table-6 1.7 cap, granite + sandstone band A) — they fail at HEAD 58ff7d88 too, unrelated to this branch.

---

THREAD 2 — Community fuel-code collision (cert 9390) — CO2/PE FIXED (S0380.212); cost +4 open

Cert 9390-2722-3520 (community mains-gas boiler, sap_main_heating_code=301, main_fuel_type=20). Authoritative: datatypes/epc/domain/epc_codes.csv (RdSAP-Schema-17.0) main_fuel,20,mains gas (community).

Root cause (the collision): the EPC main_fuel_type enum and the SAP Table 12 / Table 32 numbering overlap in 18–25 — EPC 20='mains gas (community)' but Table-12 code 20 is solid biomass (CO2 0.028). co2_factor_kg_per_kwh/primary_energy_factor/ unit_price_p_per_kwh check the Table-12 dict FIRST, so the EPC community fuel got the biomass factor instead of translating 20→51 (community mains gas: CO2 0.210, PE 1.130).

S0380.212 fix: new _heat_network_factor_fuel_code(main) translates the EPC community fuel → Table-12 code via API_FUEL_TO_TABLE_12, but ONLY for heat-network mains (_is_heat_network_main) so a genuine biomass boiler keeps its raw factor. Applied at five sites — space-heating CO2/PE/unit-price + water-heating (WHC 901) CO2/PE (9390's HW is ALSO community gas, so both paths needed it). Worksheet-validated by case 14 (community boilers + mains gas, code 301): (367) CO2 0.2100, (467) PE 1.1300 = the Table-12 code-51 values. 9390 CO2 0.44 → 3.03 t (lodged 2.8 — spec-correct factor over the API-only register residual; 9390 is unpinned, retired P2.2 per ADR-0010 §10), PE 204 → 220 (the prior 204≈205 was the collision coinciding with the register residual). Summary path uses code 1 (no collision) → CH1-6 corpus untouched. Locked by 2 unit tests in test_cert_to_inputs.py.

Cost +4 — FIXED (S0380.213), via the standing charge (NOT cost scaling). The earlier "missing 1/heat_source_eff cost scaling" hypothesis was WRONG: case 14's 10b block shows the heat price ((340)/(307) = 4.24 p/kWh) is applied to delivered heat, NOT scaled — and Table-32 code 51 already = 4.24 p/kWh (the price collision was harmless, 4.23≈4.24). The real gap was the £120 heat-network standing charge (SAP 10.2 Table 12 note (l) + §C3.2; case 14 (351) = £120): the API community fuel (20) isn't a Table-32 gas code so _is_gas_code returned False → £0 standing (the Summary path masks it via code 1). New _heat_network_standing_charge_gbp REPLACES the fuel standing for heat-network mains (£120 full / £60 DHW-only) — not additive, so the CH corpus (already £120 via the gas branch) isn't double-counted to £240. 9390 SAP +4 → -2.

STILL OPEN — 9390 ~7% demand over-count (SAP -2): the standing fix EXPOSED it — PE 220 vs lodged 205, CO2 3.03 vs 2.8 all run ~7% high. Likely the heat-source-efficiency default (_HEAT_NETWORK_HEAT_SOURCE_EFFICIENCY[301]=0.80) being too low for 9390's actual scheme, or a fabric/demand difference. 9390 is API-only (no worksheet) + unpinned, so this is a low-priority residual; needs a 9390-specific efficiency/fabric investigation.

---

THREAD 3 — Cert 0390 +7 — CLOSED (S0380.210)

0390-2954-3640 (detached, TFA 360, age F). The boiler was correctly resolved (PCDB 9005 Firebird S, 86.4% winter); the gap was a single fabric mis-route. Walking the §3 cascade localised it to the Main cavity wall: lodged wall_construction=4, wall_insulation_type=4 (as-built/assumed), description "Cavity wall, as built, partial insulation (assumed)". The cascade routed it to the Table 6 "Filled cavity" row (band F = 0.40) because _described_as_insulated matches the "partial insulation" substring. Per RdSAP 10 Table 6 (England) an as-built partial-fill cavity uses "Cavity as built" (band F = 1.0), not filled — a genuine fill lodges the distinct "Cavity wall, filled cavity" (wall_insulation_type=2). This mirrors the worksheet-validated solid-brick rule from S0380.209 (cases 9/10).

Fix: new _cavity_described_as_filled predicate, used only in u_wall's cavity filled-row branch, excludes "partial insulation" while keeping "insulated (assumed)" → filled. Wall HLC +53.6 W/K lifted all four metrics together: SAP +7 → +0, PE −27.9745 → +0.5281, CO2 −2.7134 → −0.1189. Bands I-M coincide († footnote) so 0535(M)/7536(L) are unaffected. Re-pinned in test_golden_fixtures.py.

Diagnosis lesson / latent follow-up: the fix collided with an existing test, test_cavity_as_built_insulated_assumed_uses_filled_cavity_row (heat_transmission tests). That test (from early "slice S-B25") asserts a cavity "insulated (assumed)" → filled row, citing only an assumption ("the assessor has determined the cavity is filled"), never worksheet-validated — and it is the OPPOSITE conclusion from the worksheet-backed solid-brick sibling. The narrow S0380.210 fix leaves it untouched (no current cert exercises it at a band where as-built ≠ filled). Open question for a future worksheet: does a cavity lodged "as built, insulated (assumed)" (type 4) belong on the filled row (0.7 at E) or the as-built row (1.5 at E)? If a worksheet says as-built, fold "insulated (assumed)" into the as-built routing too and retire that test.

---

Full audit — all golden certs with non-zero SAP residual

cert	SAP resid	diagnosis
0390-2954-3640	+7 +0	CLOSED S0380.210 — cavity partial-insulation → as-built row
9390-2722-3520	−2 (unpinned)	CO2/PE collision FIXED S0380.212 + standing charge S0380.213 (SAP +4→−2); remaining ~7% demand over-count (heat-source-eff default?)
0240-0200-5706	−1	NOT a bug — unpreserved 2013+ pump; true SAP 72. Roof PE-pin tightened by S0380.211 (PE +5.50 → +1.52)
2130-1033-4050	+1	minor fabric precision (multi-part solid-brick wall); low value
7536-3827-0600	+1	minor fabric precision (multi-bp D/L/F cavity); low value

All others pin at residual 0.

---

Worksheets

• Available (user-generated, sap worksheets/golden fixture debugging/simulated case N/): case 7 (combi), case 8 (unknown pump), case 9 (sandstone wall + sloping roof), case 10 (solid-brick wall), case 11 (001431 sloping-ceiling Unknown roof — used to scope Thread 1; the cohort ND certs were the real arbiter), case 12 (community CHP coal, code 302), case 13 (community CHP mains gas, code 302). Case 7's Summary is the only one mirrored into tracked fixtures (backend/.../Summary_001431_case7.pdf, used by the e2e pin).
• Still needed (Thread 2): a code-301 (community boilers, NOT CHP) + mains gas worksheet to pin 9390's exact PE/CO2/cost. case 13 is code-302 CHP-gas: it confirms the community-gas direction (heat-network (386) CO2 0.2456) but the CHP heat-power split differs from 9390's boiler scheme. API main_fuel_type=20 = community/district heating from mains gas → SAP Table 12 code 51 (CO2 0.210, PEF 1.130).

Pointers

• Golden pins + slice history: tests/domain/sap10_calculator/rdsap/test_golden_fixtures.py.
• Wall fix: domain/sap10_calculator/worksheet/heat_transmission.py (_described_as_retrofit_insulated, the wall_ins_present gate) + tests/.../worksheet/test_heat_transmission.py.
• Roof code: domain/sap10_ml/rdsap_uvalues.py u_roof (L708 + Table 18 dicts L637-668).
• Community/heat-network CO2/PE/cost: cert_to_inputs.py ~L2749-2880, L1837 (_main_fuel_code), tables/table_12.py + table_32.py (co2_factor_kg_per_kwh, API_FUEL_TO_TABLE_12/32).
• Process: one slice = one commit, spec citation (page+line), AAA tests, abs(x-y)<=tol not pytest.approx, Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>. SAP 10.2 only. No tolerance widening. mapper.py + cert_to_inputs.py each carry 32 pre-existing pyright errors; heat_transmission.py + rdsap_uvalues.py carry their own — baseline-compare with git stash for net-zero NEW.