Model/domain/sap10_calculator/docs/HANDOVER_POST_S0380_84.md

Handover — post S0380.81..84 (Table 32 default + Table 12a Grid 2 CO2 + RR fold-in)

Branch: feature/per-cert-mapper-validation. HEAD 49622f55. Predecessor: HANDOVER_POST_S0380_80.md.

Slices committed this session (S0380.81..84)

Four spec-cited slices, two clean closures + one extractor data-completion

• one structural RR fix that surfaces the next named gap.

Slice	Commit	Spec	Cert 000565 outcome
S0380.81	9338914f	RdSAP 10 §19.1 (PDF p.80-81) — "use Table 32 prices (not Table 12) for §10a/§10b"	sap_score 28 → 29 EXACT at the (28.5) rounding boundary. Cost residual £+3.62 → £−2.39.
S0380.82	27ead127	SAP 10.2 Table 12a Grid 2 (p.191) + Table 12d/12e (p.194-195) — "All other uses" off-peak dual-rate	CO2 residual −8.92 → −3.08 kg/yr (65% closed). Lighting + pumps_fans + electric_shower CO2/PE factors now blend Table 12d/12e high-rate × low-rate codes per Grid 2 fraction on off-peak certs.
S0380.83	ed8fdc6a	RdSAP 10 §3.10 + Summary PDF §8.1 schema	Extractor recognises "Exposed" + "Connected" gable_type (was Party / Sheltered / "Connected to heated space" only). Mapper elif extended to preserve cert 9501. Pure data-extraction completion; zero cascade impact.
S0380.84	49622f55	RdSAP 10 §3.9.2 + §3.10 + Table 4 (p.22)	Mapper drops Connected gables, routes Exposed → gable_wall_external with lodged U, surfaces Common Walls, applies spec area formula L × (0.25 + H) and Σ per-common-wall gable correction. Cascade adds common_wall kind handler. 11 per-BP RR surface areas EXACT vs worksheet PDF. Cascade walls 322 → 443 W/K (43% closer to worksheet 604); party 153 → 93 (68% closer to worksheet 65). cert 000565 sap_score temporarily regressed 29 → 26 — see §"Why the regression is the correct signal" below.

Test baseline at HEAD 49622f55: 555 pass + 9 expected test_sap_result_pin[000565-*] cascade-gap fails. Pyright net-zero on every touched file. Cohort + golden + cert 9501 unaffected.

Cert 000565 state (HEAD 49622f55)

Pin	Cascade	Worksheet	Δ	Cause
sap_score (int)	26	29	−3	RR fold-in (S0380.84) exposed BP main-wall gap; see §"BP main-wall residual −161 W/K diagnostic"
sap_score_continuous	26.4972	28.5087	−2.01	Downstream of HTC over-count via space_heating +2591
ecf	5.5970	5.3866	+0.21	Downstream of cost +£182.6
total_fuel_cost_gbp	4862.88	4680.26	+182.6	Downstream of space_heating fuel cost
co2_kg_per_yr	6684.52	6447.63	+236.9	Downstream of HP electricity over-fuel-use
space_heating_kwh	61599.61	59008.35	+2591.3	BP main-wall cascade gap (Curtain Wall −112 W/K + thin-wall alt −47 W/K — see diagnostic below)
main_heating_fuel	36235.07	34710.79	+1524.3	Follows space_heating via 1/COP
hot_water_kwh	3755.03	3755.03	✓ 0 EXACT	§4 cascade fully closed (S0380.77..80)
lighting	1387.02	1384.84	+2.19	Sub-spec
pumps_fans	255.00	252.52	+2.48	MEV PCDB record missing (external data)

§4 HW cascade line refs (all EXACT, unchanged)

Line	Cascade	Worksheet	Δ
(45)m sum energy_content	1286.3266	1286.3266	✓ 0
(46)m sum distribution_loss	192.9490	192.95	✓ <1e-3
(57)m sum solar_storage	596.9725	596.9725	✓ <1e-4
(59)m sum primary_loss	1176.77	1174.79	+1.98
(61)m combi_loss	0.00	0.00	✓ 0
(62)m sum total_demand	3060.07	3060.07	✓ <1e-3
(64)m sum (after solar)	2778.72	2778.7213	✓ <1e-4
(64a)m electric shower	702.94	702.94	✓ <1e-4
(217)m water-heater eff	0.74	0.74	✓ EXACT
(219) HW fuel kWh	3755.03	3755.0288	✓ <1e-3

Per-BP RR surface areas (all EXACT after S0380.84)

Verified against the cert 000565 U985 worksheet "External Walls" + "Party Walls" sections at 4 d.p. precision:

BP	Surface	Spec formula	Worksheet	Cascade
0	Main GW1 Exposed	4 × 2.45 (Simplified, no CW)	9.80	9.80 ✓
0	Main GW2 Sheltered	6 × 2.45	14.70	14.70 ✓
1	Ext1 CW1	9 × (0.25 + 1.0) (Simplified + CW)	11.25	11.25 ✓
1	Ext1 CW2	5 × (0.25 + 1.8)	10.25	10.25 ✓
1	Ext1 GW2 Exposed	8 × (0.25+9) − ((9−1)²+(9−1.8)²)/2	16.08	16.08 ✓
2	Ext2 GW2 Exposed	3 × 8 (Detailed)	24.00	24.00 ✓
3	Ext3 CW1	5 × (0.25 + 1.5) (Simplified + CW)	8.75	8.75 ✓
3	Ext3 CW2	7.5 × (0.25 + 0.3)	4.13	4.13 ✓
3	Ext3 GW1 Exposed	9 × (0.25+7) − ((7−1.5)²+(7−0.3)²)/2	27.68	27.68 ✓
4	Ext4 CW1	4 × 1 (Detailed)	4.00	4.00 ✓
4	Ext4 CW2	3.5 × 0.6 (Detailed)	2.10	2.10 ✓

Cumulative cert 000565 closure (S0380.77 → .84)

Pin	.77→	.78→	.79→	.80→	.81→	.82→	.83→	.84
hot_water_kwh	+1399	+260	−238	✓0	✓0	✓0	✓0	✓0
sap_score (int)	+1	−1	0	−1	✓0	✓0	✓0	−3 ⚠
sap_score_continuous	+0.60	−0.04	+0.06	−0.04	+0.03	+0.03	+0.03	−2.01 ⚠
ecf	−0.06	+0.00	−0.01	+0.00	−0.00	−0.00	−0.00	+0.21 ⚠
total_fuel_cost_gbp	−53	+3	−5	+4	−2	−2	−2	+183 ⚠
co2_kg_per_yr	(n/a)	(n/a)	−58	−9	−9	−3	−3	+237 ⚠

S0380.84 ⚠ rows are the documented BP main-wall surfacing (NOT a regression of S0380.84's RR fix itself).

Why the regression is the correct signal

S0380.84 closed the RR cascade routing to spec correctness. The 11 per-BP RR surface areas pin EXACT vs worksheet at 4 d.p. The cascade walls subtotal moved 322 → 443 W/K (worksheet 604, 43% closed); party 153 → 93 (worksheet 65, 68% closed).

Pre-S0380.84 the cascade had two ~equal-magnitude bugs of opposite sign that mostly cancelled at the SH-pin level:

• RR cascade: Exposed gables wrongly routed to party_walls at U=0.25 (cascade over-counts party_walls by ~88 W/K)
• BP main-wall cascade: Curtain Wall + thin-wall alt missing (cascade under-counts walls by ~161 W/K)

S0380.84 closed the first one. The second is now exposed as a +2591 kWh space_heating residual. Per [[feedback-spec-citation-in-commits]] and [[feedback-spec-floor-skepticism]] the spec-correct fix ships even when the test pin temporarily regresses; the diagnostic signal is sharper now.

BP main-wall residual −161 W/K diagnostic

Probed per-BP at HEAD 49622f55:

BP	Cascade U	Worksheet U	Δ contribution	Spec gap
0 Main	0.32	0.35	−1.6 W/K	sub-spec (Solid Brick A age, 75mm External insulation)
0 Main alt1	0.32	2.34	−46.5 W/K	_insulation_bucket(thk=120, ins_present=False) returns 100 not 0 (docstring intent vs current code) + thin-wall §6.6/§6.7 for U=2.34
1 Ext1	1.70	1.70	✓ 0	✓ Spec-correct (Stone Granite E age, Unknown insulation)
2 Ext2 (Curtain Wall)	0.60	1.40	−112.2 W/K	WALL_CURTAIN=9 defined rdsap_uvalues.py:116 but no _ENG_WALL table entry; u_wall falls through to default Cavity table
3 Ext3 (basement)	0.45	0.45	✓ 0	✓ Spec-correct
4 Ext4 (basement)	0.35	0.35	✓ 0	✓ Spec-correct

Total: −160.3 W/K (matches the observed −161 W/K).

Gap #1 — Curtain Wall (largest, −112 W/K)

Where: domain/sap10_ml/rdsap_uvalues.py:116 — WALL_CURTAIN: Final[int] = 9 is defined but has no entry in _ENG_WALL and is not in the known_types set at u_wall:373-376. When the cert lodges wall_construction=9, u_wall falls through to _DEFAULT_WALL_BY_AGE (default cavity) and returns the cavity-wall U for that age band.

Cert 000565 BP[2] Ext2 lodges Type: CW Curtain Wall + Curtain Wall Age: Post 2023 per Summary PDF §7. The "Curtain Wall Age" is a separate per-BP attribute from the dwelling-wide construction_age_band — the BP is age H (1991-1995) but the curtain wall itself was installed Post-2023. Worksheet uses Curtain Wall Post-2023 U=1.40.

Slice span:

1. Extractor (backend/documents_parser/elmhurst_extractor.py) — currently doesn't surface "Curtain Wall Age" from Summary §7
2. datatypes/epc/surveys/elmhurst_site_notes.py — add curtain_wall_age to WallDetails
3. datatypes/epc/domain/epc_property_data.py — add curtain_wall_age to SapBuildingPart
4. datatypes/epc/domain/mapper.py — thread through both API + Elmhurst paths
5. domain/sap10_ml/rdsap_uvalues.py — Curtain Wall U-value lookup by age; add WALL_CURTAIN to known_types

Spec citation needed: RdSAP 10 Table 6 or related — locate the canonical Curtain Wall U-values per age category. The worksheet says 1.40 for Post-2023; need to verify the full table.

Gap #2 — Thin-wall alt stone granite (−47 W/K)

Where: Two coupled bugs.

1. domain/sap10_ml/rdsap_uvalues.py:160 _insulation_bucket ignores insulation_present=False when thickness_mm > 0. Docstring says "when not present, the as-built (bucket 0) row applies regardless" but the code falls through to thickness-bucket selection. For BP[0] alt1 with wall_insulation_type=4 (None) but wall_insulation_thickness='120', bucket returns 100 not 0.

2. The wall_insulation_thickness='120' on SapAlternativeWall is actually the WALL thickness lodged in Summary §7 ("Alternative Wall 1 Thickness: 120 mm"), NOT an insulation thickness. Per feedback-no-misleading-insulation-type this should land on a new SapAlternativeWall.wall_thickness_mm field (mirror of SapBuildingPart.wall_thickness_mm).

3. Even with bucket 0, _TYPICAL_STONE_UNINSULATED[0] = 1.7 + dry-lined adjustment = 1.32. Worksheet wants 2.34. This is the RdSAP 10 §6.6/§6.7 "thin-wall" formula for stone walls below typical thickness — needs implementing in u_wall.

Slice span:

1. Extractor — surface "Alt 1 Thickness" as wall thickness (currently mapped to wall_insulation_thickness)
2. datatypes/epc/domain/epc_property_data.py — SapAlternativeWall.wall_thickness_mm: Optional[int]
3. datatypes/epc/domain/mapper.py — populate wall_thickness_mm from Elmhurst extractor's alt-wall-thickness field
4. domain/sap10_ml/rdsap_uvalues.py:160 _insulation_bucket — short-circuit if not insulation_present: return 0 per docstring intent (audit cohort for any cert with insulation_present=False AND thickness>0)
5. domain/sap10_ml/rdsap_uvalues.py:329 u_wall — RdSAP §6.6/§6.7 thin-wall formula keyed on wall_thickness_mm for stone constructions

Open work — prioritised next slices

S0380.85 — Curtain Wall (highest impact, −112 W/K of −161)

Extract curtain_wall_age per BP + add Curtain Wall U-value lookup keyed on age. Multi-layer (extractor + datatypes + mapper + cascade); a single slice if the spec lookup is tractable.

Spec citation candidate: RdSAP 10 Table 6 row for "CW Curtain Wall" across age categories. The worksheet (cert 000565) gives Post-2023 → U=1.40 as the empirical ground truth.

Expected impact: cert 000565 cascade walls 443 → 555 (worksheet 604). HTC fabric 795 → 907. SH residual +2591 → +~800 kWh. sap_score should move 26 → ~28 (still 1-2 short of 29 due to remaining alt1 gap).

S0380.86 — Thin-wall alt stone granite (−47 W/K)

Two coupled bugs in rdsap_uvalues.py:

1. _insulation_bucket short-circuit on not insulation_present
2. Thin-wall §6.6/§6.7 formula keyed on a new wall_thickness_mm field on SapAlternativeWall

Plus extractor + datatype + mapper plumbing for the wall_thickness field. After both gaps close: cascade walls 555 → ~610 (matches worksheet 604). HTC → ~960. SH should close to ~−72 (or smaller — the original residual pre-S0380.84).

Deferred (unchanged from post-S0380.80 handover)

• MEV PCDB Table 4f component for pumps_fans +2.5 (blocked on external data acquisition; PCDF 500755 record needed)
• HP SAP code → main_heating_category=4 mapper extension (couples with MEV; ship after MEV record acquired)
• 12 gas-combi PV certs at +0.5..+1.6 PE (no worksheets)
• 5 SAP-residual API-only certs (no worksheets)

How to run the baseline

PYTHONPATH=/workspaces/model python -m pytest \
    backend/documents_parser/tests/test_summary_pdf_mapper_chain.py \
    backend/documents_parser/tests/test_elmhurst_extractor.py \
    backend/documents_parser/tests/test_elmhurst_end_to_end.py \
    domain/sap10_calculator/worksheet/tests/test_e2e_elmhurst_sap_score.py \
    domain/sap10_calculator/worksheet/tests/test_appendix_h_solar.py \
    domain/sap10_calculator/rdsap/tests/test_cert_to_inputs.py \
    domain/sap10_calculator/rdsap/tests/test_golden_fixtures.py \
    --no-cov -q

Expected: 555 pass + 9 expected test_sap_result_pin[000565-*] fails.

The 9 expected fails (verbatim):

sap_score
sap_score_continuous
ecf
total_fuel_cost_gbp
co2_kg_per_yr
space_heating_kwh_per_yr
main_heating_fuel_kwh_per_yr
lighting_kwh_per_yr
pumps_fans_kwh_per_yr

(was 8 + sap_score EXACT at HEAD 27ead127; sap_score moved into the fail list at HEAD 49622f55 per "Why the regression is the correct signal" above).

Files touched this session

File	Slices	Change
domain/sap10_calculator/rdsap/cert_to_inputs.py	S0380.81, .82	Added RDSAP_10_TABLE_32_PRICES; switched default; new _other_use_co2_factor_kg_per_kwh + _other_use_primary_factor helpers; wired into pumps_fans / lighting / electric_shower CO2 + PE fields
domain/sap10_calculator/rdsap/tests/test_cert_to_inputs.py	S0380.81, .82	2 new tests + 3 re-pinned scalar assertions to Table 32
backend/documents_parser/elmhurst_extractor.py	S0380.83	Added "Exposed" + "Connected" to gable_type recognition set
backend/documents_parser/tests/test_summary_pdf_mapper_chain.py	S0380.83, .84	2 new tests (extractor gable_type + mapper RR routing/areas)
datatypes/epc/domain/mapper.py	S0380.83, .84	Mapper RR routing per §3.10 Table 4; drops Connected, routes Exposed external, surfaces Common Walls with §3.9.2 spec area formula
datatypes/epc/domain/epc_property_data.py	S0380.84	SapRoomInRoofSurface.kind docstring extended with common_wall
domain/sap10_calculator/worksheet/heat_transmission.py	S0380.84	Added common_wall kind handler (walls += area × U)

Spec source quick-reference

• SAP 10.2 full specification: domain/sap10_calculator/docs/specs/sap-10-2-full-specification-2025-03-14.pdf
  • Table 12a (p.191) — Grid 1 SH + WH + Grid 2 "All other uses" high-rate fractions
  • Table 12d (p.194) — monthly CO2 factors for electricity
  • Table 12e (p.195) — monthly PE factors for electricity
• RdSAP 10 specification: domain/sap10_calculator/docs/specs/RdSAP 10 Specification 10-06-2025.pdf
  • §3.9 + §3.10 (p.30-35) — Simplified Type 1/2 + Detailed RR
  • Table 4 (p.22) — RR surface variants (gable_wall U-value rules)
  • §19.1 (p.80-81) — Table 32 prices for §10a/§10b
  • Table 32 (p.95) — RdSAP unit prices + standing charges
  • Table 6 — wall U-values by construction + insulation bucket (Curtain Wall entry needed for S0380.85)
  • §6.6 / §6.7 — thin-wall stone formula (S0380.86)
• SAP 10.3 at domain/sap10_calculator/docs/specs/sap-10-3-full-specification-2026-01-13.pdf: DO NOT reference (feedback-sap-10-2-only-never-10-3)

Memory updated this session

• project_cert_000565_recovery_state — S0380.81/.82/.83/.84 entries
  • post-S0380.84 BP main-wall diagnostic table localising the −161 W/K residual to Curtain Wall + thin-wall alt
• MEMORY.md — index entry refreshed at HEAD 49622f55