Model/domain/sap10_calculator/docs/HANDOVER_POST_S0380_69.md

Handover — post S0380.69 (cert 000565 + cohort-2 golden coverage)

Branch feature/per-cert-mapper-validation. HEAD c4b27829 (Slice S0380.69 — cohort-2 added to test_golden_fixtures.py). Predecessor: HANDOVER_CERT_000565_COST_CASCADE.md covers S0380.52..63. This doc covers S0380.64..69.

Test baseline: 317 pass (was 279) + 9 expected 000565 cascade-gap fails. Pyright net-zero on every touched file.

Slices committed this session (S0380.64..69)

Slice	Commit	Domain	Headline closure
S0380.64	6b02bad0	Elmhurst per-extension wall_construction mappings (SG → 1 stone-granite, B → 6 basement, CF → 4 cavity-filled party) + strict-raise on unknown gable codes via UnmappedElmhurstLabel. RdSAP 10 §5.17 / Table 23 basement-wall routing.	sap_score 30 → 29 EXACT; space_heating Δ −1,099 → +266; HTC 1281 → 1321 W/K
S0380.65	7855a715	SAP 10.2 Table 12d + Table 12a Grid 1 dual-rate main-heating CO2 factor. New _TARIFF_HIGH_LOW_FUEL_CODES_TABLE_12 dict + _main_heating_co2_factor_kg_per_kwh(main, tariff, monthly_kwh) helper mirroring the cost-side dual-rate split from S0380.61.	CO2 Δ −624 → −20 kg/yr; main_heating_co2_factor 0.136 → 0.1533 EXACT vs worksheet line 261
S0380.66	db4f1b31	New domain/sap10_calculator/worksheet/appendix_h_solar.py — SAP 10.2 Appendix H pure math module (HW path). Line refs (H10), (H11), (H14)..(H16), (H17)..(H24) + 18 unit tests pinning to worksheet lines 407 / 410 / 411 / 412.	Pure math; no cascade integration yet
S0380.67	2795e256	Added monthly_solar_energy_available_h9_w helper + fixed (H23) unit handling: W·h → kWh integration via explicit hours_in_month parameter (S0380.66 elided this by absorbing time integration into the parameter name).	Unit consistency
S0380.68	f0ab7446	Added (H7)m flux helper (monthly_collector_solar_flux_w_per_m2) reusing existing surface_solar_flux_w_per_m2 + top-level orchestrator solar_water_heating_input_monthly_kwh. Shape test pins winter-zero / summer-peak.	Orchestrator landed; magnitude calibration DEFERRED (see §"Open / deferred — Appendix H magnitude" below)
S0380.69	c4b27829	Added 38 cohort-2 certs to test_golden_fixtures.py with SAP / PE / CO2 baseline pins.	Cert 2102's +20.36 PE / −0.79 CO2 outlier now visible to any cascade refactor (was invisible to all prior tests)

Current cert 000565 residuals (HEAD c4b27829)

Pin	Cascade	Worksheet	Δ	Status
sap_score	29	29	0 ✓	EXACT since S0380.64
sap_score_continuous	29.1421	28.5087	+0.6334	Closed from +1.7225 (S0380.64)
ecf	5.3223	5.3866	−0.0643	Closed from −0.1735
total_fuel_cost_gbp	4,624.18	4,680.26	−56.08	Closed from −150.93
co2_kg_per_yr	6,427.86	6,447.63	−19.77	EXACT at sub-spec level since S0380.65 (was −624)
main_heating_fuel	34,867.33	34,710.79	+156.54	Follows space_heating / 1.70 exactly
space_heating	59,274.46	59,008.35	+266.11	Largest remaining energy residual. Now slightly OVER-counting (was −1,099 pre-S0380.64). Basement walls add ~+170 vs worksheet's lower U formula
hot_water	4,026.87	3,755.03	+271.84	Second-largest. Blocked on Appendix H magnitude calibration
lighting	1,387.02	1,384.84	+2.19	Essentially closed (sub-spec)
pumps_fans	255.00	252.52	+2.48	MEV gap (blocked on external PCDB data)
secondary_heating_fuel	0.00	0.00	0 ✓	Green
main_heating_co2_factor	0.1533	0.1533	0 ✓	EXACT since S0380.65

Cert 000565 now has TWO exact pins (sap_score + CO2 factor) and 9 small-magnitude residuals.

Open / deferred work

A. Appendix H magnitude calibration — BLOCKED on external reference

S0380.66-68 delivered the Appendix H pure math module + top-level orchestrator. Cert 000565 worksheet pins all helpers individually (H11, H14, H15, H16 — exact). But the end-to-end orchestrator produces ~510 kWh annual H24 vs worksheet 281.35 (1.8×).

Root cause is a SAP 10.2 spec ambiguity between two formulations of Y:

Source	Spec page	Formula	Δ vs other
Top-level Eqn H1 commentary	p.75 line 4517	Y = Px × Aap × IAM × η0 × ηloop × Im × Hm / (1000 × Dm)	excludes H8
Line-ref (H23) formula	p.76 line 4620	Y = [(H18) × (H6) × (H5) × (H9) × ((41) × 24)] / [1000 × (H17)] where (H9) = (H1) × (H2) × (H7) × (H8)	includes H8

The two formulations differ by factor H8 (0.8 for cert 000565). Both formulations were also tried (removing H8 / keeping H8 / adding H5/H6 to H9 / dividing by H8 in X / etc.) — none close the 1.8× gap. The 1.8× factor isn't H8 alone.

Resolving this needs an external reference NOT in this repo:

1. BRE's own worksheet trace of (H22)/(H23) intermediates for any cert (only annual H24 is shown in the U985 worksheet)
2. The underlying EN 15316-4-3:2017 standard text (this is what Appendix H implements per SAP 10.2 p.74)
3. An open-source SAP calculator's Appendix H implementation source

Important constraint per feedback-sap-10-2-only-never-10-3: do NOT reference SAP 10.3 (the spec ambiguity is identical in 10.3 anyway).

The orchestrator is wired but NOT integrated into water_heating_from_cert.solar_monthly_kwh (still hardcoded to zero12 at domain/sap10_calculator/worksheet/water_heating.py:943). Integrating with the current 1.8× over-estimate would WORSEN cert 000565's HW residual (4027 − 510 × eff ≈ 3624 vs worksheet 3755 → Δ −131 instead of today's +272).

B. RR fold-in for space_heating +266 — DEFERRED, multi-component piece

walls_w_per_k = 322 vs worksheet 604 (Δ −282 W/K). Most of the gap is RR Common Walls + Gable Walls not folded into the (29a) external-walls channel.

Attempted in this session (S0380.69 candidate, reverted): routing gable_type='Exposed' to gable_wall_external would close the classification gap, BUT the cascade's gable AREA (raw L × H from Summary PDF) is 4× the worksheet's RR-portion-only area (e.g. Ext1 Gable 2: cascade 72 m² vs worksheet 16.08 m²). Classification fix without area fix overshoots: sap_score regresses 29 → 25, space_heating overshoots +6029 kWh.

RR fold-in requires three coordinated changes:

1. Extractor / mapper area computation per RdSAP §3.10 detailed-RR geometry — the worksheet computes some kind of triangulated / truncated area, not raw L×H
2. Classification fix (Exposed / Connected gable_type values surfaced)
3. Common Wall extraction (currently filtered at _map_elmhurst_rir_surface line 3260)

Each in isolation regresses sap_score. Reverse-engineering the area formula from cert 000565 alone wasn't tractable in this session — the cascade has a Simplified-RR formula at heat_transmission.py:389 that doesn't match worksheet's 16.08 for any plausible H_common_wall value.

Recommendation: wait for another cohort cert with cleaner RR geometry lodgement, OR get a clear read of RdSAP 10 §3.10 detailed-RR area formula, before re-attempting.

C. MEV cascade (line 230a) — BLOCKED on external BRE data

Cert 000565 worksheet line 230a: MEV = IUF × SFP × 1.22 × V = 127.5159 kWh. PCDF 500755 record carries SFP=0.1274 and a derived IUF≈1.278. The PCDB MEV / MVHR record table is NOT in the codebase (only Tables 105, 122, 143, 313, 353, 362, 391, 506 are present under domain/sap10_calculator/tables/pcdb/data/). Acquiring the PCDB MEV table from BRE is the gating step.

Couples with HP-category-derivation fix (item D) — landing alone would worsen pumps_fans from 255 → 125 W/K.

D. HP SAP code → main_heating_category=4 in mapper

_elmhurst_main_heating_category only sets category=4 when a PCDB Table 362 record is present. Cert 000565 Main 1 SAP code 224 (ASHP) with no PCDB ref → category=None → cascade routes pumps_fans to 130 default base instead of HP's 0 base. Couples with MEV (item C); see [project_cert_000565_recovery_state.md] memory.

E. Cert 2102 +20.36 PE / −0.79 CO2 — newly visible via S0380.69

Cohort-2 cert lodges House coal as secondary heating. S0380.43 closed SAP via spec-fuel routing but didn't address the PE/CO2 paths. This is now the largest cohort-2 PE residual and the cleanest next investigation target.

Conventions reinforced this session

• Verify spec before implementing (feedback-verify-handover-claims) — S0380.64 + S0380.65 cited Table 23 / Table 12d directly; S0380.66 quoted SAP 10.2 spec page numbers verbatim.
• SAP 10.2 only, never 10.3 (feedback-sap-10-2-only-never-10-3) — added this session after I reached for 10.3 to resolve the Appendix H ambiguity. The project tracks SAP 10.2 deliberately; 10.3 has the same ambiguity anyway.
• Bigger slices for uniform work (feedback-bigger-slices-for-uniform-work) — S0380.64 bundled three mapper entries + two strict-raise calls; S0380.69 bundled 38 parametrised cohort-2 pins.
• Coupling-aware sequencing — attempted RR classification fix was reverted because area-fix wasn't ready; HP-category fix is held back because MEV isn't ready. Components must land as a SET.
• Strict-raise on unmapped data (reference-unmapped-api-code / UnmappedElmhurstLabel) — extended to gable wall codes in S0380.64.
• One slice = one commit, spec-citation in commit messages (feedback-commit-per-slice + feedback-spec-citation-in-commits).
• Pyright net-zero per touched file (feedback-zero-error-strict).

How to run the baseline

PYTHONPATH=/workspaces/model python -m pytest \
    backend/documents_parser/tests/test_summary_pdf_mapper_chain.py \
    backend/documents_parser/tests/test_elmhurst_extractor.py \
    backend/documents_parser/tests/test_elmhurst_end_to_end.py \
    domain/sap10_calculator/worksheet/tests/test_e2e_elmhurst_sap_score.py \
    domain/sap10_calculator/worksheet/tests/test_appendix_h_solar.py \
    domain/sap10_calculator/rdsap/tests/test_cert_to_inputs.py \
    domain/sap10_calculator/rdsap/tests/test_golden_fixtures.py \
    --no-cov -q

Expected: 317 pass + 9 expected test_sap_result_pin[000565-*] fails (the 9 non-exact cascade-gap pins in the residuals table above).

How to probe cert 000565 residuals

PYTHONPATH=/workspaces/model python -c "
from domain.sap10_calculator.worksheet.tests._elmhurst_worksheet_000565 import build_epc
from domain.sap10_calculator.rdsap.cert_to_inputs import cert_to_inputs, SAP_10_2_SPEC_PRICES
from domain.sap10_calculator.calculator import calculate_sap_from_inputs
epc = build_epc()
inputs = cert_to_inputs(epc, prices=SAP_10_2_SPEC_PRICES)
r = calculate_sap_from_inputs(inputs)
# inspect fields per the residuals table above
"

How to probe cohort-2 golden residuals (cert 2102 is the next target)

PYTHONPATH=/workspaces/model python -c "
import json
from pathlib import Path
from datatypes.epc.domain.mapper import EpcPropertyDataMapper
from domain.sap10_calculator.calculator import calculate_sap_from_inputs
from domain.sap10_calculator.rdsap.cert_to_inputs import (
    SAP_10_2_SPEC_PRICES, cert_to_demand_inputs, cert_to_inputs,
)
fixtures = Path('/workspaces/model/domain/sap10_calculator/rdsap/tests/fixtures/golden')
doc = json.loads((fixtures / '2102-3018-0205-7886-5204.json').read_text())
epc = EpcPropertyDataMapper.from_api_response(doc)
rating = calculate_sap_from_inputs(cert_to_inputs(epc, prices=SAP_10_2_SPEC_PRICES))
demand = calculate_sap_from_inputs(cert_to_demand_inputs(epc, prices=SAP_10_2_SPEC_PRICES))
# Pinned residuals: PE +20.36, CO2 −0.79 (see test_golden_fixtures.py)
"

Spec source quick-reference

• SAP 10.2 full specification: domain/sap10_calculator/docs/specs/sap-10-2-full-specification-2025-03-14.pdf
  • Table 12d (monthly electric CO2 factors): p.194
  • Table 12a (Grid 1 SH + Grid 2 other uses fractions): p.191
  • Appendix H (Solar thermal systems): p.74-78 + Tables H1-H4 p.78
  • Appendix U §U3.2 (horizontal solar flux + tilt polynomial): p.127
• RdSAP 10 specification: domain/sap10_calculator/docs/specs/RdSAP 10 Specification 10-06-2025.pdf
  • §12 page 62 (dual-meter tariff dispatch)
  • Table 32 page 95 (unit prices + standing charges)
• BRE technical papers: domain/sap10_calculator/docs/specs/sap10 technical papers/ (STP09-B04 + S10TP-{02..13})
• SAP 10.3 at domain/sap10_calculator/docs/specs/sap-10-3-full-specification-2026-01-13.pdf: DO NOT reference (feedback-sap-10-2-only-never-10-3).

Key file map (added / touched this session)

Path	Role	Touched in
datatypes/epc/domain/mapper.py	_ELMHURST_WALL_CODE_TO_SAP10 + _ELMHURST_PARTY_WALL_CODE_TO_SAP10 dicts + strict-raise helpers	S0380.64
domain/sap10_calculator/rdsap/cert_to_inputs.py	_TARIFF_HIGH_LOW_FUEL_CODES_TABLE_12 + _main_heating_co2_factor_kg_per_kwh helper	S0380.65
domain/sap10_calculator/worksheet/appendix_h_solar.py	NEW SAP 10.2 Appendix H pure math + orchestrator	S0380.66-68
domain/sap10_calculator/worksheet/tests/test_appendix_h_solar.py	NEW 22 unit tests pinning Appendix H math	S0380.66-68
domain/sap10_calculator/rdsap/tests/test_golden_fixtures.py	Cohort-2 38 _GoldenExpectation entries	S0380.69
backend/documents_parser/tests/test_summary_pdf_mapper_chain.py	5 new tests for cert 000565 gable-code coverage + strict-raise	S0380.64
domain/sap10_calculator/rdsap/tests/test_cert_to_inputs.py	2 new tests for dual-rate CO2 factor	S0380.65

When this handover becomes stale

• After Appendix H magnitude calibration resolves (EN 15316-4-3 sourced, or BRE worksheet intermediates trace, or empirical multi-cert calibration) — wire solar_water_heating_input_monthly_kwh into water_heating_from_cert.solar_monthly_kwh, expect cert 000565 HW residual to close from +272 to ~0.
• After MEV PCDB data lands + HP-category-derivation fix lands as a SET — pumps_fans pin closes 255 → 252.5.
• After RR fold-in lands (3-slice coordinated piece) — cert 000565 walls_w_per_k closes 322 → 604; space_heating closes +266 → ~0.
• After cert 2102 House-coal secondary PE/CO2 cascade closes — cohort-2 largest residual drops from +20.36 / −0.79.