Model/domain/sap10_calculator/docs/HANDOVER_POST_S0380_76.md

Handover — post S0380.74..76 + Appendix H integration

Branch: feature/per-cert-mapper-validation. HEAD a532f75d. Predecessor: HANDOVER_POST_S0380_73_APPENDIX_H_BLOCKED.md.

Slices committed this session (S0380.74..76)

Long-standing Appendix H 1.81× over-count CLOSED, orchestrator wired into the HW cascade.

Slice	Commit	What
S0380.74	3bf728ce	Appendix H (H7) U3.3 monthly-integrated convention closes 1.81× over-count. SAP 10.2 internal ambiguity for (H7)m between p.75 (W/m² flux) and p.76's "from U3.3 in Appendix U" (kWh/m²/month integrated). Elmhurst follows U3.3; cascade was using U3.2. Fix: convert flux × hours/1000 inside (H9). 47/48 month-observations pin at <1e-4 kWh across 4 fixtures (000565 + new A/B/C at sap worksheets/Solar PV tests/).
S0380.75	a9143d09	Wire Appendix H orchestrator into water-heating cascade. New solar_water_heating_monthly_kwh_override param on water_heating_from_cert; helper _solar_hw_monthly_override in cert_to_inputs.py calls orchestrator with RdSAP 10 §10.11 Table 29 defaults + cert-lodged collector orientation/pitch/overshading from Elmhurst Summary §16.0. Extended Renewables + EpcPropertyData + extractor + mapper. Cert 000565 HW pin: +271.84 → −68.96 kWh/yr (4× closer).
S0380.76	a532f75d	Combined-cylinder H12/H13 routing. Empirical pattern across 4 worksheets (cert 000565 H12=53 ≈ 160/3, cert A/B/C H12=37 ≈ 110/3) → combined-cylinder default H12 = ⅓ × cylinder volume per f-chart pre-heat-zone convention. Derives H12/H13 from epc.has_hot_water_cylinder + sap_heating.cylinder_size. Cert 000565 solar Q_s: 268 → 283 kWh/yr (worksheet 281.35, Δ +1.73 = 0.6% error).

Test baseline at HEAD a532f75d: 547 pass + 9 expected test_sap_result_pin[000565-*] cascade-gap fails. Pyright net-zero on every touched file.

Appendix H closure narrative (the trap that closed)

The cascade vs worksheet ratio for cert 000565 was 1.81× — handover HANDOVER_POST_S0380_73_APPENDIX_H_BLOCKED.md concluded this was blocked on BS EN 15316-4-3:2017 access. That framing was wrong. The answer lives in the SAP 10.2 spec itself, in the cross-reference between (H7) and Appendix U §U3.3 that page 76 makes verbatim.

The diagnostic that closed the trap:

1. 3 new solar-HW worksheets generated by the user at sap worksheets/Solar PV tests/ (A-baseline-south-modest, B-highY, C-lowY) — pooled with cert 000565 → 48 month-observations.
2. Empirical fit attempts (Klein 6-coef refit, 9-coef extended with XY interactions, multiplicative Y/X correction) all rejected — none matched the worksheet's polynomial output without sign flips or overfitting.
3. ChatGPT-mediated documentary research ruled out hidden BRE errata, SBEM-style Method 2 corrections, EN-style Im definitions in W/m². Identified SAP 10.2 internal p.75 vs p.77 inconsistency over the H8 overshading factor (real but wrong direction).
4. Back-solving the polynomial at fixed X for Y_eff across 24 worksheet-positive observations revealed Y_eff/Y_cascade took ONLY two distinct values: 0.7200 (exact) for 30-day months, 0.7440 (exact) for 31-day months — i.e., days × 24 / 1000 exactly. No utilizability function, no missing constant — a per-month unit-conversion factor.
5. Spec text resolution: SAP 10.2 p.76 reads (H7)m as "Monthly solar radiation per m² from U3.3 in Appendix U". §U3.3 (p.130) defines the conversion S_monthly = 0.024 × n_m × S(orient,p,m) — i.e. kWh/m²/month, NOT W/m². The cascade's surface_solar_ flux_w_per_m2 returns the §U3.2 24h-avg flux in W/m² (verified bit-exact against worksheet line 295: SE 90° Jan region 0 = 36.7938 W/m²). Equation H1 expected the U3.3 monthly integrated value. The page-77 (H23) formula's × hours / 1000 term double-converts when (H7) is W/m² instead of kWh/m²/month.

Full diagnostic in BRIEF_APPENDIX_H_EN_15316_RESEARCH.md §"Closure — 4-cert empirical investigation (2026-05-29)".

Cert 000565 state (HEAD a532f75d)

Pin	Cascade	Worksheet	Δ	Root cause
sap_score (int)	29	29	0 ✓ EXACT	unchanged
sap_score_continuous	29.2905	28.5087	+0.7818	downstream of HW + space_heating
ecf	5.3073	5.3866	−0.0793	downstream
total_fuel_cost_gbp	4611.14	4680.26	−69.12	downstream
co2_kg_per_yr	6352.61	6447.63	−95.02	downstream
space_heating_kwh	59274.46	59008.35	+266.11	RR fold-in (RdSAP §3.10 detailed-RR geometry)
main_heating_fuel	34867.33	34710.79	+156.53	follows space_heating (Δ × 1/COP)
hot_water_kwh	3668.54	3755.03	−86.49	demand cascade gaps (see below)
lighting	1387.02	1384.84	+2.19	sub-spec
pumps_fans	255.00	252.52	+2.48	MEV PCDB record missing

Appendix H solar Q_s — DONE (spec-pinned)

Quantity	Cascade	Worksheet	Δ
Solar Q_s annual	283.08	281.35	+1.73 kWh (0.6%)
Solar Q_s monthly	various	various	<1e-4 kWh per month for 47/48 observations across 4 fixtures

The orchestrator is now spec-pinned. Remaining HW pin gap is in the demand cascade, not in solar.

Open thread #1 — Cert 000565 HW demand cascade gaps (3 bugs)

The −86 kWh HW residual is the net of three independent demand-cascade bugs that were previously masked by the +357 kWh "no solar credit" over-count:

A) Primary loss (59)m missing — biggest single fix (+1175 kWh)

For HP main + HW cylinder routing, the cascade leaves primary_loss_monthly_kwh = 0. Worksheet line (59)m for cert 000565 sums to 1174.79 kWh/yr. SAP 10.2 §4 line 7700 + Table 3 (PDF p.159) defines primary loss for indirect cylinders. The current _primary_loss_override helper at cert_to_inputs.py:3322 gates on a _primary_loss_applies(main, cylinder_present, hp_record) check that returns False for cert 000565 — likely because the HP main has integral vessel info in PCDB but cert 000565's HP route is to an EXTERNAL cylinder (not integral). Audit _primary_loss_applies against SAP 10.2 §4 line 7700 specifically for the HP + external cylinder case.

Closing this would shift cert 000565 HW pin from −86 → −86+1175 = +1089 (over-shoot, but then the next two fixes bring it back).

B) (45)m energy_content over by 903 kWh

Component	Cascade	Worksheet
(45)m sum	2189	1286
(62)m sum	3181	3060

Cascade energy_content_monthly_kwh over-counts by 903 kWh/yr. Likely candidates:

• Occupancy formula difference (cert 000565 TFA 319.91 m², worksheet uses occupancy from line 42)
• Hot water demand per occupant — cert lodges "showers are both electric and non-electric" per RdSAP Table 29 default for solar systems, which may reduce the non-electric shower demand

Audit assumed_occupancy() and the (42)-(45) cascade vs worksheet lines for cert 000565 specifically.

C) Storage loss (56) over by 98 kWh + missing (57)m solar adjustment

Cascade (56) = 992 vs worksheet (56) = 894 (Δ +98).

Worksheet additionally computes (57)m = (56) × (H13−H12)/H13 — the solar-adjusted storage loss. For cert 000565: (57) = 596.91 (vs (56) = 893.95). When solar HW is present, (62)m uses (57) NOT (56). Cascade currently passes (56) as solar_storage_monthly_kwh_override without the (H13−H12)/H13 reduction. Fix:

• Audit cascade's storage loss formula at cert_to_inputs.py:3289 _cylinder_storage_loss_override — what's adding the extra 98?
• Add (57)m solar adjustment when solar HW is present. The multiplier is (cylinder_volume_l - dedicated_solar_storage_l) / cylinder_volume_l derivable from the existing _hot_water_cylinder_volume_l(epc) helper and the same H12 = cylinder_volume / 3 rule used in _solar_hw_monthly_override.

Total impact estimate

If all three close: HW pin = −86 + 1175 (A) − 903 (B) − 396 (C: 98

• 298. = −210 before re-pinning solar Q_s under the corrected (62)m. The Q_s itself would shift slightly since (H17)m changes. Expect HW pin to land within ±50 kWh of zero after all three + re-equilibrium.

Open thread #2 — Cert 000565 RR fold-in (space_heating +266)

Unchanged. RdSAP §3.10 detailed-RR geometry / area formula not in repo. Largest energy residual after HW closes. Was already noted in predecessor handover.

Open thread #3 — Cert 000565 MEV pumps_fans (+2.48)

Unchanged. PCDB MEV record not in repo (cert lodges PCDF 500755). Acquiring the PCDB MEV table is the gating step.

Open thread #4 — 12 gas-combi PV certs at +0.5..+1.6 PE

Unchanged. S0380.73 cooking fix surfaced these — no worksheets available for these certs. Re-pinned at current residuals.

Open thread #5 — 5 SAP-integer-residual certs

Unchanged. All API-only (no worksheets). User has agreed not to chase these without worksheet ground truth.

How to run the baseline

PYTHONPATH=/workspaces/model python -m pytest \
    backend/documents_parser/tests/test_summary_pdf_mapper_chain.py \
    backend/documents_parser/tests/test_elmhurst_extractor.py \
    backend/documents_parser/tests/test_elmhurst_end_to_end.py \
    domain/sap10_calculator/worksheet/tests/test_e2e_elmhurst_sap_score.py \
    domain/sap10_calculator/worksheet/tests/test_appendix_h_solar.py \
    domain/sap10_calculator/rdsap/tests/test_cert_to_inputs.py \
    domain/sap10_calculator/rdsap/tests/test_golden_fixtures.py \
    --no-cov -q

Expected: 547 pass + 9 expected test_sap_result_pin[000565-*] fails.

Recommended next slice — primary_loss (59)m for HP + external cylinder

Biggest single residual: +1175 kWh. Audit _primary_loss_applies(main, cylinder_present, hp_record) at cert_to_inputs.py against SAP 10.2 §4 line 7700. The HP + external cylinder case (cert 000565 shape: HP main 1 + gas combi main 2 servicing DHW via WHC 914 + cylinder present) likely falls outside the current gate. Test by adding the HP-external-cylinder path and verifying the primary_loss matches worksheet line (59)m for cert 000565.

What NOT to do

• Don't re-investigate the Appendix H 1.81× over-count. Closed.
• Don't propose more Appendix H polynomial / utilizability fixes. S0380.74's U3.3 fix is the correct answer.
• Don't try to close cert 000565's HW pin in one slice. The −86 is the net of three independent demand-cascade bugs; each closes a separate residual.
• Don't widen pin tolerances or xfail residual gaps (feedback-zero-error-strict). The 9 cert 000565 fails are the work queue.
• Don't reference SAP 10.3 (feedback-sap-10-2-only-never-10-3).

Standard workflow per slice

1. Read SAP 10.2 spec page for the change — quote it in commit
2. Probe current cascade output, identify exact spec-vs-cascade gap
3. Write failing test FIRST (AAA structure)
4. Implement helper / change
5. Verify test passes
6. Run full handover suite (command above)
7. Check pyright on touched files — net-zero from baseline
8. Commit with spec citation
9. Update relevant memory if state changed

Files touched this session

File	Slice	Change
domain/sap10_calculator/worksheet/appendix_h_solar.py	S0380.74	Rename monthly_solar_energy_available_h9_w → _h9_kwh_per_month, add hours_in_month param, apply U3.3 conversion
domain/sap10_calculator/worksheet/tests/test_appendix_h_solar.py	S0380.74	Cert 000565 (H24)m magnitude pin at abs < 1e-3 kWh; H9 + Y23 unit tests updated for kWh/month units
domain/sap10_calculator/docs/BRIEF_APPENDIX_H_EN_15316_RESEARCH.md	S0380.74	New "Closure" section with empirical evidence + root cause
domain/sap10_calculator/docs/HANDOVER_POST_4_CERT_EMPIRICAL.md	S0380.74	NEW — closure handover (points to brief)
datatypes/epc/surveys/elmhurst_site_notes.py	S0380.75	Renewables gains solar_hw_collector_orientation / _pitch_deg / _overshading
datatypes/epc/domain/epc_property_data.py	S0380.75	Same three fields added
datatypes/epc/domain/mapper.py	S0380.75	from_elmhurst_site_notes propagates the three new fields
backend/documents_parser/elmhurst_extractor.py	S0380.75	§16.0 section parsing
domain/sap10_calculator/worksheet/water_heating.py	S0380.75	solar_water_heating_monthly_kwh_override param on water_heating_from_cert
domain/sap10_calculator/rdsap/cert_to_inputs.py	S0380.75, S0380.76	Table 29 constants + _solar_hw_monthly_override + _orientation_from_summary_string + _hot_water_cylinder_volume_l + combined-cylinder H12/H13 derivation; demand_pass intermediate call

Spec source quick-reference

• SAP 10.2 full specification: domain/sap10_calculator/docs/specs/sap-10-2-full-specification-2025-03-14.pdf
  • Appendix H (solar thermal): p.74-78
    • Equation H1 + Y/X definitions: p.75
    • (H7) "from U3.3" + (H9) formula: p.76
    • (H22)-(H24) worksheet: p.77
    • Table H1 (collector params), Table H2 (overshading), Table H3 (coefficients): p.78
  • Appendix U §U3.2 (W/m² flux polynomial): p.128
  • Appendix U §U3.3 (kWh/m²/month integrated): p.130
  • §4 line 7700 + Table 3 (primary loss): p.159
  • Table 12d/12e (monthly electric factors): p.195-196
• RdSAP 10 specification: domain/sap10_calculator/docs/specs/rdsap-10-specification.pdf
  • §10.5 Table 28 (cylinder size codes → litres): p.[lookup]
  • §10.11 Table 29 (solar panel defaults): p.58
• S10TP-04 (BRE Appendix H change note): domain/sap10_calculator/docs/specs/sap10 technical papers/S10TP-04 - Change to Appendix H to include solar space heating - V1_3.pdf
• SAP 10.3 at domain/sap10_calculator/docs/specs/sap-10-3-full-specification-2026-01-13.pdf: DO NOT reference (project tracks 10.2 only per feedback-sap-10-2-only-never-10-3)

Memory updated this session

• project_cert_000565_recovery_state — Appendix H closure + S0380.75/76 outcomes
• MEMORY.md — index entry refreshed