junte/Model
1
0
Fork 0
mirror of https://github.com/Hestia-Homes/Model.git synced 2026-06-08 11:17:27 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

docs: handover for Table 3c two-profile combi loss → close 4 Elmhurst fixtures

Browse source 
Rewrites HANDOVER_NEXT.md for the next agent. Two-ticket sequence:

1. Table 3c (immediate): implement SAP10.2 Appendix J §J3 two-profile
   combi-loss formula + route PCDB records with separate_dhw_tests=2
   through it. Closes 000477/000480/000487/000516 from SAP delta
   +1/+12/+11/+12 to delta=0. Currently those fall through to Table 3a
   keep-hot 600 kWh/yr default = ~25× overshoot.
2. RdSAP API integration test (end-state): real RdSAP10 API response
   → EpcPropertyDataMapper → cert_to_inputs → SAP integer == lodged.
   User generating exotic fixtures to pressure-test first.

SPEC_COVERAGE §4 row updated to call out the Table 3c gap. ADR-0010
gains a "Cohort residual hunt + SAP 10.2 rating constants" amendment
documenting the 5 component closures (secondary heating, ventilation
cert lodgement, Table 4f pumps_fans, SAP 10.2 rating constants,
000477 partial) and naming the deferred Table 3c work.

Carries a PCDF parser concern: raw row at index 52 has 13.729 which
looks like F2-annual-kWh but parser reads F2 from fields[55] = 0.0.
Verify field positions per BRE PCDF Spec §7.11 before assuming F2=0.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Khalim Conn-Kowlessar 2026-05-22 12:14:00 +00:00
parent 960419a901
commit 6c966ffe2b
3 changed files with 206 additions and 195 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

25
docs/adr/0010-sap10-calculator-spec-target-and-validation.md
View file

@ -127,3 +127,28 @@ Two engine bugs surfaced during the wire-up:
- **000490 / cohort SAP-integer closure (residual hunt).** Next ticket. Suspects above. Driven by user's next batch of test fixtures (battle-testing the engine) → emergent residual identification.
- **`predicted_lighting_kwh` deletion.** Future cleanup ticket once `domain.ml.ecf` + `domain.ml.transform` are off the legacy heuristic.
- **RdSAP10 → API integration test.** End-state e2e harness: RdSAP API response → `cert_to_inputs` → `calculate_sap_from_inputs` → SAP integer = lodged integer. Once enough cohort fixtures pass delta=0 on isolated components.
## Amendment — Cohort residual hunt + SAP 10.2 rating constants (2026-05-22)
The post-Appendix-L 000490 residual (SAP delta +6, cost -£104) closed in four micro-cycles after a per-component diagnostic walk down the spec cascade. Five engine pieces landed end-to-end:
1. **Secondary heating cascade** (`607e52a3`): cert lodges SAP code 691 (Electricity Electric Panel, 100% efficiency); build_epc wasn't passing it through. Closes -£104 on 000490.
2. **Ventilation cert lodgement** (`af6fcfb1`): `SapVentilation` schema gains 4 new fields (`sheltered_sides`, `has_suspended_timber_floor`, `suspended_timber_floor_sealed`, `has_draught_lobby`). `cert_to_inputs` now reads them. Removes a long-standing `sheltered_sides=2` hardcode + 4 TODOs. All 6 fixtures' (25)m monthly effective ACH closes to U985 PDF at abs=1e-3 (72 assertions).
3. **Table 4f gas-combi pumps_fans** (`b536b46a`): keyed by `main_heating_category`. Category 2 (gas boilers) → 115 kWh pump + 45 kWh flue fan = 160 kWh/yr. Other categories still on the legacy 130 sentinel.
4. **SAP 10.2 rating constants** (`a41ac6bd`): `worksheet/rating.py` was using SAP 10.3 constants (deflator 0.36, slope 16.21/120.5). Per ADR-0010 §1 active spec target IS SAP 10.2 (14-03-2025). Restored SAP 10.2 values: **deflator 0.42**, linear branch slope **13.95**, log branch intercept **117**, log slope **121**. The two errors were near-cancelling for the Elmhurst combi-gas cohort (low-cost dwellings on the linear branch).
5. **000477 build_epc lodgement (partial — Table 3c blocker)** (`960419a9`): mirrors the Appendix L slice 2 fix on 000477 (lodge windows + bulbs + PCDB index + secondary 691 + number_baths=0). Closes 000477 SAP delta from +6 to +1. Remaining +1 blocked by Table 3c (next ticket).
### Consequences
- **000474 + 000490 both hit SAP integer delta=0**. First two Elmhurst fixtures across the rdsap engine integration gate. 685 tests pass + 1 xfail (000477 pending Table 3c).
- **Per-component pins now landed**: lighting kWh, monthly infiltration ACH, secondary heating fuel, pumps_fans, plus the pre-existing §4 HW + §5 + §6 + §7 + §8 + §10a sections.
- 000477 cost residual -3.5% remaining is the Table 3c 600-kWh-overshoot on combi-loss.
- 000480/000487/000516 still at SAP delta +11/+12 because their build_epc lodgement is also incomplete (mirror the 000477 fix). Their PCDB records (16839/18119/18118) also have `separate_dhw_tests=2` for sustain models → Table 3c blocker.
### Deferred work (named in cohort slice 5)
- **Table 3c two-profile combi-loss override** — Next ticket. SAP10.2 Appendix J §J3. Blocks 000477/000480/000487/000516 closure.
- **Build_epc lodgement on 000480/000487/000516** — Same pattern as 000477 (windows + bulbs + PCDB index + secondary 691 + number_baths). Lands with the Table 3c ticket since SAP closure requires both.
- **RdSAP API integration test** — End-state validation gate. User generating exotic fixtures to pressure-test first.
- **§12a CO2 + §13a PE per-component pins** — Engine produces `result.co2_kg_per_yr` and `result.primary_energy_kwh_per_m2`. Not yet validated against U985 (272) + (282) for any fixture.
- **PCDF field-position audit**: parser reads F2 from fields[55]. PCDB 18118 raw row has 13.729 at index 52 — unclear which field that maps to per BRE PCDF Spec §7.11. Verify before assuming F2=0 is the lodged value.

374
docs/sap-spec/HANDOVER_NEXT.md
View file

@ -1,4 +1,4 @@
# Handover — Appendix L lighting closure → §11a / §12a / §13a sweep
# Handover — Table 3c two-profile combi-loss → close 000477/000480/000487/000516 to delta=0
**For the agent picking up the next chunk of work.** Read this BEFORE invoking `/grill-me`. Read all of it. Caveman mode is the house style — terse, technical, no filler.
@ -6,256 +6,235 @@ Owner: `khalim@domna.homes`. Branch: `ara-backend-design-prd`.
Two tickets in priority order:
1. **Immediate — Appendix L lighting predictor swap.** Closes the residual +9.2% cost gap on 000474 that §4 HW exposed. ~1 slice, ~50 LoC.
2. **Section sweep — §11a / §12a / §13a "Individual heating systems incl micro-CHP".** Worksheet line refs (256)..(272). Builds the rating + CO2 + primary-energy per-end-use cascade on top of the §10a fuel-cost orchestrator. ~3 slices.
1. **Immediate — Table 3c two-profile combi-loss override.** Closes the +£20–25 cost residual (and +1 SAP integer delta) on every Elmhurst fixture whose PCDB record lodges `separate_dhw_tests=2` (Vaillant ecoTEC sustain 24/28 — affects 000477, 000480, 000487, 000516). Without this, those certs fall through to the Table 3a "keep-hot time-clock" 600 kWh/yr default = ~25× overshoot vs spec-faithful ~24 kWh/yr.
2. **Next — RdSAP API integration test.** End-state e2e harness: real RdSAP10 API response → `EpcPropertyDataMapper.from_api_response` → `cert_to_inputs` → `calculate_sap_from_inputs` → assert SAP integer = lodged integer. The user is generating an exotic worksheet to pressure-test before this lands.
Hard rules (unchanged):
- **Caveman mode** house style.
- **Tolerance**: don't loosen test tolerances to make them pass. If a refactor can't hit the locked tolerance, pause and ask the user.
- **Tolerance**: don't loosen test tolerances to mask drift. If a refactor can't hit the locked tolerance, pause and ask.
- **Spec PDFs**: don't scan more than ~50 lines without checking with the user.
- **Commit per slice**, one slice = one commit, AAA test convention (`# Arrange / # Act / # Assert`), `Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>` trailer.
- **Commit per slice**, AAA test convention (`# Arrange / # Act / # Assert`), `Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>` trailer.
- **Don't widen ceilings to hide bugs.** Per `feedback-e2e-validation-philosophy` memory: component pins at <abs=1e-4 against U985 line refs; SAP integer = PDF integer is the integration gate.
---
## §A — Current state (what just shipped)
## §A — Current state on `ara-backend-design-prd`
Last commits on this branch:
Last commits (most-recent-first):
```
c9eb231a §4 HW slice 3: docs — SPEC_COVERAGE row + Remaining work + golden note
02fc9e4d §4 HW slice 2: Equation D1 monthly water-eff cascade
760e25de §4 HW slice 1: PCDB Table 3b combi-loss override
ae8c9461 docs: §10a slice 3 — ADR-0010 amendment + SPEC_COVERAGE row
adfa7f60 §10a slice 2: cert_to_inputs._fuel_cost + calculator delegation
0f255165 §10a slice 1: table_32 + table_12a + fuel_cost orchestrator
6d6767ce docs: handover for §10a Fuel costs + SPEC_COVERAGE PCDB-followup updates
960419a9 Cohort residual slice 5: 000477 build_epc lodgement (partial — Table 3c blocker)
a41ac6bd Cohort residual slice 4: SAP 10.2 rating constants — 000490 closes to delta=0
b536b46a Cohort residual slice 3: Table 4f gas-combi pumps_fans = 160 kWh/yr
af6fcfb1 Cohort residual slice 2: cert→ventilation cascade closes useful kWh on all 6 fixtures
607e52a3 Cohort residual slice 1: 000490 secondary heating cascade closes -£104 cost gap
fd9df9e5 Appendix L slice 3: docs — SPEC_COVERAGE + ADR-0010 amendment + heuristic deprecation note
54cc9bd3 Appendix L slice 2: cert→cascade lighting kWh + 000474 e2e closes to delta=0
f4352587 Appendix L slice 1: annual_lighting_kwh extraction
```
486 tests passing on the domain package. Sections fully implemented: §1, §3, §4 (combi-gas + PCDB Table 3b + Eq D1), §5, §6, §7, §8, §8c, §8f, §9a (single-main), §10a (RdSAP10 Table 32 + Table 12a + note (a) standing), §13 SAP-rating equation. PCDB Tables 105 (gas/oil boilers) + combi-loss fields wired end-to-end.
**685 tests pass + 1 xfail (strict) on 000477 SAP integer pending Table 3c.** No xfails outside the named Table 3c blocker.
**000474 + 000490 post-§4 HW residuals:**
**Six engine components closed end-to-end with U985 worksheet pins:**
| metric | 000474 actual | 000474 PDF | 000490 actual | 000490 PDF |
| component | pin | tolerance | scope |
|---|---|---|---|
| Appendix L lighting | `(232)` annual kWh | abs=1e-4 | all 6 fixtures |
| Ventilation infiltration | `(25)m` monthly ACH | abs=1e-3 | all 6 fixtures, 72 assertions |
| Hot water demand | `(64)m` + `(65)m` + `(219)` | ≤1e-2 / ≤0.1% | all 6 fixtures (§4 conformance) |
| Secondary heating | `(215)` annual kWh | abs=0.1 | 000490 (the only Elmhurst with secondary) |
| Pumps/fans Table 4f | `(231)` annual kWh | abs=1e-3 | 000474, 000490 (gas-combi cat 2) |
| §10a fuel cost | `(255)` total cost | rel=0.05 | 000490 (was xfail, now passes) |
**SAP integer status (the rdsap engine integration gate):**
| fixture | actual SAP | PDF SAP | Δ | notes |
|---|---|---|---|---|
| SAP score | 59 | 62 (Δ-3) | 60 | 57 (Δ+3) |
| main_heating_fuel kWh | 12304.8 | ~11968 (+2.8%) | 13001.3 | 13003.85 (≤0.1%) |
| **HW fuel kWh** | **2291.6** ✓ | **2291.78 (≤0.1%)** | **2846.8** ✓ | **2850.57 (≤0.1%)** |
| lighting kWh | 528.1 | ~169 (back-derived) | ? (similar overshoot) | ? |
| total_fuel_cost £ | 715.83 | 655.69 (+9.2%) | 769.70 | 807.54 (−4.7%) |
| e2e SAP ceiling | 3 (was 4 post-§10a) | — | 3 (was 2 post-§10a) | — |
| 000474 | 62 | 62 | **0** ✓ | fully lodged + closed |
| 000477 | 66 | 65 | **+1** (xfail) | needs Table 3c |
| 000480 | 73 | 61 | **+12** | needs build_epc lodgement + Table 3c |
| 000487 | 73 | 62 | **+11** | needs build_epc lodgement + Table 3c |
| 000490 | 57 | 57 | **0** ✓ | fully lodged + closed |
| 000516 | 75 | 63 | **+12** | needs build_epc lodgement + Table 3c |
Two residuals remain:
- **000474 lighting overshoots ~3x** (528 vs ~169 back-derived). This is the dominant remaining cost residual — drives the +£60 over PDF.
- **000490 cost residual −4.7%** is spec-version drift per ADR-0010 §3 Validation Cohort (cert pre-dates 14-Mar-2025 amendment; PDF cost embeds older cert-assessor prices). HW kWh closure is the spec-faithful direction.
000474 + 000490 hit delta=0. The other 4 need both Table 3c AND build_epc lodgement.
---
## §B — Ticket 1: Appendix L lighting predictor swap
## §B — Ticket 1: Table 3c two-profile combi-loss override
### B.1 Mission
Replace the legacy `domain.ml.demand.predicted_lighting_kwh` (heuristic `9.3 × TFA × (1 − reduction)`) with the **spec-faithful Appendix L L1-L12 annual lighting kWh** that already exists in `domain.sap.worksheet.internal_gains`. Single slice. Closes 000474 cost residual from +9.2% toward ~0%.
Implement SAP10.2 Appendix J Table 3c (Profile M + Profile L two-profile combi-loss formula) and route PCDB records with `separate_dhw_tests=2` through it. Currently `_pcdb_table_3b_combi_loss_override` at [cert_to_inputs.py:725](packages/domain/src/domain/sap/rdsap/cert_to_inputs.py#L725) rejects them and they fall to Table 3a "keep-hot time-clock" = 600 kWh/yr default. Target: <abs=1.0 kWh annual per-fixture vs lodged LINE_61.
### B.2 Why
The Appendix L cascade is already implemented correctly for the **§5 internal-gains side** (used to compute (67) lighting gains as monthly watts). But the **cost side** (`inputs.lighting_kwh_per_yr`) reads the heuristic `predicted_lighting_kwh`. They diverge by ~3x on 000474.
Vaillant ecoTEC sustain models (and most modern multi-test combis) lodge `separate_dhw_tests=2`. This is the modal PCDB combi configuration in the UK cert corpus going forward. Without Table 3c our cert→SAP integration is ~600 kWh/yr too high on combi loss → wrong HW fuel kWh → wrong cost → wrong SAP integer.
For 000474: TFA=71.55, N=1.8896, 8 low-energy bulbs (LED/CFL undistinguished).
- Heuristic: `9.3 × 71.55 × (1 − 0.45) ≈ 366` kWh/yr (with rough LED+CFL share assumption) → actual 528 lodged.
- Spec Appendix L cascade with daylight factor + λ_b + λ_req + topup: ~169 kWh/yr.
### B.3 Per-fixture combi-loss residuals (current state)
### B.3 Current state in code
| fixture | PCDB | separate_dhw_tests | LINE_61 (PDF) | Our combi loss | overshoot |
|---|---|---|---|---|---|
| 000474 | 16839 (ecoTEC pro 28) | 1 (Table 3b row 1 ✓) | 337.19 | 337.19 | 0 |
| 000477 | 18118 (ecoTEC sustain 24) | **2** | 24.35 | 600 | +575 |
| 000480 | 16839 (ecoTEC pro 28) | 1 | needs check | — | check first |
| 000487 | 18119 (ecoTEC sustain 28) | **2** | needs check | — | check first |
| 000490 | 10328 (Ecotec Pro 28) | 1 | 337.19 | 337.19 | 0 |
| 000516 | 18118 (ecoTEC sustain 24) | **2** | needs check | — | check first |
[packages/domain/src/domain/ml/demand.py:222-243](packages/domain/src/domain/ml/demand.py#L222-L243) — `predicted_lighting_kwh` heuristic. Used by cert_to_inputs to populate `CalculatorInputs.lighting_kwh_per_yr`.
Confirm `separate_dhw_tests` value for 000480/000487/000516 via `gas_oil_boiler_record(pcdb_id).separate_dhw_tests`.
[packages/domain/src/domain/sap/worksheet/internal_gains.py:208-265](packages/domain/src/domain/sap/worksheet/internal_gains.py#L208-L265) — `_lighting_gains_monthly_w` (private) builds `e_l_annual_kwh` (line 253) per Appendix L L1-L12, but **doesn't expose it**. The annual kWh figure is computed then converted to monthly W gains.
### B.4 Spec anchors
Search for `e_l_annual_kwh` to confirm — it's a local variable inside the gains function.
- **SAP10.2 spec PDF**: `docs/sap-spec/sap-10-2-full-specification-2025-03-14.pdf`. Appendix J §J3 — Table 3c. Ask the user for the specific page range (handover §F caps spec scanning at ~50 lines without permission).
- **BRE PCDF Spec v1.0 §7.11**: field layout for separate_dhw_tests + F1/F2/F3/R1/R2/F3'. The parser at [parser.py:165-168](packages/domain/src/domain/sap/tables/pcdb/parser.py#L165-L168) reads R1=fields[50], F1=fields[51], F2=fields[55], F3=fields[56]. The PCDF spec PDF (separate from SAP10.2) defines the exact column meanings.
- **PCDF parser concern**: PCDB record 18118 raw row has `13.729` at field index 52 (between F1 at 51 and F2 at 55). That value looks like F2 in annual kWh (or maybe a different field — annual vs daily). Parser currently treats fields[52] as ignored; F2 read from fields[55] = 0.0. **Verify the parser field positions match the PCDF spec before assuming F2=0 is the lodged value.**
### B.4 Likely shape
### B.5 Likely shape
```python
# packages/domain/src/domain/sap/worksheet/internal_gains.py
# packages/domain/src/domain/sap/worksheet/water_heating.py
def annual_lighting_kwh(
def combi_loss_monthly_kwh_table_3c_two_profile_instantaneous(
*,
total_floor_area_m2: float,
n_occupants: float,
fixed_lighting_capacity_lm: float,
fixed_lighting_efficacy_lm_per_w: float,
daylight_factor: float,
) -> float:
"""SAP 10.2 Appendix L L1-L12 — annual lighting kWh/yr (cost side).
Mirrors the kWh internal to `_lighting_gains_monthly_w` but surfaces
it for `inputs.lighting_kwh_per_yr` to consume."""
# extract the e_l_annual_kwh derivation; either factor it out into
# this fn and have the gains fn call it, OR have both fns share
# a small `_lighting_annual_kwh_components` helper.
# Profile M test data
rejected_energy_proportion_r1: float, # R1 (M profile)
loss_factor_f1_kwh_per_day: float, # F1 (M profile)
# Profile L test data
rejected_energy_proportion_r2: float, # R2 (L profile)
loss_factor_f2_kwh_per_day: float, # F2 (L profile)
# Per-litre rejected factor (applies to both profiles)
rejected_factor_f3_per_litre: float, # F3
# Worksheet bootstrap inputs
energy_content_monthly_kwh: tuple[float, ...],
daily_hot_water_monthly_l_per_day: tuple[float, ...],
) -> tuple[float, ...]:
"""SAP 10.2 Appendix J §J3 Table 3c — two-profile combi loss.
Formula: ... [spec needs to be read for exact equation]
"""
...
```
Then `cert_to_inputs` swaps the call:
Then extend `_pcdb_table_3b_combi_loss_override` (or rename + split):
```python
# Before:
lighting_kwh = predicted_lighting_kwh(
total_floor_area_m2=epc.total_floor_area_m2,
cfl_count=epc.cfl_fixed_lighting_bulbs_count,
led_count=epc.led_fixed_lighting_bulbs_count,
incandescent_count=epc.incandescent_fixed_lighting_bulbs_count,
)
# After: reuse the §5 cascade inputs (already computed for internal_gains_from_cert).
lighting_kwh = annual_lighting_kwh(
total_floor_area_m2=epc.total_floor_area_m2,
n_occupants=..., # Appendix J Table 1b (already computed for §4)
fixed_lighting_capacity_lm=..., # cert-derived, mirror §5 path
fixed_lighting_efficacy_lm_per_w=..., # cert-derived, mirror §5 path
daylight_factor=..., # L2a/L2b — already computed for §5
)
def _pcdb_combi_loss_override(pcdb_record, ...):
if pcdb_record.separate_dhw_tests == 1:
return combi_loss_monthly_kwh_table_3b_row_1_instantaneous(...)
if pcdb_record.separate_dhw_tests == 2:
# Table 3c path
return combi_loss_monthly_kwh_table_3c_two_profile_instantaneous(...)
return None # fall through to Table 3a
```
The cert→Appendix-L-inputs derivation already lives in `internal_gains_from_cert` (called from cert_to_inputs at line ~1146). Reuse it.
### B.5 Slice plan
### B.6 Slice plan
```
S1 — annual_lighting_kwh free function in worksheet/internal_gains.py
(factor out e_l_annual_kwh from _lighting_gains_monthly_w; both
callers share the derivation). Synthetic unit test pins 000474
PDF value (~169 kWh/yr) given the cert's bulb counts.
S2 — cert_to_inputs swaps lighting_kwh derivation from predicted_lighting_
kwh to annual_lighting_kwh. Two ALL_FIXTURES e2e closures: 000474
cost residual closes +9.2% → ~0% (ceiling 3 → 1 or below);
000490 ceiling re-checks.
S3 — docs: SPEC_COVERAGE Appendix L row + ADR-0010 amendment if needed.
S1 — Verify PCDF field positions. Read BRE PCDF Spec §7.11 carefully.
If the parser is wrong, fix it + add a test cross-checking the
raw row → parsed fields mapping for PCDB 18118 (raw[52]=13.729
should be... what?).
S2 — Synthetic Table 3c test. Hand-compute LINE_61 for PCDB 18118 on
a known fixture (000477). Pin annual combi-loss to ~24 kWh ± 1.
RED.
S3 — Implement Table 3c orchestrator in water_heating.py. GREEN.
S4 — Extend cert_to_inputs gate to route separate_dhw_tests=2 through
Table 3c. RED→GREEN on the 4-fixture parametrized e2e SAP integer
test (added in S5).
S5 — Lodge build_epc fields on 000480/000487/000516 (mirror 000477's
pattern: windows + bulbs + PCDB index + secondary 691 + number_
baths). Add parametrized e2e SAP integer pin for all 4.
S6 — Remove xfail on 000477. Tighten ceilings.
S7 — Docs (SPEC_COVERAGE Table 3c row, ADR-0010 amendment if needed).
```
### B.6 Tests
### B.7 Tests
- Synthetic: `annual_lighting_kwh(TFA=71.55, N=1.8896, C_L_fixed=..., ε_fixed=..., D=...) == 169 ± 5` against a hand-computed Appendix L example.
- ALL_FIXTURES: 6 fixtures already pin lighting kWh tuples via §5 — those don't change. The cost-side closure shows in `inputs.lighting_kwh_per_yr` matching the §5 internal `e_l_annual_kwh` exactly.
- e2e: 000474 SAP score ceiling tightens 3 → 1 (or below).
- **Synthetic** (S2): `combi_loss_monthly_kwh_table_3c_two_profile_instantaneous(R1=0.015, F1=0.73143, R2=?, F2=?, F3=0.00014, ...)` for a hand-computed dwelling.
- **PCDB integration**: `_pcdb_combi_loss_override(pcdb_18118, ...)` returns ~24 kWh/yr for 000477's energy_content / daily_hot_water inputs.
- **e2e**: `test_elmhurst_000477_end_to_end_sap_score_matches_pdf` un-xfailed; same for 000480/000487/000516.
### B.7 Don't list
### B.8 Don't list
- Don't delete `predicted_lighting_kwh` immediately — leave it in `domain.ml.demand` with a deprecation comment in case external callers rely on it. Future slice rips it.
- Don't change the §5 lighting GAINS path. The kWh figure feeds both gains and cost — just expose it.
- Don't shoehorn Table 3c into the Table 3b helper — they're distinct formulas. Keep separate functions.
- Don't change Table 3a "keep-hot" default — that's spec-correct for combis WITH keep-hot. Just route PCDB records away from it when test data is available.
- Don't scan more than ~50 lines of SAP10.2 spec PDF without checking with the user.
---
## §C — Ticket 2: §11a / §12a / §13a "Individual heating systems incl micro-CHP"
## §C — Ticket 2: RdSAP API integration test (end-state validation)
### C.1 Mission
Build the per-end-use rating + CO2 + primary-energy cascade on top of the §10a `FuelCostResult`. SAP 10.2 worksheet line refs:
End-to-end harness from a real RdSAP10 API response → `EpcPropertyDataMapper.from_api_response(api_json)` → `cert_to_inputs(epc)` → `calculate_sap_from_inputs(inputs)` → assert `result.sap_score == api_json["sap_rating_current"]` (or equivalent lodged field).
- **§11a SAP rating** — (256), (257), (258). Spec lines ~8085-8110.
- **§12a CO2 emissions** — (259)..(265). Spec lines ~8110-8150. Per-end-use CO2 mirroring §10a's (240)..(250) cost structure.
- **§13a Primary Energy** — (266)..(272). Same per-end-use mirror but with primary-energy factors.
The user is generating exotic test fixtures to pressure-test the engine before this lands. After cohort closure on the 6 Elmhurst fixtures (delta=0 each), this is the user's validation gate.
The calculator currently aggregates these inline (calculator.py:441-505 has `co2 = main_heating_co2 + secondary_heating_co2 + ...` and `primary_energy_kwh = max(0, space_heating_primary_kwh + ... − pv_primary_offset_kwh)`). The cascade is correct at aggregate level but does NOT produce the per-end-use line refs in worksheet shape.
### C.2 Existing scaffolding
Worksheet-shape fidelity rule (memory `feedback_worksheet_shape_fidelity`): mirror SAP10.2 sections completely with per-line dataclasses, even when no consumer needs the per-line detail.
### C.2 SAP10.2 spec anchors
User needs to point at PDF pages — likely around pages 32-38 for §11a/§12a/§13a (after §10 fuel costs at pages 29-32). The xlsx at the repo root maps line refs to cells.
- `EpcPropertyDataMapper.from_api_response(...)` already exists ([packages/domain/.../mapper.py](../../packages/domain/src/datatypes/epc/domain/mapper.py)).
- `test_golden_fixtures.py` already calls this on 4 non-Elmhurst golden certs, but PE tolerance was widened 30→35 to absorb the Appendix L closure on non-Elmhurst PE residuals (still on the residual hunt for those cohorts).
- Per ADR-0010 §3 Validation Cohort: only certs lodged ≥ 2025-07-01 are spec-comparable on cost / SAP rating.
### C.3 Likely shape
Mirror §10a's pattern:
```python
# packages/domain/src/domain/sap/worksheet/sap_rating.py
@dataclass(frozen=True)
class SapRatingResult:
"""SAP 10.2 §11a line refs (256)..(258)."""
ecf_gbp_per_m2_per_yr: float # (256)
sap_rating_continuous: float # (257) un-rounded
sap_rating_integer: int # (258) rounded
@pytest.mark.parametrize("api_fixture", _ELMHURST_API_FIXTURES, ids=...)
def test_api_response_round_trip_matches_lodged_sap_integer(
api_fixture: dict[str, Any]
) -> None:
# Arrange
epc = EpcPropertyDataMapper.from_api_response(api_fixture)
def sap_rating_from_fuel_cost(...) -> SapRatingResult:
...
# Act
result = Sap10Calculator().calculate(epc)
# packages/domain/src/domain/sap/worksheet/co2_emissions.py
@dataclass(frozen=True)
class Co2EmissionsResult:
"""SAP 10.2 §12a line refs (259)..(265). Per-end-use CO2 in kg/yr."""
main_1_co2_kg_per_yr: float # (259)
main_2_co2_kg_per_yr: float # (260)
secondary_co2_kg_per_yr: float # (261)
hot_water_co2_kg_per_yr: float # (262)
pumps_fans_co2_kg_per_yr: float # (263)
lighting_co2_kg_per_yr: float # (264)
pv_credit_co2_kg_per_yr: float # (265) — negative
total_co2_kg_per_yr: float # sum, clamped >=0
# packages/domain/src/domain/sap/worksheet/primary_energy.py
@dataclass(frozen=True)
class PrimaryEnergyResult:
"""SAP 10.2 §13a line refs (266)..(272). Per-end-use kWh primary/yr."""
# mirror Co2EmissionsResult shape with primary energy factors
...
# Assert — integration gate: SAP integer = lodged integer.
assert result.sap_score == api_fixture["sap_rating_current"]
```
Per-end-use CO2 factors source from **Table 12 / Table 32** (same values per RdSAP10 §19.2). Per-end-use PEF source from **Table 12 / Table 32** primary energy factor column.
### C.4 Fixture sourcing
### C.4 Calculator coupling
User will provide API JSONs from real RdSAP10 cert lodgements. Likely sources:
- Live API pulls from `gov-epc` endpoint for known cert addresses.
- Saved JSONs from prior pulls (some may exist in `etl/customers/*` paths — check `kwh_client_for_deletion.pkl`?).
Mirror §10a path (i) — cert_to_inputs precompute. `CalculatorInputs` gains:
- `sap_rating: SapRatingResult`
- `co2_emissions: Co2EmissionsResult`
- `primary_energy: PrimaryEnergyResult`
User stated next steps: "Next I will be generating more test files to battle test and then I want to build an integration test to get a rdsap10 API response through to the modelled sap where I will be expecting 0 error. This would then be a huge validation point that we're there because this will be our integration test and we'll then look to do this across a few hundred API responses."
`calculate_sap_from_inputs` reads them; deletes the inline CO2 + primary-energy blocks. The `SapResult` fields stay (sap_score, co2_kg_per_yr, primary_energy_kwh_per_yr, etc.) — populated from the composite slots.
### C.5 Slice plan
```
Slice 1 — worksheet/co2_emissions.py: Co2EmissionsResult + orchestrator + Table 12/32 CO2 factor lookups (or reuse table_12.co2_factor_kg_per_kwh). Synthetic unit tests + 6-fixture conformance via SapResult.co2_kg_per_yr parity.
Slice 2 — worksheet/primary_energy.py: PrimaryEnergyResult + orchestrator. Same shape.
Slice 3 — worksheet/sap_rating.py: SapRatingResult (refactor of worksheet/rating.py: existing `energy_cost_factor` + `sap_rating` + `sap_rating_integer` move into a single orchestrator that takes a FuelCostResult and returns the result dataclass).
Slice 4 — CalculatorInputs composite slots + cert_to_inputs wiring + calculator delegation (drop inline CO2/PE blocks). Re-verify 6-fixture e2e.
Slice 5 — docs: SPEC_COVERAGE rows for §11a / §12a / §13a, ADR-0010 amendment if cost target needs cross-ref.
```
### C.6 Tests
- Synthetic: per-orchestrator unit tests (kWh × factor arithmetic per end-use, PV credit sign, total clamps).
- 6-fixture ALL_FIXTURES: re-pin `SapResult.co2_kg_per_yr` + `primary_energy_kwh_per_yr` per fixture if PDF lodges them.
- e2e ceiling on 000474 SAP score: should stay at 1 (or wherever §B closes it).
### C.7 Don't list
- Don't touch §13 `worksheet/rating.py` Equation 7-9 logic — the formulas are spec-faithful and already pinned. The refactor moves them under a `sap_rating_from_fuel_cost` orchestrator.
- Don't introduce a new CO2 factor table — use the existing `table_12.co2_factor_kg_per_kwh`. RdSAP10 §19.2 says CO2 is unchanged from SAP10.2 Table 12.
- Don't widen e2e ceilings to absorb refactor regressions. If anything regresses, pause + ask.
So: scale target is ~few hundred API responses, with SAP integer delta=0 required across the cohort.
---
## §D — Codebase pointers
### Appendix L lighting (ticket 1)
- Legacy heuristic: [packages/domain/src/domain/ml/demand.py:222](packages/domain/src/domain/ml/demand.py#L222) `predicted_lighting_kwh`
- Spec cascade (gains side): [packages/domain/src/domain/sap/worksheet/internal_gains.py:204](packages/domain/src/domain/sap/worksheet/internal_gains.py#L204) `_lighting_gains_monthly_w` — extract `e_l_annual_kwh`
- cert_to_inputs callsite: [packages/domain/src/domain/sap/rdsap/cert_to_inputs.py](packages/domain/src/domain/sap/rdsap/cert_to_inputs.py) — search for `predicted_lighting_kwh`
### Table 3c (ticket 1)
### §11a/§12a/§13a sweep (ticket 2)
- §10a orchestrator (template): [packages/domain/src/domain/sap/worksheet/fuel_cost.py](packages/domain/src/domain/sap/worksheet/fuel_cost.py) — mirror the kwargs-orchestrator + Result dataclass shape.
- Existing inline CO2 + primary-energy: [packages/domain/src/domain/sap/calculator.py:467-505](packages/domain/src/domain/sap/calculator.py#L467-L505).
- SAP rating equations: [packages/domain/src/domain/sap/worksheet/rating.py](packages/domain/src/domain/sap/worksheet/rating.py) — `energy_cost_factor`, `sap_rating`, `sap_rating_integer`.
- Table 12 factors: [packages/domain/src/domain/sap/tables/table_12.py](packages/domain/src/domain/sap/tables/table_12.py) — CO2 + PEF columns already typed.
- Existing Table 3b row 1: [worksheet/water_heating.py:308](../../packages/domain/src/domain/sap/worksheet/water_heating.py#L308) — `combi_loss_monthly_kwh_table_3b_row_1_instantaneous`. Mirror this shape.
- Table 3a "keep-hot time-clock" default: [water_heating.py:341](../../packages/domain/src/domain/sap/worksheet/water_heating.py#L341) — `combi_loss_monthly_kwh_table_3a_keep_hot_time_clock` = 600 kWh/yr.
- PCDB parser: [tables/pcdb/parser.py:165-168](../../packages/domain/src/domain/sap/tables/pcdb/parser.py#L165) — field-position mapping.
- Override gate: [cert_to_inputs.py:725](../../packages/domain/src/domain/sap/rdsap/cert_to_inputs.py#L725) — `_pcdb_table_3b_combi_loss_override`.
- `GasOilBoilerRecord` dataclass: [tables/pcdb/parser.py:50](../../packages/domain/src/domain/sap/tables/pcdb/parser.py#L50).
### RdSAP API integration (ticket 2)
- API → domain mapper: `datatypes/epc/domain/mapper.py` → `EpcPropertyDataMapper.from_api_response`.
- Golden cert harness: [packages/domain/src/domain/sap/rdsap/tests/test_golden_fixtures.py](../../packages/domain/src/domain/sap/rdsap/tests/test_golden_fixtures.py).
### Spec / docs
- SAP10.2 PDF: `docs/sap-spec/sap-10-2-full-specification-2025-03-14.pdf`. §11/§12/§13 are around pages 32-38 (ask the user for precise spec-page anchors before scanning).
- RdSAP10 PDF: `docs/sap-spec/RdSAP 10 Specification 10-06-2025.pdf`. §19.2 confirms CO2 + PEF match SAP10.2 Table 12.
- ADR-0010 (cost target): [docs/adr/0010-sap10-calculator-spec-target-and-validation.md](docs/adr/0010-sap10-calculator-spec-target-and-validation.md). Carries the §10a amendment + handover-time deferred list.
- SPEC_COVERAGE: [docs/sap-spec/SPEC_COVERAGE.md](docs/sap-spec/SPEC_COVERAGE.md).
- SAP10.2 PDF: `docs/sap-spec/sap-10-2-full-specification-2025-03-14.pdf`. Appendix J §J3 (Table 3c).
- BRE PCDF Spec v1.0 §7.11: field layout for separate_dhw_tests + F1..F3 + R1..R2.
- RdSAP10 PDF: `docs/sap-spec/RdSAP 10 Specification 10-06-2025.pdf`.
- ADR-0010: [docs/adr/0010-sap10-calculator-spec-target-and-validation.md](../adr/0010-sap10-calculator-spec-target-and-validation.md). Carries amendments.
- SPEC_COVERAGE: [docs/sap-spec/SPEC_COVERAGE.md](SPEC_COVERAGE.md).
### Fixtures
- 000474 + 000490 are the two fixtures with PDF e2e expectations (`test_e2e_elmhurst_sap_score.py`). 4 others (000477, 000480, 000487, 000516) only pin worksheet-section LINE_* values.
- Lighting fixture inputs: each `_elmhurst_worksheet_*.py` has CFL/LED/incandescent counts on the cert build_epc().
- 6 Elmhurst worksheets at `sap worksheets/U985-0001-NNNNNN.{pdf,txt}`.
- Fixture builders at `packages/domain/src/domain/sap/worksheet/tests/_elmhurst_worksheet_NNNNNN.py`. Each has section-level pinned constants (LINE_X_*) + a `build_epc()` builder.
- Shared elmhurst test harness: `_elmhurst_fixtures.py` (ALL_FIXTURES + parametrize helpers).
- 4 non-Elmhurst golden JSONs at `packages/domain/src/domain/sap/rdsap/tests/fixtures/golden/`.
---
@ -264,43 +243,50 @@ Slice 5 — docs: SPEC_COVERAGE rows for §11a / §12a / §13a, ADR-0010 amendme
The dev container ships `/grill-me`, `/tdd`, `/caveman`. Default flow:
```
/grill-me → walk down design tree, recommend scope cuts
/tdd implement Appendix L lighting swap → one test → one impl → repeat
/grill-me → walk the design tree
/tdd implement Table 3c two-profile combi loss → one test → one impl → repeat
```
Same chain for §11a/§12a/§13a. Consider `/grill-with-docs` if domain language drifts.
---
## §F — Definitely do NOT
- Do **not** loosen the 000474 e2e ceiling 3 → 4 etc. to mask the lighting overshoot. The whole point of the Appendix L slice is to close that ceiling further.
- Do **not** loosen the existing component pins to mask drift. Table 3c is a real engine fix; its closure tightens, not loosens.
- Do **not** scan more than ~50 lines of spec PDF without asking the user for the specific page/table range.
- Do **not** touch `inputs.fuel_cost` (the §10a precompute) or `inputs.energy_requirements` (the §9a precompute). They're load-bearing and tested.
- Do **not** invoke `/ultrareview` yourself — it's user-triggered.
- Do **not** delete the legacy scalar cost fields (`space_heating_fuel_cost_gbp_per_kwh` etc.) from `CalculatorInputs` in this work — they're a synthetic-test backwards-compat shim per the §10a slice-2c fallback. Rip out only in a dedicated cleanup ticket after the synthetic test corpus migrates.
- Do **not** touch the SAP rating constants in `worksheet/rating.py` — they're SAP 10.2 (per `a41ac6bd`) and pinned by 8+ tests.
- Do **not** invoke `/ultrareview` yourself — user-triggered only.
---
## §G — Known follow-ups (named on §10a + §4 HW deferred lists)
## §G — Known follow-ups (named on prior deferred lists)
Reference: ADR-0010 amendment "Deferred work" + SPEC_COVERAGE §4 HW / §10a "Remaining work" sections.
Reference: ADR-0010 amendment lists.
**Worksheet:**
- Table 12a `Table12aSystem` cert→row mapping for off-peak electric mains (currently zero-sentinel fallback).
- Table 13 immersion / HP-DHW-only WH fractions.
- Off-peak per-row (230a)-(230g) Table 12a split for pumps/fans (spec line 8076).
### Worksheet
- **Table 3c two-profile combi loss** — Ticket 1 above.
- Table 3b storage / FGHRS rows (no fixture yet).
- Electric CPSU Appendix F (no fixture yet).
- §4 cylinder + solar + WWHRS + PV diverter + FGHRS branches (no fixture yet).
- Table 12a `Table12aSystem` cert→row mapping for off-peak electric mains.
- Table 13 immersion / HP-DHW WH fractions.
- Off-peak per-row (230a)–(230g) split for pumps/fans.
- (247a) Instant electric shower kWh routing.
- (252) per-row Appendix M/N split (PV / wind / hydro / micro-CHP).
- (252) per-row Appendix M/N split.
- (253)/(254) Appendix Q routes.
- §4 cylinder + solar + WWHRS + PV diverter + FGHRS branches.
- §4 PCDB Table 3b storage / FGHRS rows + Table 3c two-profile + Electric CPSU Appendix F.
**Infra / cleanup:**
- Drop legacy scalar fuel-cost fields from `CalculatorInputs` once synthetic test corpus migrates to `fuel_cost=...`.
- §11 FEE compliance conditions (different ventilation / HW / lighting / gains column) — relevant for new-build compliance, not the rating sweep.
- Heat-pump Appendix N cascade via PCDB Table 362 (replaces SCOP 2.30 Table 4a fallback for `main_category=4`).
- Table D1/D2/D3 condensing-boiler control-class corrections (Ecodesign).
### Heating
- **Appendix N heat-pump cascade via PCDB Table 362** (replaces SCOP 2.30 Table 4a fallback for `main_category=4`).
- **Table D1/D2/D3 Ecodesign condensing-boiler control-class corrections**.
- Two-main heating system §9a (213) — single-main currently default.
### Pumps/fans Table 4f
- Currently only category 2 (gas combi) is keyed to 160 kWh/yr; categories 3 (oil), 4 (heat pump), 5 (warm-air), 7 (electric storage), 12 (micro-CHP) all fall back to the legacy 130 sentinel.
### Cooling
- Table 10c SEER → cooling fuel kWh — all 6 Elmhurst have `has_fixed_air_conditioning=False`.
### Infra
- Drop legacy scalar fuel-cost fields from `CalculatorInputs` once synthetic test corpus migrates to `fuel_cost=...` composite.
---

2
docs/sap-spec/SPEC_COVERAGE.md
View file

@ -13,7 +13,7 @@ The canonical SAP10.2 algorithm lives in [`2026-05-19-17-18 RdSap10Worksheet.xls
| 1 | 1–120 (approx) | Dimensions | `worksheet/dimensions.py` | **Full** | Porches, conservatories, RIR deferred per ADR-0009 |
| 2 | 121–206 (approx) | Ventilation | `worksheet/ventilation.py` | Partial | No mechanical ventilation (MVHR/MEV), no wind-shelter factor, no pressure-test override (worksheet lines 17-18), no AP4 override (worksheet line 19) |
| 3 | 121–207 | Heat transmission | `worksheet/heat_transmission.py` | **Full (non-RR)** | LINE_31/33/36/37 exact for both non-RR Elmhurst fixtures (000474, 000490). Suspended-timber + Table 20 exposed-floor routes wired. RR sub-areas (gable/slope/stud-wall) deferred until `SapRoomInRoof` carries them. Global y-factor (Table R2 per-junction deferred). |
| 4 | 207–304 | Hot water + Appendix J + Appendix D | `worksheet/water_heating.py` + cert_to_inputs `_water_heating_worksheet_and_gains` + `_apply_water_efficiency` | **Closed for combi-gas — PCDB Table 3b combi loss + Equation D1 monthly cascade wired.** Worksheet line refs (42)..(65) + Appendix D §D2.1 (2) Equation D1 (`water_efficiency_monthly_via_equation_d1`) + Appendix J Table 3b row 1 (`combi_loss_monthly_kwh_table_3b_row_1_instantaneous`). 000474 + 000490 HW kWh match PDF to ≤0.1%. cert_to_inputs splits the §4 worksheet from the efficiency divisor: (45..65) runs early so §5 has (65)m heat gains; HW fuel kWh computed after §8 produces (98c)m for the Eq D1 cascade. PCDB Table 105 parser exposes 5 new combi-loss fields (separate_dhw_tests, r1, F1, F2, F3 + subsidiary_type + store_type) per BRE PCDF Spec v1.0 §7.11. **Deferred**: Cylinder + solar + WWHRS + PV diverter + FGHRS branches (no fixture yet); Table 3b storage / FGHRS rows (no fixture yet); Table 3c two-profile boilers (no fixture yet); Electric CPSU Appendix F path (no fixture yet). |
| 4 | 207–304 | Hot water + Appendix J + Appendix D | `worksheet/water_heating.py` + cert_to_inputs `_water_heating_worksheet_and_gains` + `_apply_water_efficiency` | **Closed for combi-gas (Table 3b row 1) — Table 3c two-profile pending.** Worksheet line refs (42)..(65) + Appendix D §D2.1 (2) Equation D1 (`water_efficiency_monthly_via_equation_d1`) + Appendix J Table 3b row 1 (`combi_loss_monthly_kwh_table_3b_row_1_instantaneous`). 000474 + 000490 HW kWh match PDF to ≤0.1% (both lodge PCDB records with `separate_dhw_tests=1`). cert_to_inputs splits the §4 worksheet from the efficiency divisor: (45..65) runs early so §5 has (65)m heat gains; HW fuel kWh computed after §8 produces (98c)m for the Eq D1 cascade. PCDB Table 105 parser exposes 5 new combi-loss fields (separate_dhw_tests, r1, F1, F2, F3 + subsidiary_type + store_type) per BRE PCDF Spec v1.0 §7.11. **Table 3c two-profile combi loss not implemented**: PCDB records with `separate_dhw_tests=2` (Vaillant ecoTEC sustain 24/28 — affects 000477, 000480, 000487, 000516 from the Elmhurst cohort) fall through to Table 3a "keep-hot time-clock" 600 kWh/yr default, 25× over spec-faithful ~24 kWh/yr. Next ticket — see HANDOVER_NEXT.md. **Deferred**: Cylinder + solar + WWHRS + PV diverter + FGHRS branches (no fixture yet); Table 3b storage / FGHRS rows (no fixture yet); Electric CPSU Appendix F path (no fixture yet). |
| 5 | Internal gains + Appendix L | `worksheet/internal_gains.py` | **Full** | Worksheet-driven (66)..(73), Table 5 Column A, Table 5a 9-row dispatch + heating-season mask, Appendix L L1-L12 with RdSAP §12-1 bulb defaults + Table 6d Z_L (light access factor). Wired into `calculator.py` via `cert_to_inputs`. Six Elmhurst fixtures conform end-to-end to ≤0.6% lighting / ≤0.2 W (73). **Appendix L slice update**: `annual_lighting_kwh` surfaced as a public leaf returning the worksheet-lodged (232) value (Σ L11 monthly distribution; cosine integral 0.998539). `InternalGainsResult.lighting_kwh_per_yr` exposes the same value so `cert_to_inputs` populates `inputs.lighting_kwh_per_yr` from the cascade — single source of truth shared with §5 (67). New worksheet-level per-component pin: `internal_gains_from_cert(...).lighting_kwh_per_yr` matches U985 (232) to abs=1e-4 for all 6 Elmhurst fixtures (000474:139.9452, 000477:201.6754, 000480:212.5531, 000487:227.6861, 000490:171.4217, 000516:230.8853). |
| 6 | Solar gains + Tables 6b/6c/6d + Appendix U | `worksheet/solar_gains.py` | **Full** | Worksheet-driven (74)..(83). Table 6b g⊥ via manufacturer `window_transmission_details` first, Table 6b code lookup fallback; Table 6c FF by frame_material substring; Table 6d Z (heating column) by `OvershadingCategory`; roof windows pitched at RdSAP10 Table 24 default 45°; rooflights horizontal per §U3.2 p128. `solar_gains_from_cert` wired into `cert_to_inputs` + `calculator.py`. Six Elmhurst fixtures conform end-to-end to ≤5e-3 W on (83) + (84). |
| 7 | Mean internal temperature | `worksheet/mean_internal_temperature.py` | **Full** | Worksheet-driven (85)..(94) via `mean_internal_temperature_monthly`. Table 9c steps 1-9 sequential (per-zone η: (86) η_living at Ti=T_h1, (89) η_elsewhere at Ti=T_h2, (94) η_whole at Ti=(93)). Table 9b u-formula consumes weighted R for two-main case 1 (single-main is default). Wired into `calculator.py` + `cert_to_inputs` via two new `CalculatorInputs` fields. Six Elmhurst fixtures conform end-to-end to ≤5e-3 °C on all 9 line tuples + 2 scalars per month (588 assertions). Table 4e adj defaults 0 (cert-side mapping deferred — all 6 fixtures = 0); two-main case 2 (different parts heated separately) deferred. |