mirror of
https://github.com/Hestia-Homes/Model.git
synced 2026-06-08 11:17:27 +00:00
Handover: replace stale docs with focused §3-close + Table-11 brief
Delete HANDOVER_FRESH_REVIEW (22-slice, MAE-5.34 era) and HANDOVER_SYSTEMATIC_REVIEW (pre-Elmhurst-conformance). Both described a state the Elmhurst worksheet work has since superseded. Add HANDOVER_S3_CLOSE.md with: - Accurate §3 status: §1/§2 fully done; LINE_31/LINE_36 exact for non-RR fixtures; LINE_33 gap diagnosed as missing floor_construction codes (not a window-area problem as previously assumed) - Concrete investigation steps to close LINE_33 for 000474 + 000490 - Table 11 Secondary Heating framed as next slice after §3 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Khalim Conn-Kowlessar
parent
2fd0fe1c08
commit
49e8c65ae8
3 changed files with 149 additions and 1228 deletions
|
|
@ -1,136 +0,0 @@
|
|||
# Handover: fresh-context review of the SAP 10.2 calculator
|
||||
|
||||
Audience: a fresh agent in a new context window. Read this first, then the SAP 10.2 + RdSAP 10 spec PDFs, then the calculator code. Your job is to find spec-vs-implementation gaps that the previous (long-context) agent has missed or got wrong.
|
||||
|
||||
## TL;DR — where we are
|
||||
|
||||
- Deterministic SAP 10.2 calculator at `packages/domain/src/domain/sap/`.
|
||||
- 22 slices shipped under ADR-0009.
|
||||
- 300-cert parity probe: **SAP MAE 5.34, bias +0.29** (we're slightly over-predicting SAP score on average).
|
||||
- **Primary-energy bias +51.6 kWh/m²** ← biggest surprise; we over-predict primary energy by ~50%. This was discovered just before this handover; previous slices weren't accounting for it correctly.
|
||||
- 17/300 (5.7%) certs match the cert's `energy_rating_current` exactly.
|
||||
|
||||
Goal per ADR-0009: typical-subset SAP MAE ≤ 1.0.
|
||||
|
||||
## Critical context
|
||||
|
||||
1. **Two truth-sources collide.** `tables/table_12.py` carries the spec-correct SAP 10.2/10.3 prices (mains gas 3.64p, std elec 16.49p). `tables/table_12_cert_calibration.py` carries the empirical lower prices that match the cert assessor's actual output (3.48p, 13.19p). The parity probe uses the cert-calibration table; the engine's default is spec.
|
||||
2. **The cert assessor diverges from the published SAP 10.2 spec in several places** we've found:
|
||||
- Unit prices: cert uses ~10-25% lower than published Table 12
|
||||
- Tariff routing: cert applies off-peak to electric room heaters (code 691) when meter_type=1 (Dual), even when Table 12a says these should bill at the high rate
|
||||
- Unknown meter (RdSAP energy_tariff=3): cert defaults to Single (per Elmhurst test), our code also matches this
|
||||
3. **PEUI bias was discovered right at handover time.** Our `primary_energy_kwh_per_m2` runs +51 kWh/m² over the cert's `energy_consumption_current`. This is the biggest clue and the most efficient next dig.
|
||||
|
||||
## Repo layout
|
||||
|
||||
```
|
||||
packages/domain/src/domain/sap/
|
||||
├── calculator.py # Sap10Calculator + calculate_sap_from_inputs
|
||||
├── tables/
|
||||
│ ├── table_12.py # SAP 10.2 spec prices, CO2, PEF
|
||||
│ └── table_12_cert_calibration.py # empirical cert prices
|
||||
├── worksheet/
|
||||
│ ├── dimensions.py # §1
|
||||
│ ├── ventilation.py # §2 (incl wind shelter S-B21)
|
||||
│ ├── heat_transmission.py # §3 (incl DwellingExposure)
|
||||
│ ├── internal_gains.py # §5 + Appendix L
|
||||
│ ├── solar_gains.py # §6 + Appendix U §U3.2
|
||||
│ ├── utilisation_factor.py # Table 9a
|
||||
│ ├── mean_internal_temperature.py # §7 + Table 9/9b/9c
|
||||
│ ├── space_heating.py # §9
|
||||
│ └── rating.py # §13 (SAP rating equations)
|
||||
├── climate/
|
||||
│ └── appendix_u.py # Tables U1/U2/U3 + solar declination
|
||||
├── rdsap/
|
||||
│ └── cert_to_inputs.py # EpcPropertyData → CalculatorInputs mapping
|
||||
├── validation/
|
||||
│ └── parity_report.py # ParityReport aggregator
|
||||
└── tests/ # 103 unit tests
|
||||
|
||||
services/ml_training_data/src/ml_training_data/
|
||||
└── sap_parity_probe.py # runs calculator on N random certs from corpus
|
||||
|
||||
docs/sap-spec/
|
||||
├── sap-10-2-full-specification-2025-03-14.pdf (199pp) — primary spec
|
||||
├── sap-10-3-full-specification-2026-01-13.pdf (201pp) — newer spec (Table 12 identical)
|
||||
├── rdsap-10-specification-2025-06-10.pdf (114pp) — RdSAP rules (separate from SAP)
|
||||
├── SPEC_COVERAGE.md — our coverage map
|
||||
└── PARITY_FINDINGS.md — earlier probe findings
|
||||
|
||||
docs/adr/0009-deterministic-sap-calculator.md — accepted ADR
|
||||
```
|
||||
|
||||
## How to run the parity probe
|
||||
|
||||
```bash
|
||||
python -c "
|
||||
import sys
|
||||
sys.path.insert(0, 'packages/domain/src')
|
||||
sys.path.insert(0, '.')
|
||||
sys.path.insert(0, 'services/ml_training_data/src')
|
||||
from ml_training_data.sap_parity_probe import main
|
||||
main(['300','7']) # 300 certs, seed=7
|
||||
"
|
||||
```
|
||||
|
||||
## Where to dig (priority-ordered, by likely MAE impact)
|
||||
|
||||
### Tier 1 — the PEUI mystery (50% over)
|
||||
|
||||
Our `primary_energy_kwh_per_m2` runs +51 kWh/m² over the cert's `energy_consumption_current`. Possibilities:
|
||||
|
||||
- **Wrong primary energy factors in `tables/table_12.py PRIMARY_ENERGY_FACTOR`**. I populated this from approximate spec values; verify each one against SAP 10.2 Table 12 (page 189). Especially electricity PEF=1.501 — that's ~30% of corpus uses electricity for some end-use.
|
||||
- **HW demand over-counted.** Look at `domain.ml.demand.predicted_hot_water_kwh`. Cylinder loss + primary circuit loss may be over-stated. SAP §J + Appendix J details exact formulas. We use bucket-rounded `_STORAGE_LOSS_FACTOR` instead of interpolation.
|
||||
- **Space heating demand over-counted.** Could come from:
|
||||
- Living-area-fraction defaults (Table 27): we use {1:0.75, 2:0.50, 3:0.30, 4:0.25, ≥5:0.21}; double-check against the RdSAP 10 PDF.
|
||||
- Control-temperature adjustment (Table 4e): we always pass 0; spec applies ~-0.7°C in some configurations.
|
||||
- Thermal mass parameter: we use 250 kJ/m²K always; spec varies by construction type.
|
||||
- **Lighting/pumps over-counted.** Currently using Appendix L existing-dwelling fallback (no fixed lighting). Newer dwellings should use lower lighting energy.
|
||||
|
||||
### Tier 2 — wall U-value cascade
|
||||
|
||||
Worst-residual certs have `wall_construction=4 (cavity)`, `wall_insulation_type=2`, `wall_insulation_thickness="NI"`. We treat as uninsulated cavity (column 0). Cert assessor may know it's insulated (the type=2 code says so). See `domain.ml.rdsap_uvalues._insulation_bucket` — when `thickness=0` AND `present=True`, spec says use 50mm row but our parser converts "NI"→0 which short-circuits to "uninsulated".
|
||||
|
||||
I tried switching "NI"→None in S-B5 cycle but it over-corrected aggregate MAE. Worth re-trying with the new understanding (compare PRIMARY energy delta on affected certs specifically).
|
||||
|
||||
### Tier 3 — cost-side residuals
|
||||
|
||||
Per S-B17 hand-trace: cert 2389-4472 has correct delivered energy but our SAP is 10 points lower than the cert's. Implied cert blended unit-cost rate is lower than ours. Likely cause: cert assessor applies different rate logic in edge cases (oil + off-peak meter, electricity-and-gas mix, etc.). Worth tracing more carefully.
|
||||
|
||||
### Tier 4 — known unimplemented spec pieces
|
||||
|
||||
(per `SPEC_COVERAGE.md`)
|
||||
- Cooling §10 (rare)
|
||||
- FEE §11 (new-build only)
|
||||
- Per-junction thermal bridging Table R2 (ADR says defer)
|
||||
- Multi-main heating Table 11 with non-zero secondary (we have this conditionally)
|
||||
- Standing charges (Table 12 note (a))
|
||||
|
||||
## What's been validated
|
||||
|
||||
- §13 SAP rating equations: 108.8 − 120.5 log10(ECF) for ECF ≥ 3.5, else 100 − 16.21·ECF. Verified against SAP 10.2 PDF page 38.
|
||||
- §12.2 fuel price rule: "Other prices must not be used". We have spec-correct prices + cert-calibration prices as separate tables.
|
||||
- Appendix U: tables verbatim.
|
||||
- Appendix U rating-uses-UK-average rule: applied (S-B18).
|
||||
- Solar gains §6.1 + Appendix U §U3.2 polynomial: implemented.
|
||||
|
||||
## Suggested first session
|
||||
|
||||
1. **Read SAP 10.2 §§4 + Appendix J carefully** (hot water demand). Map every formula against our `domain.ml.demand.predicted_hot_water_kwh`. Note divergences. The PEUI bias is largely driven by HW + heating demand.
|
||||
2. **Read SAP 10.2 §14** (CO2 and primary energy). Compare to our `calculate_sap_from_inputs` primary_energy aggregation. Note especially: does the cert's `energy_consumption_current` use the same end-use list (space + HW + lighting + pumps/fans) or a different one?
|
||||
3. **Read RdSAP 10 §11 (Heating)**. Check our `domain.ml.sap_efficiencies.seasonal_efficiency` cascade against the RdSAP rules. Especially heat pump efficiency (we use 2.30 for category 4 fallback).
|
||||
4. Open issues in the parity-decomp data:
|
||||
- 26 certs with correct energy but SAP MAE 4.12 → cost-side
|
||||
- 51 kWh/m² primary-energy bias → demand-side
|
||||
|
||||
## Don't repeat these dead-ends
|
||||
|
||||
- ❌ Switching "NI" wall thickness to None — over-corrected in aggregate (S-B5)
|
||||
- ❌ Aggressive efficiency rescue for missing sap_main_heating_code — over-corrected (S-B5)
|
||||
- ❌ Using SAP 10.2 spec prices for parity validation — the cert assessor uses legacy lower prices despite reporting sap_version=10.2 (S-B9, S-B10)
|
||||
- ❌ Applying off-peak to electric main heating regardless of meter_type — the meter_type field is the truth (S-B15)
|
||||
- ❌ Always applying 10% secondary heating — should be conditional on cert lodging or main system being electric storage (S-B20)
|
||||
|
||||
## Commit history
|
||||
|
||||
The last 22 commits are S-B1..S-B22. Each commit message documents the slice's hypothesis, change, and measured impact. Worth reading 5-10 of the latest commit messages for context on what's been tried.
|
||||
149
docs/sap-spec/HANDOVER_S3_CLOSE.md
Normal file
149
docs/sap-spec/HANDOVER_S3_CLOSE.md
Normal file
|
|
@ -0,0 +1,149 @@
|
|||
# Handover — Close §3, then Table 11 Secondary Heating
|
||||
|
||||
**Audience:** Fresh agent continuing the deterministic SAP 10.2 calculator
|
||||
(`packages/domain/src/domain/sap/`). Read this document first, then skim
|
||||
the two key source files listed below.
|
||||
|
||||
---
|
||||
|
||||
## What we're building
|
||||
|
||||
A deterministic SAP 10.2 calculator that replicates cert-software output
|
||||
(Elmhurst, Stroma, etc.) exactly for RdSAP 10 input certs. The domain
|
||||
concept is **Calculated SAP10 Performance** — see
|
||||
`docs/adr/0009-deterministic-sap-calculator.md`. Progress is tracked in
|
||||
`docs/sap-spec/SPEC_COVERAGE.md`.
|
||||
|
||||
The workflow is strict TDD: **one failing test → minimal implementation →
|
||||
commit**. Each commit is one slice.
|
||||
|
||||
---
|
||||
|
||||
## Current state
|
||||
|
||||
### §1 Dimensions — DONE
|
||||
All 6 Elmhurst fixtures pass exactly (`test_section_1_matches_elmhurst_worksheet`).
|
||||
|
||||
### §2 Ventilation — DONE
|
||||
All 6 Elmhurst fixtures pass exactly (`test_section_2_matches_elmhurst_worksheet`).
|
||||
|
||||
### §3 Heat transmission — PARTIALLY DONE
|
||||
|
||||
What passes today:
|
||||
- **Internal invariants** (all 6 fixtures): `(33) = Σ per-element`,
|
||||
`(37) = (33) + (36)`.
|
||||
- **Exact LINE_31 + LINE_36** (non-RR fixtures 000474 and 000490 only):
|
||||
`test_section_3_non_rr_line_31_and_36_match_elmhurst_worksheet`.
|
||||
|
||||
What does NOT yet pass:
|
||||
- **Exact LINE_33** (fabric heat loss) for any fixture. This is the
|
||||
remaining §3 close task (see below).
|
||||
- **RR sub-areas** (fixtures 000487, 000480, 000477, 000516): gable/
|
||||
slope/stud-wall/flat-ceiling areas are not in `SapRoomInRoof`; these
|
||||
fixtures are **formally deferred** — see gap notes in
|
||||
`test_section_3_partial_match_against_elmhurst_worksheet`.
|
||||
|
||||
---
|
||||
|
||||
## Task A — Close LINE_33 for non-RR fixtures (investigation slice)
|
||||
|
||||
**Goal:** assert exact LINE_33 and LINE_37 for 000474 and 000490.
|
||||
|
||||
### The diagnostic gap
|
||||
|
||||
Running `heat_transmission_from_cert(epc, window_total_area_m2=0, door_count=actual)` on
|
||||
000474 gives `fabric = 193.83 W/K`. The Elmhurst `LINE_33 = 209.11 W/K`.
|
||||
The gap is +15.28 W/K — and it cannot be explained by window area alone,
|
||||
because `u_wall (1.5) > u_window_eff (1.33)`, so adding windows would
|
||||
*decrease* fabric heat loss, not increase it.
|
||||
|
||||
The gap is therefore in one or more of the other elements. Most likely
|
||||
culprits, in priority order:
|
||||
|
||||
1. **Floor construction missing from fixture.**
|
||||
`SapFloorDimension.floor_construction` is `None` in all Elmhurst
|
||||
fixture files (field not set). Our `u_floor` fallback may not match
|
||||
the Elmhurst value. The 000490 fixture comment records the expected
|
||||
U-values explicitly: *"suspended timber ground floor on main (U=0.71),
|
||||
exposed timber floor on Extension 1 (U=1.20)"*. Set the correct
|
||||
`floor_construction` and `floor_insulation` codes on each
|
||||
`SapFloorDimension` and see if the gap closes.
|
||||
|
||||
2. **Roof construction / insulation thickness missing from fixture.**
|
||||
Similarly, `roof_insulation_thickness` may not be set on the building
|
||||
parts. The Elmhurst cert will have a specific roof type and insulation
|
||||
depth that drives a specific `u_roof`.
|
||||
|
||||
3. **Wall insulation re-check.** All fixtures use `wall_insulation_type=4`
|
||||
(`_WALL_INSULATION_NONE`), giving `u_wall = 1.5` for cavity age B.
|
||||
Confirm this matches the actual Elmhurst worksheet row.
|
||||
|
||||
### How to proceed
|
||||
|
||||
1. Read the EPC API field encoding for `floor_construction` and
|
||||
`floor_insulation` in `datatypes/epc/domain/epc_property_data.py`
|
||||
and `packages/domain/src/domain/ml/rdsap_uvalues.py` (the `u_floor`
|
||||
function + its construction constants).
|
||||
2. Look up the actual floor type for 000474 and 000490 from the PDF
|
||||
(ask the user — PDFs were supplied manually; not stored in repo).
|
||||
3. Set `floor_construction` + `floor_insulation` + `floor_insulation_thickness`
|
||||
on the `SapFloorDimension` objects in the fixture files.
|
||||
4. Re-run the debug calc (`r0.fabric` with `window_area=0`) and check
|
||||
whether the gap collapses.
|
||||
5. Once floor/roof are resolved, back-calculate window area:
|
||||
`A_w = (LINE_33 - r0.fabric) / (window_u_eff - u_wall)`.
|
||||
If the gap is now ≤ the window contribution, this formula should give
|
||||
a physically plausible positive area (5–15 m² for a 2-storey terrace).
|
||||
6. Add `WINDOW_TOTAL_AREA_M2: float` and `WINDOW_AVG_U_VALUE: float = 1.4`
|
||||
constants to each non-RR fixture file.
|
||||
7. Write a new parametrised test asserting exact LINE_33 and LINE_37 for
|
||||
000474 and 000490. Commit as one slice.
|
||||
|
||||
---
|
||||
|
||||
## Task B — Table 11 Secondary Heating (highest-MAE-impact gap)
|
||||
|
||||
Per `SPEC_COVERAGE.md`, this is the **next priority after §3**.
|
||||
|
||||
Most boiler-main certs allocate ~10 % of space heating to a secondary
|
||||
system (electric room heater or similar). We currently model 0 %. This
|
||||
causes a systematic bias on the large majority of boiler certs.
|
||||
|
||||
**SAP 10.2 Table 11** gives the secondary fraction keyed on main-heating
|
||||
type. **RdSAP 10 Appendix A** identifies the heating type from cert codes.
|
||||
|
||||
Starting point: `packages/domain/src/domain/sap/calculator.py` (entry
|
||||
point) and `packages/domain/src/domain/sap/rdsap/cert_to_inputs.py`
|
||||
(cert→inputs adapter). The `SapInputs` struct carries `main_heating_*`
|
||||
fields — see how space heating demand is calculated and where a secondary
|
||||
fraction would hook in.
|
||||
|
||||
---
|
||||
|
||||
## Key files to read
|
||||
|
||||
| File | Why |
|
||||
|---|---|
|
||||
| `packages/domain/src/domain/sap/worksheet/heat_transmission.py` | §3 implementation — `heat_transmission_from_cert` |
|
||||
| `packages/domain/src/domain/sap/worksheet/tests/test_heat_transmission.py` | all §3 tests including the partial Elmhurst conformance test |
|
||||
| `packages/domain/src/domain/sap/worksheet/tests/_elmhurst_worksheet_000474.py` | non-RR fixture to close |
|
||||
| `packages/domain/src/domain/sap/worksheet/tests/_elmhurst_worksheet_000490.py` | non-RR fixture to close |
|
||||
| `packages/domain/src/domain/ml/rdsap_uvalues.py` | all U-value lookups — `u_floor`, `u_wall`, `u_roof` |
|
||||
| `docs/sap-spec/SPEC_COVERAGE.md` | overall progress tracker |
|
||||
| `docs/adr/0009-deterministic-sap-calculator.md` | scope + architectural decisions |
|
||||
|
||||
Spec PDFs are at `docs/sap-spec/` — SAP 10.2 (March 2025), SAP 10.3
|
||||
(Jan 2026), RdSAP 10 (June 2025).
|
||||
|
||||
The canonical reference Excel worksheet is at the repo root:
|
||||
`2026-05-19-17-18 RdSap10Worksheet.xlsx`. A loader for it is at
|
||||
`packages/domain/src/domain/sap/worksheet/tests/_xlsx_loader.py`.
|
||||
|
||||
---
|
||||
|
||||
## Test suite
|
||||
|
||||
```
|
||||
python -m pytest packages/domain/src/domain/sap/worksheet/tests/ -q
|
||||
# Should show 122 passed
|
||||
```
|
||||
File diff suppressed because it is too large
Load diff
Loading…
Add table
Reference in a new issue