junte/Model
1
0
Fork 0
mirror of https://github.com/Hestia-Homes/Model.git synced 2026-07-12 13:29:04 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

docs: handover — fold in S0380.227-231 + mapper parity work; refresh error landscape + worksheet asks

Browse source 
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Khalim Conn-Kowlessar 2026-06-04 21:23:07 +00:00
parent f326e4eb53
commit 2a29b29aa5
1 changed files with 72 additions and 17 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

89
domain/sap10_calculator/docs/HANDOVER_API_SAMPLE_ACCURACY.md
View file

@ -4,10 +4,50 @@ Point-in-time note. Start from [`AGENT_GUIDE.md`](AGENT_GUIDE.md) for methodolog
1e-4 bar, the per-line debugging loop, the section helpers, and the suite command. 1e-4 bar, the per-line debugging loop, the section helpers, and the suite command.
- **Branch:** `feature/per-cert-mapper-validation` - **Branch:** `feature/per-cert-mapper-validation`
- **HEAD:** `0f6b4023` (S0380.229). Next slice: **S0380.230**. - **HEAD:** `f326e4eb`. Next SAP slice: **S0380.232**.
- **Baseline (§4 suite):** `tests/domain/sap10_calculator/ backend/documents_parser/tests/` - **Baseline (§4 suite):** `tests/domain/sap10_calculator/ backend/documents_parser/tests/`
→ green (2397 passed, 1 skipped). Pre-existing out-of-scope failures unchanged → green (2407 passed, 1 skipped). Pre-existing out-of-scope failures unchanged
(stone-§5.6 in `domain/sap10_ml/tests/`; `test_from_rdsap_schema.py::...test_total_floor_area`). (stone-§5.6 in `domain/sap10_ml/tests/`; `test_from_rdsap_schema.py::...test_total_floor_area`).
## Headline now (1,000-cert 2026 API sample, HEAD `f326e4eb`)
| metric | value | was (handover baseline `9c0a373f`) |
|---|---|---|
| computed | 882 / 1000 | 882 |
| **% \|err\| < 0.5** | **42.9%** | 41.8% |
| % < 1 / < 2 / < 5 | 56.7% / 74.6% / 90.1% | 54.9 / 71.9 / 87.8 |
| median / mean \|err\| | 0.73 / **2.04** | 0.79 / ~2.4 |
| mean signed | −0.41 | +0.2 |
**Error by heating cluster** (the load-bearing cut — re-run `analyse_api_sap_clusters.py`):
| cluster | n | mean \|err\| | %<0.5 | note |
|---|---|---|---|---|
| cat 2 gas boiler + PCDB | 639 | 1.27 | 49.6% | well-trodden |
| cat 2 gas, NO PCDB idx | 91 | 3.18 | 35.2% | non-PCDB Table-4b boilers |
| cat 6 community | 45 | 2.59 | 31.1% | known-hard |
| cat 7 electric storage | 40 | **5.25** | 10.0% | was 7.33 → S0380.227-229 |
| cat 10 electric room heaters | 48 | **5.26** | 16.7% | was 9.49 → S0380.230-231 (bias gone) |
| cat 4 HP + PCDB | 8 | 6.11 | 12.5% | small n, APM |
| Flats (any) | 282 | 2.57 | 30.5% | geometry / communal |
| real PV | 45 | 3.90 | 26.7% | Appendix M |
**Worst individual offenders** (the long tail — `eval` TOP 40): `2100-5421-0922-1622-3463`
(−60.8, our SAP **negative** −24.8 vs lodged 36 — a flat, 2 bps, cat-2; the single worst, likely
a geometry/communal blow-up — START a per-cert dig here), `2958-8008` (+32, age 6=tiny),
`9836-5829` (−29.5, cat-10 tail), several cat-7/cat-10 in the −20s.
## Work shipped (this session — S0380.227-231 + 3 mapper commits)
| commit | what |
|---|---|
| **S0380.227** | dedicated DHW-only system (WHS 911) is NOT separately timed → no Table 2b ×0.9; TF (53) 0.54→0.60, (59) h=3→5 |
| **S0380.228** | electric SECONDARY on off-peak bills at Table 12a `OTHER_DIRECT_ACTING_ELECTRIC` (1.00 high-frac), not 100% low |
| **S0380.229** | dedicated water boiler/circulator (WHC 911-931) feeds cylinder via primary loop → Table 3 primary loss applies |
| **S0380.230** | electric room heaters (cat 10) on off-peak → `OTHER_DIRECT_ACTING_ELECTRIC` (mirror of .228 for the MAIN). cat-10 9.49→7.11 |
| **S0380.231** | Dual-meter electric room heaters → 10-hour tariff (RdSAP §12 Rule 3; codes 691-694,699). cat-10 7.11→5.26, bias +5.08→−0.86 |
| `bd25a3c7` | SY system-built vs B basement: code 6 stays system-built; basement → explicit `wall_is_basement`/`is_basement` flag. `system_build` is a derived property (wall type). API path post-processes via addendum. (issue #1177 — see `docs/PR_NOTE_system_built_basement_1177.md`: field-vs-property merge landmine) |
| `f326e4eb` | Elmhurst path now populates `roof_construction` (int) via `_elmhurst_roof_construction_int` for cross-mapper parity (API set it, Elmhurst didn't) |
- **Toolkit (committed):** `scripts/fetch_2026_epc_sample.py`, - **Toolkit (committed):** `scripts/fetch_2026_epc_sample.py`,
`scripts/eval_api_sap_accuracy.py`, `scripts/analyse_api_sap_clusters.py`. The 1,000 cached `scripts/eval_api_sap_accuracy.py`, `scripts/analyse_api_sap_clusters.py`. The 1,000 cached
JSONs live in `/tmp/epc_2026_sample/` (gitignored scratch — re-fetch with the sampler; JSONs live in `/tmp/epc_2026_sample/` (gitignored scratch — re-fetch with the sampler;
@ -103,7 +143,7 @@ PDF — only in the P960 header).
| **S0380.228** | cost (255) | electric SECONDARY on off-peak bills at Table 12a `OTHER_DIRECT_ACTING_ELECTRIC` (7-hr high-frac **1.00** = £0.1529), not the flat off-peak low (£0.0550). Worksheet (242): "1.00*15.29 + 0.00*5.50". THE primary cost driver (−340). | 60.1→**50.67** | | **S0380.228** | cost (255) | electric SECONDARY on off-peak bills at Table 12a `OTHER_DIRECT_ACTING_ELECTRIC` (7-hr high-frac **1.00** = £0.1529), not the flat off-peak low (£0.0550). Worksheet (242): "1.00*15.29 + 0.00*5.50". THE primary cost driver (−340). | 60.1→**50.67** |
| **S0380.229** | (62) 2493.30→**3169.98** | dedicated water-heating boiler/circulator (WHC 911-931) feeds the cylinder via a primary loop → Table 3 row 1 primary loss applies (keyed off `water_heating_code`, since `_water_heating_main` returns the electric SPACE main). Restored the missing (59)=676.68 kWh/yr. | 50.67→50.33 | | **S0380.229** | (62) 2493.30→**3169.98** | dedicated water-heating boiler/circulator (WHC 911-931) feeds the cylinder via a primary loop → Table 3 row 1 primary loss applies (keyed off `water_heating_code`, since `_water_heating_main` returns the electric SPACE main). Restored the missing (59)=676.68 kWh/yr. | 50.67→50.33 |
**The ONE remaining case-19 cause — the PV diverter (63b) — is S0380.230 (next).** Worksheet **The ONE remaining case-19 cause — the PV diverter (63b) — is S0380.232.** Worksheet
header line 124 "Diverter = Yes"; Summary §19 "Diverter present: Yes". Per **SAP 10.2 Appendix header line 124 "Diverter = Yes"; Summary §19 "Diverter present: Yes". Per **SAP 10.2 Appendix
G4 (PDF p.72-73)** surplus PV is diverted to the cylinder immersion: G4 (PDF p.72-73)** surplus PV is diverted to the cylinder immersion:
`S_PV,diverter,m = EPV,m × (1 − βm) × 0.8 × 0.9`, clamped to ≤ (62)m + (63a)m, entered as a `S_PV,diverter,m = EPV,m × (1 − βm) × 0.8 × 0.9`, clamped to ≤ (62)m + (63a)m, entered as a
@ -134,15 +174,22 @@ pages → ElmhurstSiteNotesExtractor(...).extract() → from_elmhurst_site_notes
## Remaining work, prioritised ## Remaining work, prioritised
### A. Accuracy clusters (highest value — 80+ certs, mean err 7–10) ### A. Accuracy clusters (highest value)
1. **Electric storage heaters (cat 7, 39 certs).** Distinct cascade — off-peak tariff split, 1. **PV diverter (S0380.232)** — closes case 19 to 1e-4 AND helps the real-PV cluster (45 certs,
charge control (2401/2402), 7-hr/24-hr charge, Table 4a efficiency, responsiveness. **No mean 3.90). Fully spec'd in the case-19 section above (Appendix G4). **Has a worksheet** →
worksheet currently validates this path.** Errs both directions (−27..+16). 1e-4 bar. Do this first: it's the one open cause on a validated worksheet.
2. **Electric room heaters (cat 10, 43 certs).** Likewise (controls 2601/2602/2603). Worst 2. **Electric storage heaters (cat 7, 40 certs, mean 5.25).** S0380.227-229 took it 7.33→5.25;
cluster by mean (10.26). the case-19 PV diverter will help further. Beyond that the tail is per-cert — a **dedicated
3. **Flats (242, 29% <0.5)** and **PV (40, 28%)** — secondary. cat-7 worksheet** (no PV, no diverter) would let you pin charge-control / responsiveness at
1e-4 instead of the ±0.5 lodged fallback.
3. **Electric room heaters (cat 10, 48 certs, mean 5.26).** S0380.230-231 fixed the systematic
tariff bias (mean 9.49→5.26, signed +5.08→−0.86); the residual is now scattered per-cert
(e.g. `9836-5829` −29.5, an under-rater). A **cat-10 worksheet** pins the tail at 1e-4.
4. **Non-PCDB gas boilers (cat 2, no idx, 91 certs, mean 3.18)** and **Flats (282, mean 2.57)** —
the next volume levers once the electric clusters are worksheet-pinned. Flats = geometry /
communal; start with the worst (`2100-5421` negative SAP).
### B. Remaining raises (18 certs — all U-value / heat-loss-sensitive, NOT enum guesses) ### B. Remaining raises (16 certs — all U-value / heat-loss-sensitive, NOT enum guesses)
- **`gable_wall_type` 2 & 3 (14 certs).** RdSAP 10 **Table 4** RR walls: 0=Party (U=0.25), - **`gable_wall_type` 2 & 3 (14 certs).** RdSAP 10 **Table 4** RR walls: 0=Party (U=0.25),
1=Exposed (U=common wall), 2/3 = **Sheltered (U=external×R0.5)** + **Adjacent-to-heated 1=Exposed (U=common wall), 2/3 = **Sheltered (U=external×R0.5)** + **Adjacent-to-heated
(U=0)**, code↔type order unconfirmed (schema says "not yet seen"). Needs (i) a worksheet to (U=0)**, code↔type order unconfirmed (schema says "not yet seen"). Needs (i) a worksheet to
@ -161,16 +208,24 @@ is what the strict-raise guard exists to prevent.
--- ---
## ★ Additional worksheets that would help most ## ★ Additional worksheets that would help most (the user will generate these on request)
Case 19 (above) already covers electric storage heaters + loose-jacket cylinder + RR. The two The two electric clusters are now systematic-bias-free (S0380.227-231) but their TAILS sit at
that would add the most NEW coverage: the ±0.5-vs-lodged fallback bar because **no worksheet validates them at 1e-4**. The three
1. **An electric ROOM-heater dwelling** (SAP code ~691, control 2601/2602) — the **cat-10 highest-value worksheets to ask the user for:
cluster (43 certs, worst by mean error 10.26)**, which case 19 does not touch. 1. **An electric ROOM-heater dwelling** (SAP code ~691, control 2601/2602/2603, Dual meter) —
2. **A room-in-roof with a SHELTERED gable and an ADJACENT-TO-HEATED gable** (Table 4 types pins the cat-10 tail (48 certs, mean 5.26) at 1e-4. Make it PV-free + cylinder-free to
isolate the space-heat path from the diverter/HW.
2. **An electric STORAGE-heater dwelling distinct from case 19** (no PV, no WHS-911) — pins the
cat-7 tail (40 certs, mean 5.25): charge control (2401/2402), 7-hr vs 24-hr, responsiveness.
3. **A room-in-roof with a SHELTERED gable and an ADJACENT-TO-HEATED gable** (Table 4 types
beyond Party/Exposed) — closes the `gable_wall_type` 2/3 raise (14 certs) and pins the beyond Party/Exposed) — closes the `gable_wall_type` 2/3 raise (14 certs) and pins the
Sheltered (U=ext×R0.5) / Adjacent (U=0) U-values the calculator must add. Sheltered (U=ext×R0.5) / Adjacent (U=0) U-values the calculator must add.
Per worksheet send BOTH the **Summary PDF** (input) and the **P960/dr87 worksheet PDF** (the
`(1)..(286)` ground truth). Drop them in `sap worksheets/golden fixture debugging/<name>/` and
run the case-19 debug recipe.
The original "design one property" guidance (kept below for reference) is what case 19 was The original "design one property" guidance (kept below for reference) is what case 19 was
built from. built from.