diff --git a/docs/HANDOVER_API_PROFILING.md b/docs/HANDOVER_API_PROFILING.md index 7b76b07b..bb842098 100644 --- a/docs/HANDOVER_API_PROFILING.md +++ b/docs/HANDOVER_API_PROFILING.md @@ -13,16 +13,56 @@ deproven approaches + the meter/shower data-fidelity findings), and the earlier `energy_rating_current`. Headline gauge: `PYTHONPATH=/workspaces/model python scripts/eval_api_sap_accuracy.py`. -| metric | session-3 (`a8e5563a`) | session-4 (`faf29942`) | session-5 (`19235d11`) | **session-6 (`a7761ea8`)** | +| metric | session-4 (`faf29942`) | session-5 (`19235d11`) | session-6 (`a7761ea8`) | **session-7 (`3e05c95e`)** | |--------|------------------|------------------|------------------|------------------| -| **% \|err\| < 0.5** | 45.1% | 47.6% | 53.1% | **53.14%** | -| % \|err\| < 1.0 | 59.4% | 62.6% | 68.0% | **67.99%** | -| % \|err\| < 2.0 | 77.7% | 79.6% | ~81% | **81.85%** | -| mean \|err\| | 1.702 | 1.586 | 1.329 | **1.312** | -| median \|err\| | — | — | 0.467 | **0.467** | +| **% \|err\| < 0.5** | 47.6% | 53.1% | 53.14% | **54.24%** | +| % \|err\| < 1.0 | 62.6% | 68.0% | 67.99% | **69.64%** | +| % \|err\| < 2.0 | 79.6% | ~81% | 81.85% | **83.50%** | +| mean \|err\| | 1.586 | 1.329 | 1.312 | **1.248** | +| median \|err\| | — | 0.467 | 0.467 | **0.457** | | computed / raises | 909 / 0 | 909 / 0 | 909 / 0 | **909 / 0** | | unsupported_schema | 100 (deferred) | 100 (deferred) | 100 (deferred) | 100 (deferred) | +### SESSION-7 — sheltered alternative walls (RdSAP Table 4 R=0.5), `3e05c95e` +The headline-moving audit. User: 53% is poor enough to indicate a MAJOR error +— audit again. The decisive diagnostic CHAIN (reusable): +1. **Error by `dwelling_type`** → flats are the drag (houses 60-66% within 0.5, + flats 28-47%; top-floor flat −1.19, mid-floor 28% within 0.5). +2. **Split flats by `mains_gas`** → ELECTRIC flats are the killer (gas flats + ~45%, electric flats 13-19%; top-floor electric mean|err| 3.62). +3. **Invert the SAP equation** (ECF = 0.42·cost/(TFA+45) → `sap_rating`) to get + `our_cost / lodged_cost` → electric flats over-cost ~3% median (houses 1.00), + amplified by the 4× electric price + steep low-band log curve (+3% ≈ +1.5 SAP + at band 40). +4. **Under-rate tracks space-heating kWh/m² precisely** — accurate certs 14-110, + under-rating certs 130-289 (cert 2021: 11 275 kWh for 39 m²) → over-stated + FABRIC, not tariff (storage flats on the correct 5.5p rate under-rate just as + much as room-heater flats at 13p). +5. **Field-by-field on the worst** → cert 0340-2976 (band-A flat) computed wall + 128 W/K though its main wall is a FILLED cavity (U 0.7); the excess was a + SECOND `u_wall` call — a `sap_alternative_wall_1` timber-frame sub-area at + U=2.5 lodging **`sheltered_wall="Y"`**. + +ROOT: the gov-EPC API lodges `sheltered_wall="Y"` per alt-wall, but it was +DROPPED by the schema + domain dataclasses, so `_alt_wall_w_per_k` billed the +sub-area at its full exposed U. RdSAP 10 Table 4 (PDF p.22) "Sheltered": added +external resistance R=0.5 m²K/W → U_sheltered = 1/(1/U + 0.5) — the SAME +adjustment the MAIN wall already applies for `gable_wall_type=2` +(`gable_wall_sheltered`, `_SHELTERED_GABLE_ADDED_RESISTANCE_M2K_W`). Threaded +end-to-end: schema (21.0.0/21.0.1) + domain `SapAlternativeWall.is_sheltered` +(default False → Summary/Elmhurst path unchanged, goldens untouched) + +`from_api_response` `"Y"→True` + `_alt_wall_w_per_k` applies the 0.5 resistance +(lodged-U + basement alts return before it). 140 certs (15% of corpus) carry a +sheltered alt-wall; they under-rated at median −0.82 / signed −1.33 / 23% within +0.5. Eval: 102 improved / 38 regressed (offsetting-error cases — applied +spec-uniformly per the determinism principle); +10 net within-0.5. Goldens + +full regression green, pyright net-zero. **OPEN audit leftovers: electric-flat +tail persists (2021 −3.9 genuine uninsulated solid brick = data-fidelity; +top-floor "Flat, limited insulation" roofs); detached houses 45% (balanced +bidirectional scatter); the 38 sheltering regressions = offsetting errors +elsewhere; main-wall (non-gable) sheltering not audited; multi-element wall +description joined-string-to-all-BPs (113 certs, mild).** + ### SESSION-6 — community fuels 30/31/32 collide with electricity codes, `a7761ea8` Picked up the deferred "fuel-collision part 2". The profiler's strongly-biased `main_control=2306` bucket (n=11, signed −3.75, nearly uniform) was a PROXY: every