Model/docs/HANDOVER_API_PROFILING.md

# Handover — API SAP accuracy (session 3): raises cleared, now profile-driven

> **➡️ SESSION 10 STARTS HERE: `docs/HANDOVER_SUMMARY_AUDIT.md`.** HEAD `872bc585`, **56.8%
> within-0.5** (909 computed / 0 raises). Session 9 ran FIVE data-driven audit angles — all
> converged on "remaining gap is diffuse" — and shipped the glazing Table-24 win (+16 certs) +
> HW-only heat-network DLF. The data-driven seam is mined out; **session 10 switches to the
> summary-report-based per-cert worksheet audit.** Read that doc first.

**Branch:** `feature/per-cert-mapper-validation` (long-lived working branch — **NEVER PR to
main**; the user pushes/PRs when ready). **HEAD `a8e5563a`+** (the profiler commit), local-only
ahead of origin.

**READ ALSO:** the auto-memory `project_per_cert_mapper_validation_state` (full slice log +
deproven approaches + the meter/shower data-fidelity findings), and the earlier
`docs/HANDOVER_API_ACCURACY_S2.md` (session-2 method).

## THE GOAL (unchanged)
100% of API records with a lodged SAP compute within **0.5 SAP** of the API's
`energy_rating_current`. Headline gauge:
`PYTHONPATH=/workspaces/model python scripts/eval_api_sap_accuracy.py`.

| metric | session-7 (`3e05c95e`) | session-8 (`71b378b9`) | session-8b (`e7af6fda`) | **session-8c (`e6dda705`)** |
|--------|------------------|------------------|------------------|------------------|
| **% \|err\| < 0.5** | 54.24% | 55.01% | 55.12% | **54.90%** ⚠ |
| % \|err\| < 1.0 | 69.64% | 70.08% | 70.08% | **70.19%** |
| % \|err\| < 2.0 | 83.50% | ~83.6% | ~83.6% | **84.60%** |
| mean \|err\| | 1.248 | 1.233 | 1.232 | **1.224** |
| median \|err\| | 0.457 | 0.448 | 0.446 | **0.448** |
| computed / raises | 909 / 0 | 909 / 0 | 909 / 0 | **909 / 0** |
| unsupported_schema | 100 (deferred) | 100 (deferred) | 100 (deferred) | 100 (deferred) |

### SESSION-8c — Table 4g default SFP for index-less MEV (HEADLINE TRADE-OFF), `e6dda705`
The 11 index-less MEV certs (mostly GAS houses, `mechanical_ventilation=2`, no PCDB
index) billed ZERO fan electricity → +2.2 over-rate (signed +1.23, median +2.19).
SAP 10.2 §2.6.3 / Table 4g note 1 (p.176): default SFP **0.8 W/(l/s)** used directly
as SFPav in (230a). `_mev_decentralised_kwh_per_yr_from_cert` now falls back to it
for a mechanical-EXTRACT system when no Table 322 record resolves. Cohort closed
+1.23 → +0.18 signed. **TRADE-OFF: within-0.5 55.12% → 54.90% (−2)** — the fan energy
is only ~half each cert's over-rate, so the cohort lands at ~+1.0 (still outside 0.5)
while 2 borderline offsetting-error certs cross out; within-1.0/within-2.0/mean|err|
all improve. Spec-correct, applied uniformly per the determinism principle. **NEXT:
the unmasked ~+1.0 residual on gas-house MEV — the OTHER half (not electric-flat
fabric, since these are gas houses).** Goldens green, pyright net-zero.

### SESSION-8b — MEV fan electricity (PCDB index plumbed), `e7af6fda`
Follow-up to 8: the +0.9 residual on MEV after the §2 heat-loss fix was the FAN
electricity (§5 Table 4f (230a) `SFPav × 1.22 × V`, PCDB Tables 322/329).
`_mev_decentralised_kwh_per_yr_from_cert` already composes it but reads
`epc.mechanical_ventilation_index_number` + `mechanical_vent_duct_type`, which the
API builder never set → `pcdf_id is None` zeroed the fan energy on every API cert.
Wired both through the 21.0.1 construction. The 9 MEV certs with a PCDB index
closed +0.90 → +0.13; the **11 index-less MEV certs still sit at +1.36** — they
need the SAP Table 4h DEFAULT specific fan power (no PCDB record), a clean next
slice (verify the default SFP value against spec first). New e2e test + golden
fixture (cert 1300, dMEV index 500777). NB the 21.0.0 API builder didn't get the
8/8b ventilation edits (only 21.0.1, the corpus schema).

### SESSION-8 — API `mechanical_ventilation` enum never mapped, `71b378b9`
Re-profiling after the sheltering fix surfaced `mechanical_ventilation=2` as a
clean systematic over-rate (n=20, signed +1.90, 5% within 0.5, every cert
positive). ROOT: `from_api_response` DROPPED the doc-level
`mechanical_ventilation` field, so `mechanical_ventilation_kind` was always None
and the §2 (24a..d) cascade defaulted to NATURAL — under-stating the air-change
rate for every mechanical system (only the Elmhurst path mapped it). New
`_api_mechanical_ventilation_kind` maps the RdSAP-Schema-21 enum →
MechanicalVentilationKind: 0→NATURAL, 1 MV→"MV", 2/3 MEV + 6 PIV-outside →
"EXTRACT_OR_PIV_OUTSIDE", 5 PIV-loft→NATURAL; **code 4 (MVHR) DEFERRED** (needs
the lodged HR efficiency, PCDB Table 326, the API path doesn't plumb — stays
NATURAL rather than mis-modelling as MV). Unmapped → `UnmappedApiCode`. Extract
cohort +1.90→+0.9 median (within-0.5 5%→35%), 20 improved / 3 regressed. The
**+0.9 residual is the MEV FAN ELECTRICITY** (§2.6.4 SFP, PCDB Table 322
decentralised-MEV — 9 of 20 carry `mechanical_ventilation_index_number`): a
clean next slice. Goldens + regression green, pyright net-zero.

### SESSION-7 — sheltered alternative walls (RdSAP Table 4 R=0.5), `3e05c95e`
The headline-moving audit. User: 53% is poor enough to indicate a MAJOR error
— audit again. The decisive diagnostic CHAIN (reusable):
1. **Error by `dwelling_type`** → flats are the drag (houses 60-66% within 0.5,
   flats 28-47%; top-floor flat −1.19, mid-floor 28% within 0.5).
2. **Split flats by `mains_gas`** → ELECTRIC flats are the killer (gas flats
   ~45%, electric flats 13-19%; top-floor electric mean|err| 3.62).
3. **Invert the SAP equation** (ECF = 0.42·cost/(TFA+45) → `sap_rating`) to get
   `our_cost / lodged_cost` → electric flats over-cost ~3% median (houses 1.00),
   amplified by the 4× electric price + steep low-band log curve (+3% ≈ +1.5 SAP
   at band 40).
4. **Under-rate tracks space-heating kWh/m² precisely** — accurate certs 14-110,
   under-rating certs 130-289 (cert 2021: 11 275 kWh for 39 m²) → over-stated
   FABRIC, not tariff (storage flats on the correct 5.5p rate under-rate just as
   much as room-heater flats at 13p).
5. **Field-by-field on the worst** → cert 0340-2976 (band-A flat) computed wall
   128 W/K though its main wall is a FILLED cavity (U 0.7); the excess was a
   SECOND `u_wall` call — a `sap_alternative_wall_1` timber-frame sub-area at
   U=2.5 lodging **`sheltered_wall="Y"`**.

ROOT: the gov-EPC API lodges `sheltered_wall="Y"` per alt-wall, but it was
DROPPED by the schema + domain dataclasses, so `_alt_wall_w_per_k` billed the
sub-area at its full exposed U. RdSAP 10 Table 4 (PDF p.22) "Sheltered": added
external resistance R=0.5 m²K/W → U_sheltered = 1/(1/U + 0.5) — the SAME
adjustment the MAIN wall already applies for `gable_wall_type=2`
(`gable_wall_sheltered`, `_SHELTERED_GABLE_ADDED_RESISTANCE_M2K_W`). Threaded
end-to-end: schema (21.0.0/21.0.1) + domain `SapAlternativeWall.is_sheltered`
(default False → Summary/Elmhurst path unchanged, goldens untouched) +
`from_api_response` `"Y"→True` + `_alt_wall_w_per_k` applies the 0.5 resistance
(lodged-U + basement alts return before it). 140 certs (15% of corpus) carry a
sheltered alt-wall; they under-rated at median −0.82 / signed −1.33 / 23% within
0.5. Eval: 102 improved / 38 regressed (offsetting-error cases — applied
spec-uniformly per the determinism principle); +10 net within-0.5. Goldens +
full regression green, pyright net-zero. **OPEN audit leftovers: electric-flat
tail persists (2021 −3.9 genuine uninsulated solid brick = data-fidelity;
top-floor "Flat, limited insulation" roofs); detached houses 45% (balanced
bidirectional scatter); the 38 sheltering regressions = offsetting errors
elsewhere; main-wall (non-gable) sheltering not audited; multi-element wall
description joined-string-to-all-BPs (113 certs, mild).**

### SESSION-6 — community fuels 30/31/32 collide with electricity codes, `a7761ea8`
Picked up the deferred "fuel-collision part 2". The profiler's strongly-biased
`main_control=2306` bucket (n=11, signed −3.75, nearly uniform) was a PROXY: every
cert in it is `sap_main_heating_code=301` (community heating). ROOT: gov-API
`main_fuel_type` codes **30=waste-combustion / 31=biomass / 32=biogas** (all
"(community)" in `epc_codes.csv`) collide in VALUE with the Table-32 electricity
codes 30 (standard) / 31 (7h-low) / 32 (7h-high). All three sit in
`_ELECTRIC_FUEL_CODES`, so `is_electric_fuel_code` flagged a community-scheme main as
electric and `_is_electric_main` routed its cost through the off-peak electricity
branch — BYPASSING `_heat_network_factor_fuel_code`. Cert 8536 (biomass community)
billed 5.5 p grid electricity instead of the 4.24 p heat-network rate → −17.2 SAP.
Per SAP 10.2 Table 12 the community waste/biomass/biogas rows are 42/43/44 (the same
rows the backwards-compat enum codes 11/12/13 already map to). Added 30→42, 31→43,
32→44 to `API_FUEL_TO_TABLE_12` + `API_FUEL_TO_TABLE_32`.
**CRITICAL — the remap is GATED, NOT global.** A prototype putting 30/31/32 in
`canonical_fuel_code` regressed certs 2211 (+16 SAP) and 3420 (+7): the cascade uses
the bare Table-32 code 30 internally as `_STANDARD_ELECTRICITY_FUEL_CODE` (the RdSAP
no-water-heating whc=999 immersion default writes `water_heating_fuel=30`), so a
blanket remap mis-prices genuine grid electricity as community waste. New
`_heat_network_community_fuel_code(fuel, main)` translates only when
`_is_heat_network_main(main)` is true, wired into `_main_fuel_code` AND
`_water_heating_fuel_code` (water gated via `_water_heating_main(epc)`).
**STRICT-RAISE (user-requested):** a heat-network main lodging a colliding community
fuel the table doesn't cover raises `UnmappedSapCode` rather than silently falling
through to the same-numbered electricity code (currently can't fire — all of
{30,31,32} are mapped — but guards future community fuels 56/57/58/99). Cert 8536
−17.25 → −6.51 (residual now flat fabric), 5036 −6.29 → +1.36; mean|err|
1.329→1.312, within-1.0/2.0 up, within-0.5 held. 3 AAA tests, regression green
(only pre-existing 2 stone-wall U + a flaky historic-epc ordering test), pyright
net-zero. **STILL DEFERRED: dual-fuel API-9; the per-cert FABRIC tail (8536's −6.5
is now flat fabric, 2100 demand 110905); the 100 unsupported-schema certs.**

### SESSION-5 — fuel-code collision (anthracite/coal), `19235d11`
The re-audit traced the cohort's WORST cert (2100 anthracite, −61) + the −20/−21 coal cluster to
the **fuel-code collision** (`reference_fuel_code_collision`): the shared price/CO2/PE lookups check
Table-32/12-code membership BEFORE translating the gov-API fuel enum, so API-5 (anthracite) priced at
the bulk-LPG rate (12.19 p) and API-33 (coal) at the electricity-10h rate (7.5 p). KEY constraint
(goldens caught it): code 33 is ALSO the electricity-10h TARIFF code used by the dual-rate CO2/PE
split — so the fix CANNOT live in the shared table functions (breaks golden 000565). Instead
`canonical_fuel_code` (table_32) normalises the colliding SOLID-fuel enums at the **fuel-TYPE
boundary** (`_main_fuel_code`/`_water_heating_fuel_code`): 5→15 anthracite 3.64 p, 33→11 house coal
3.67 p; also fixes `is_electric_fuel_code(33)` mis-flagging coal as electric. Scoped to {5, 33}
(unambiguous large mispricings). mean|err| 1.424→1.329 (2100 −61→−11, residual now FABRIC: 110 905
kWh demand = a separate area over-statement); within-0.5 flat at 53.1%. **DEFERRED follow-ups:**
dual-fuel API-9 (0.45 p delta, net-neutral, shifts certs in an un-understood direction — needs its own
look); community API-20/25/31 (route through the heat-network standing/CO2 path, NOT `unit_price` —
cert 8536 fuel-31 still mis-prices at 5.5 p electricity → −17). Method: the `decompose_api_cost_error.py`
heat:high/low tail + field-by-field audit of the worst certs surfaced it; `is_electric_fuel_code`
collisions are the tell.

### THE UNIFYING PRINCIPLE (user, load-bearing) — "unknown insulation → as-built, NOT uninsulated"
An EPC insulation field that is UNDETERMINED (thickness `'ND'`/`'AB'`/absent → parsed None, or
description "as built / (assumed)") must map to the **age-band default** ("as built"), which is
INSULATED at newer bands — never to the uninsulated row. The recurring bug shape: a fixed
uninsulated U (cavity Filled-row, roof Table-16 2.30) is MASKED at old bands (where the age
default coincides with uninsulated) and only diverges (catastrophic under-rate) at newer bands.
All three session-5 fixes are instances. Review status across elements (all now conform):
- **Flat roofs** — FIXED `58cff932` (this slice). **Pitched roofs** — "unknown"→Table 18 (`a64e857b`);
  "no insulation" only appears at bands A/B where 2.30 IS the age default (verified, no new-band bug).
- **Cavity walls** — FIXED `2e466ed1`. **System/timber/solid walls** — already on the as-built age
  row (verified: bidirectional scatter, not a one-cause under-rate). **Floors** — undetermined
  thickness already routes to the Table 19 age default (I=25/J=75/K=100 mm; verified).
- CAVEAT: do NOT broadly reroute PITCHED `'ND'`/`'NI'`→Table 18 (the parsed-0 `'NI'` case) — that
  was empirically net-negative (pitched "no insulation" lodgements genuinely use 2.30 even at newer
  bands; the description is load-bearing for pitched lofts). The principle holds for flat roofs,
  cavity, floors; pitched lofts are the documented exception.

## SESSION-5 UPDATE (HEAD `2e466ed1`) — whc=903 immersion HW + as-built cavity-U both closed

**Shipped (47.6 → 52.1%, two spec-grounded fixes):**

**(2) `2e466ed1` as-built "insulated (assumed)" cavity → Cavity-as-built row, not Filled cavity
(48.6 → 52.1%, the bigger win).** Robust-sweep lead: `wall_desc="Cavity wall, as built, insulated
(assumed)"` median +0.26, n=145, but split by age band it was a CLEAN G/H signal (G +1.38 n37,
H +1.61 n18; I-L neutral). RdSAP 10 Table 6 (England, p.41) "Filled cavity" row carries the † footnote
("assumed as built") ONLY at bands I-M, where it equals "Cavity as built"; at A-H the filled row is a
GENUINE fill. An as-built cavity (type 4) must use "Cavity as built" at all bands (G/H 0.60 not 0.35).
This was the SAME latent A-H bug slice S0380.210 fixed for "partial insulation (assumed)" but left for
"insulated (assumed)" by a legacy convention. Retired `_cavity_described_as_filled`; genuine fills
(wall_insulation_type=2) still hit the filled row. Per-band confirmation: I-M unchanged, G/H corrected
exactly. Bucket within-0.5 47% → 66%; eval +32 net (36 improved, 4 regressed — offsetting-error
electric-storage flats). 3 tests updated to the corrected behaviour (the legacy tests literally said
"we follow the legacy convention for parity").

**(1) `43d4c67d` WHC-903 electric immersion off-peak HW → SAP 10.2 Table 13 high-rate fraction
(47.6 → 48.6%).** The session-4 robust-sweep `whc=903`
lead (median +0.87, n=84).
- `43d4c67d` **WHC-903 electric immersion off-peak HW → SAP 10.2 Table 13 high-rate fraction.**
  Was billing 100% at the off-peak low rate; Table 12a "Immersion water heater" row (p.191) routes
  the WH column to Table 13 (p.197). New `tables/table_13.py` evaluates the Note-2 equations (f of
  cylinder volume V, occupancy N, single/dual immersion), clamped [0,1]; 18-/24-hour use the 10-hour
  column (Note 1). Wired into `_hot_water_fuel_cost_gbp_per_kwh` (threads V / N / immersion-single
  from the caller; absent any → old 100%-low fallback, no regression). Off-peak WHC-903 cohort
  (n=57): within-0.5 16% → 33%, median |err| 1.56 → 0.86.
- **IMMERSION CODE MAPPING CORRECTED: `immersion_heating_type` 1 = DUAL, 2 = SINGLE.** The
  session-3 handover lead #3's "(1=single, 2=dual)" was UNVERIFIED and BACKWARDS. Confirmed via
  RdSAP 10 §10.5 (p.54 — immersion "assumed dual" on a dual/off-peak meter) cross-checked with the
  cohort: code 1 sits 3.6:1 on dual meters (40 vs 11), code 2 on single meters (22 vs 16). Dual
  carries Table 13's small fraction → matches the over-rating direction; the single mapping
  overshot in a prototype (cohort within-0.5 16% → 14%). The description-vs-code-audit lesson
  again: skeptical of unverified handover code-semantics claims.
- **Next robust leads (post-BOTH-fixes sweep, ranked by net directional skew + MEDIAN):** every
  top bucket is now an UNDER-rate cluster (negative median = fabric/flat scatter, per-cert not one-bug):
  `property_type=2` flats med −0.39 (n=283, netDir +75), roof "(another dwelling above)" −0.46 (n=182),
  `wall_desc="Solid brick, as built, no insulation"` −0.22 (n=114). No clean OVER-rate single-cause
  bucket remains (cavity-insulated dropped to −0.13, main_heat_cat=7 to −0.31, whc=903 off the top —
  all addressed). The flats under-rate is the biggest front but DIFFUSE (fabric/tariff per-cert) — likely
  needs worksheets, not one rule. The 100 unsupported-schema certs remain the deferred big ticket.
  METHOD NOTE: the cavity win came from splitting the +0.26 bucket BY AGE BAND — the mild median hid a
  sharp G/H spike. When a description bucket has a modest median but a plausible single mechanism,
  re-split by age band / sub-field before dismissing it as scatter.

**SESSION-4 shipped (45.1 → 47.6%):** four spec-grounded fixes + closed one false lead.
See the `## SESSION-4 …` blocks below and the auto-memory for full detail. The systematic bias
is gone; the winning method this session was the **description-vs-code audit** + an
**outlier-robust categorical sweep** (rank by net directional skew + MEDIAN, not mean — the
mean-based metric is fooled by multi-cause outliers). 47.6% is still the target's halfway point.

## WHAT SHIPPED THIS SESSION (7 slices, all green, pyright net-zero)
1. `e41a0bc0` **PCDB heat pump w/o SAP code → Table 12a ASHP_APP_N SH split** (0.80 high-rate).
2. `2bc73fb0` **HP-DHW (WHC 901/902/914 + PCDB HP) → Table 12a WH 0.70 split.** Together (1)+(2)
   killed the cat-4 heat-pump over-rating bias (+1.43 → +0.06).
3. `449d8c5b` **direct-acting electric boiler (191) → zero primary circuit loss** (SAP Table 3
   p.160 zero list names it verbatim).
4. `f4048588` **wall_insulation_thermal_conductivity ignored → §5.8 default λ=0.04.** (See KEY
   INSIGHT below — the gov field is RdSAP *output*, not an input.)
5. `1c5675a0` **floor_heat_loss=8 → no floor heat loss** (extension floor over a heated space;
   RdSAP §3, like code 6).
6. `a8e5563a` **main_heating_category=9 (warm air) → Table 11 secondary fraction 0.10.**
   (4)(5)(6) cleared **all 4 raises** — eval now has zero raises.
7. `(profiler)` **`scripts/profile_api_error.py`** — the new diagnostic (below).

## SESSION-4 UPDATE (HEAD `faf29942`) — read before re-working the leads below

### Shipped this session (45.1 → 47.6%)
1. `b40e0f67` **exposed-floor-on-flats** (floor_heat_loss=1) — §3.12; per-BP override of the
   dwelling-level flat suppression.
2. `8741fbdf` **floor_heat_loss=3 → above partially heated space, U=0.7** (§3.12/§5.14) +
   re-pinned golden 7536 (its "irreducible residual" was THIS bug).
3. `5e7ef5c7` **boiler interlock for TRVs+bypass controls 2107/2111** (§9.4.11) — biggest single
   win (+1.6pts). The no-interlock set was keyed off the wrong signal (the "+0.6 °C" annotation);
   2107/2111 lack a room thermostat → −5pp + Table 4f ×1.3 pump.
4. `faf29942` **description-lodged secondary heating** (§A.2.2/Table 11) — gas/oil boilers with an
   API-description-only secondary ("Portable electric heaters (assumed)", code field None)
   dropped the secondary (sec_kWh=0); now `_has_lodged_secondary_description` fires Table 11.
   Also added cat-8 (electric underfloor) Table-11 fraction 0.10.
- `560c912c`/`d0f57a0e` docs: **roof_construction=8 lead CLOSED as data-fidelity** (not a bug — see
  the roof-8 section below; user worksheet sim-case-29 proved we ≡ Elmhurst).

### The two methods that worked (reuse these)
- **Description-vs-code audit:** join each int code (`floor_heat_loss`, `roof_construction`,
  `wall_construction`, secondary type) to its authoritative `…[].description`, **on single-element
  certs only** (multi-element `[]` arrays are LOSSY). Mis-maps fall out (floor-3, secondary).
- **Outlier-robust categorical sweep** (`/tmp/cat_audit2.py`): rank field-values by **net
  directional skew** (#under−0.5 minus #over+0.5) + **MEDIAN** error. The mean-based directionality
  metric (`/tmp/cat_audit.py`) gets FOOLED by multi-cause outliers (e.g. "Solid brick no insulation"
  looked systematic at mean −1.07 but median is −0.22 = scatter; 2100 −61/RR drove it).

### Open robust leads (verify with `/tmp/cat_audit2.py` — they shift; check MEDIAN not mean)
- `whc=903` electric-immersion HW: **median +0.87, n=84** — likely off-peak immersion handling
  (the handover noted WHC 903 raises NotImplementedError on the Table-12a off-peak-immersion row).
- `main_heat_cat=7` electric storage: median +1.05, n=41 — over-rate (tariff/cost; partly artifact).
- `immersion_type=2` dual: +1.50, n=43 — we OVER-credit (so §12 dual→off-peak would worsen it).
- `dwelling_type=Top-floor flat`: median −1.24, n=99 — under-rate, mostly fabric scatter/artifacts.
- **Low-dir = SCATTER, do NOT single-fix:** non-PCDB main / data_source=2 (n=242, 28% within-0.5),
  mains_gas=N electric (n=145), most flats. These are per-cert/data-fidelity, not one bug.

### Resolved/closed this session (don't re-chase)
- **Lead #1 `floor_codes=3` RESOLVED — the code IS authoritative.** The diagnostic that cracked
  it: join each **single-BP** cert's `floor_heat_loss` code to its independent
  `floors[].description` (the multi-BP tally was contaminated because a cert's `floors[]` summary
  is LOSSY — it drops some BPs' descriptions). Single-BP gives a perfect 1:1 enum: code 1↔"To
  external air"(exposed), 2↔"To unheated space"(semi-exposed), **3↔"(other premises below)"
  (9/9)**, 6↔"(another dwelling below)"(party), 7↔Solid/Suspended(ground). Per RdSAP §3.12
  (p.25) code 3 = "above a partially heated space" (non-domestic premises below) → §5.14 constant
  **U=0.7** (NOT Table-20 semi-exposed, NOT ground). SHIPPED `8741fbdf`.
- **SHIPPED `b40e0f67`:** exposed-floor-on-flats (code 1) area fix — §3.12. A flat's code-1
  floor was area-zeroed by `_dwelling_exposure`; now the per-BP `is_exposed_floor` overrides the
  flat suppression upward (mirrors the "another dwelling below" party override).
- **SHIPPED `8741fbdf`:** code 3 → `is_above_partially_heated_space` (U=0.7) + area override.
  **RE-PINNED golden 7536-3827** — its Ext2(bp3) code-3 floor was mis-read as "ground U=1.12" by
  a prior agent (the lossy floors[] dropped its description), who declared the residual an
  "irreducible register-rounding artifact, DO NOT chase". It was this bug: U 1.12→0.70, PE/CO2
  residuals moved toward 0. **LESSON: "irreducible residual" golden notes are suspect — a real
  mapper bug can hide there.** Eval (both slices): 45.1→45.3%, mean|err| 1.702→1.659, <1.0
  59.5→60.2%. User is generating a fresh `0380-2087-8190-2996-3075` worksheet to independently
  confirm U=0.7 (0380 now −0.63) — validate when it lands.
- **Leads re-checked, NOT clean:** `immersion_type=2` (+1.86) is high-scatter (mean|err| 3.71,
  bidirectional). `main_control=2107` (+1.63) is correctly mapped ("Programmer, TRVs and bypass"
  type 2 Table 4c(2)) — over-rate is diffuse gas-boiler/flat-fabric, not a dispatch bug.
  `roof_codes=1` broad bucket is mean −0.15 (the −1.78 was top-floor-electric-flat outliers
  −29/−25). Remaining gains need per-cert worksheets (start code-3) or the unsupported-schema ticket.

## SESSION-4 AUDIT — description-vs-code cross-check (other element types)
After floor-3, audited every API field that carries BOTH an integer code AND an
independent `…[].description` (roofs, walls, floors, main_heating, controls), by joining
code↔description on **single-element certs** (the multi-element `[]` summary is lossy).
- **Walls / heating / controls: CLEAN** — codes map 1:1 to their description families;
  residual error is per-cert scatter, not mis-mapping.
- **Roof `roof_construction=8`: investigated hard, NOT a bug — DO NOT re-chase.** It looked
  like a huge lead (n=57, ~189 |err|, mean −2.43) because code-8 sloping ceilings described
  "Pitched, insulated" with no numeric thickness compute U=2.30 (e.g. 7921-0052 roof_w 241
  W/K, SAP 56.75 vs lodged 80). **User worksheet (simulated case 29) settled it: at band C,
  RdSAP/Elmhurst assumes UNINSULATED for an "insulated (assumed)" roof+wall with no measured
  thickness → Elmhurst SAP 55, which MATCHES our 56.75.** The lodged 80 needed real insulation
  the API record doesn't carry = data-fidelity artifact (meter-3 class). KEY LESSON (user):
  **"insulated (assumed)" = the age-band DEFAULT insulation level, NOT "well insulated"** — at
  old bands that default is ~uninsulated. Re-audit by band confirmed: old-band no-numeric =
  artifacts (we ≡ Elmhurst); newer-band code-8 already gets correct insulated U (2031 band I
  U≈0.18, 1436 band E 0.17, 0536 band B 0.09) — their under-rates are elsewhere; numeric-
  thickness = accurate (9884 +0.06). So the force-uninsulated-at-pre-1950 rule is CORRECT.
  A roof-8 "fix" would push us AWAY from Elmhurst's faithful 55 toward the unreproducible 80.
- **Roof `roof_construction=3`: latent mis-map (inert).** gov desc "(another dwelling above)"
  (party roof) but we map to "Pitched, no access to loft"; masked by dwelling exposure
  (mean −0.06). Correct for robustness only if touching the roof mapper; not worth chasing.
- **Worksheet-gen constraints (user, for future repros):** Elmhurst no longer lets you pick
  build form for a flat; and a band-C repro defaults to uninsulated walls+roof.

## KEY INSIGHT (load-bearing, from the user)
**The gov EPC API JSON is the published OUTPUT of RdSAP software (Elmhurst), not its input.**
So any API field Elmhurst doesn't expose as an *input* is register metadata the RdSAP10 method
does **not** consume — route it to the spec default, don't try to "use" it. This is exactly why
`wall_insulation_thermal_conductivity` (slice 4) → always λ=0.04. Apply the same lens to any
new "extra" API field before wiring it.

## THE NEW DIAGNOSTIC — `scripts/profile_api_error.py` (run this first)
`PYTHONPATH=/workspaces/model python scripts/profile_api_error.py` joins each computed cert's
signed error with a rich feature set from its **raw API JSON** (not the mapped EpcPropertyData),
and ranks (feature, value) buckets by error carried + by |mean signed| bias. This is how to find
"silly API-path handling" gaps. `--min-n N` sets the bucket floor.

### PRIORITISED LEADS (from the run at `a8e5563a` — verify with the profiler, they'll shift)
Cleanest "API-path handling" candidates first (small, biased buckets = likely a mapper/dispatch
bug, not noise):

1. **`floor_codes=3` → mean signed +5.37 (n=10).** We map API `floor_heat_loss=3` → "To unheated
   space" (same as code 2). The +5.37 over-rate says that's wrong — code 3 likely isn't "unheated
   space" (or its U is wrong). Pull the n=10 certs, check what code 3 really is (ask the user the
   Elmhurst floor dropdown — the API=output lens). **Highest bias, smallest scope = start here.**
2. **Control-code biases:** `main_control=2306` −2.96 (n=11), `2602` +2.49 (n=14), `2107` +1.65
   (n=38), `2402` +1.14 (n=10), `2307` +0.74 (n=11). Several control codes carry systematic bias
   → Table 4c/4e control dispatch gaps. `2107`/`2602` are the biggest. Check
   `_CONTROL_TYPE_BY_CODE` + the Table 4c efficiency-adjustment / Table 4e control coverage.
3. **`immersion_type=2` (dual immersion) → +2.00 (n=43, mean|err| 3.85).** RdSAP §12 lists "dual
   electric immersion" as an off-peak trigger; the cascade does NOT consume `immersion_heating_type`
   for tariff (verified — only comments reference it). Wiring the §12 dual-immersion → off-peak
   rule for Unknown meters is a clean spec slice. (1=single, 2=dual per the Elmhurst Summary.)
4. **`roof_codes=1` −1.78 (n=27)** (flat roof under-rate) and **`roof_insulation_thickness=None`
   −1.18 (n=52)** — flat-roof / no-thickness roof handling.
5. **`main_data_source=2` / `has_pcdb_main=False` → 28% within 0.5, mean|err| 3.17 (n≈242).**
   Non-PCDB heating systems (SAP-table efficiency) are a big under-rating cluster. Likely
   Table 4b default-efficiency or fabric, but worth a look — it's 1/4 of the sample.

### Big scattered segments (need worksheets, NOT clean single fixes)
- **`whc=903` (electric immersion HW): 13% within 0.5, n=84** — looks like the worst bucket but
  it's the electric **storage(cat-7)+room-heater(cat-10)** segment compounding (worst certs span
  −29…+32, bidirectional). Not one bug.
- **`mains_gas=N` (electric): 21% within 0.5, mean|err| 4.27 (n=145)** — the hardest segment;
  per-cert fabric/tariff scatter.
- **Flats (`property_type=2`): 31% within 0.5 (n=283)** — still the worst dwelling type.
- **cat-7 storage (+0.75) / cat-10 room heaters (+0.75)** — both net over-rate; bidirectional.

## DEPROVEN — do NOT retry (empirically failed in earlier sessions; details in memory)
- Routing **roof `'ND'` → Table 18** (description is load-bearing even with 'ND').
- Broad **"all Unknown(meter 3) electric → off-peak"** (over-credits room heaters). NOTE: the
  meter-3 under-rate is partly an **irreducible data-fidelity artifact** — the register stores
  meter_type=3 ("Unknown") on certs whose lodged rating actually used an off-peak meter (cert
  2474: lodged 78 needs 18-hour, but API says Unknown → spec-faithful ~68). Don't chase those to
  the lodged value.
- **RR shell U Table-17-50mm** (golden 6035 disproves it).
- **Shower enum is settled (non-bug):** API `shower_outlet_type` 1=non-electric(mixer)/2=electric
  (cohort 2636/0330 validate at 1e-4); types 3/4/5 are finer gov-output sub-types (type 3 is all
  on unsupported schema 19.1.0; type 4 already accurate). `shower_wwhrs` 1/2/3/4 = none / inst-
  WWHRS-1 / inst-WWHRS-2 / storage. Low headline value — not worth pursuing.

### SESSION-9 — profile sweep, six candidates ruled out (no shipped fix)
Re-ran `profile_api_error.py --min-n 10` at HEAD `da094feb` and chased every
biased/error-carrying bucket field-by-field. All resolved to proxy / already-deproven /
already-fixed — **no clean systematic bug found at this resolution**:
- **roof_construction code 5/8 (vaulted/sloping) under-rate** (gas: `5`−0.67, `4,5`−0.59,
  `4,8`−0.85): a PROXY. `u_roof` returns the SAME value for code 4 and code 5 at a given
  age/thickness (band G ND → 0.40 both; verified in `rdsap_uvalues.u_roof`). Code-5 certs
  correlate with electric HW / flats / multi-part dwellings — those are the cause, not the roof.
- **age_band dropped on the roof path:** NOT a bug. Doc-level `construction_age_band` is None for
  ALL 909 certs (age lives per-building-part); the mapper reads the bp-level band correctly
  (`heat_transmission.py:735 part.construction_age_band`; cert 2270 → bp0 age=G mapped fine).
- **roof description "(same dwelling above)"** (a heated dwelling above → should be zero-loss
  internal element): only 5 certs carry it and 4 are already within 0.5. Cert 0700 (−6.37) is a
  messy 4-building-part data-fidelity cert (mixed Flat-no-insulation + Roof-room parts), not a
  "same dwelling above" bug. Not systematic.
- **index-less MEV gas residual (the post-e6dda705 "next lead"):** RESOLVED by e6dda705. The 17
  index-less-MEV gas certs are now signed +0.09 / median +0.43 / 41% within-0.5 — centred scatter,
  not a systematic over-rate. The ~6 certs at +1.0..+1.9 are per-cert, no common cause.
- **wit=4 (as-built cavity) gas −0.25 over 478 certs:** tail-driven, not a uniform Table-6 shift.
  Splitting by age band, the worst bands (B −0.54, F −0.60, G −1.00) still hold 70–77% within-0.5
  — a few big-negative outliers drag the mean; correcting a Table-6 value would over-shift the
  majority that are already accurate.
- **community-main + whc=903 electric-immersion HW under-cost (−6.3, n=3):** = the deproven
  **meter_type=3** data-fidelity issue. All three (0380/2270/2673) carry meter_type=3; the HW
  electricity tariff ambiguity (lodged HW ≈17.3 p/kWh vs our standard ≈22.36 p) is exactly the
  Unknown-meter artifact already on this list.
- **The biggest +32 outlier 2958** (fuel=0 / sapcode=699): per-cert, NOT a class bug — cert 3420
  has an identical heating profile and sits at +0.18. 2958's error is fabric/geometry-specific.
Method note: `decompose_api_cost_error.py` cluster table = heat:high 311 (47% within) /
heat:low 206 / hw:low 173 / hw:high 125 / balanced 94 — i.e. the residual is a broad per-cert
fabric+HW tail, not one lever. The clean systematic wins (sheltered walls, MEV, fuel collisions)
are harvested; remaining headway is per-cert worksheet grind or the 100-cert schema big-ticket.

### SESSION-9 (cont.) — silent-fallback audit + Tier-1 strict-raise SHIPPED, `7878a969`
User pivoted from accuracy-chasing to a robustness audit: "where do we map codes but NOT raise
on an unmapped code (silent fallback)?" Four parallel Explore agents swept the API mapper,
`cert_to_inputs`, the U-value/table lookups, and the worksheet dispatch. Findings triaged into
tiers (full list in the session report I wrote inline). **Tier 1 (shipped):** the fuel-type
helpers fed Table 12/32 cost/CO2/PE via a silent `API_FUEL_TO_TABLE_12.get(fuel, fuel)`
passthrough at 5 sites → an unmapped/colliding fuel hit the **mains-gas default** in
`unit_price_p_per_kwh`/`co2_factor`/`primary_energy_factor` (the cert-8536 −17-SAP class). New
`_table_12_factor_fuel_code` is byte-identical to `.get(fuel,fuel)` for every recognised input
and raises `UnmappedSapCode("table_12_factor_fuel", …)` only when the resolved code is in NO
table. **Verified 0/909 corpus raises; eval unchanged 54.9%/909/0** — pure future-proofing.
2 AAA tests, goldens + gate green, pyright net-zero (44=44).
**Tiers NOT actioned (user chose Tier-1 only — candidates for a future robustness slice):**
- Tier 2: `_api_glazing_transmission` ([mapper.py:2925]) maps only glazing codes [1,2,3,13,14];
  4–12/15+ silently default U≈2.5 (the `reference_unmapped_api_code` "pending" gap). Window
  orientation drop ([solar_gains.py:391]) silently drops a window from solar gains on an unmapped
  orientation. Both would make currently-computed certs RAISE → need the codes MAPPED first, not
  just a guard added (coverage tradeoff).
- Tier 3 (low impact, bands are schema-bounded A–M): `.get(band, DEFAULT)` swallows a non-None
  unrecognised age band in `u_door` (→3.0), `u_basement_wall` (→0.7), `u_basement_floor` (→0.50),
  `u_roof` (→0.4). The `age_band is None` branches above each are justified absent-data defaults;
  only a typo/unknown band silently passing is the gap.

### SESSION-9 (cont. 2) — glazing single/secondary/triple per RdSAP 10 Table 24, `a0432977` (BIGGEST WIN)
Chasing Tier-2 turned up the session's biggest lever. `_API_GLAZING_TYPE_TO_TRANSMISSION` mapped
only the DOUBLE-glazing codes [1,2,3,13,14]; single (5/15), secondary (4/11/12), triple
(6/8/9/10) returned None → the cascade routed them to the `u_window` all-None default **U=2.5**
instead of their Table 24 value. **Single glazing (U=4.8) modelled at half its heat loss** was
the killer — 79 corpus certs carry code 5; the 94 certs with any unmapped glazing code sat at
**32% within-0.5 vs 54.9% baseline**. Extended the transmission + gap-override tables with the
RdSAP 10 Table 24 (spec p.50) (U, g, FF) rows for every RdSAP-21 glazing code. **Eval 54.90% →
56.66% within-0.5** (net +16: 22 in, 6 offsetting-error out), within-1.0 70.2→71.9, mean|err|
1.224→1.203, 909/0. 7 AAA tests, goldens + gate green, pyright net-zero. **Method that found it:
profile by `sap_windows[].glazing_type`, split known vs unmapped codes, decode against
`epc_codes.csv` `glazed_type`.**

### OPEN — KNOWN bugs still on the board (ranked)
1. **Glazing SOLAR/LIGHT g mis-keyed** (the natural follow-on): `_G_PERPENDICULAR_BY_GLAZING_TYPE`
   (solar_gains.py) + `_G_LIGHT_BY_GLAZING_CODE` (internal_gains.py) are keyed on the SAP-10.2
   Table-6b cascade ordering (1=single…) but `_api_cascade_glazing_type` only translates {1:2} —
   every OTHER API code passes through UNTRANSLATED, so API code 5 (single) reads the cascade-5
   slot (secondary/DG-low-E g) for solar+daylight gains. Smaller than the U effect (gains are
   second-order) but a real systematic mis-map. Fix = complete `_api_cascade_glazing_type` →
   Table-6b, OR feed the transmission-tuple g instead. Verify magnitude first.
2. **Window orientation silently dropped** (solar_gains.py:391 `if orientation is None: continue`)
   — an unmapped orientation code loses the whole window's solar gain. HIGH-confidence data-loss.
3. **Tier-2 glazing strict-raise** — now that codes 1-15 are mapped, make `_api_glazing_transmission`
   raise `UnmappedApiCode` on a present-but-unmapped (>15) code (zero current regression).
4. Tier-3 age-band swallows (low impact). MVHR (mech_vent=4) deferred. 21.0.0 builder missing the
   8/8b/8c ventilation + (now) glazing edits — extend if a 21.0.0 cert surfaces.
5. Per-cert: cert 0370-2933 still +15 after glazing (7 single windows but the over-rate is
   non-window — separate fabric/heating cause). The 100 unsupported-schema certs (big ticket).

### SESSION-9 (cont. 3) — glazing g remap (correctness, 0 impact) + post-glazing re-profile
- **Glazing g remap `49fb6c1b`:** completed the design's divergent-code remap —
  `_API_TO_SAP10_CASCADE_GLAZING_CODE` gained {4:5, 5:1} so API single (5) / secondary (4) read
  the right cascade g-slot (single 0.85/0.90, not secondary 0.76/0.80). CORRECTNESS ONLY: solar/
  daylight gains are second-order → 0 certs flip, eval unchanged 56.66%. The single-glazing
  *U-value* (a0432977) was the whole accuracy effect.
- **Re-profiled at 56.7%.** Remaining biased buckets ruled out as deproven / spec-faithful /
  tail-driven (verified, do NOT re-chase):
  - `wall_construction=3` solid brick gas (n=197, signed −0.52, survives gas-split): **spec-faithful,
    NOT a bug.** `u_wall` applies RdSAP §5.7 Table 13 thickness (≤200→2.5, 200-280→1.7, 280-420→1.4,
    >420→1.1); the API plumbs `wall_thickness` and the mapper passes it; wit=4 "insulated (assumed)"
    correctly takes the as-built U (no §5.8 reduction). Direction is also wrong for a thickness gap
    (using thickness LOWERS U → over-rate). Residual = old solid-brick houses outperform as-built.
  - `main_control=2113` (n=26, −0.76 but 77% within-0.5): tail-driven by messy multi-part certs
    (0700/9092), not uniform. `wall_construction=5` (timber, n=35): tail-driven (7921 −23, 0370 +15).
  - worst remaining under-rater **7921 (−23)**: roof code-8 sloping-ceiling "Pitched, insulated"
    thick=None → uninsulated 2.3 (roof 241 W/K). On the DEPROVEN list (= data-fidelity, we ≡ Elmhurst
    at uninsulated). Don't chase.
  - NOT yet run down (genuine open candidates): `main_heat_cat=4` heat pumps (n=20, −0.78),
    `water_fuel=20` community HW over-rate (n=40, +0.54, distinct direction — all sapcode 301).

## THE 100 unsupported_schema CERTS (deferred — bigger ticket)
SAP-Schema-19.1.0 (and other pre-21). The user is planning a separate big piece: map old schemas
→ new + **predict missing fields from similar-looking properties** (needs an EPC-prediction
method). That needs its own grilling session — do NOT start it here.

## WORKSHEET WORKFLOW (the user generates them on request)
For per-cert scatter that needs ground truth, ask the user to generate **P960 + Summary**
worksheets from the cert's OWN API JSON (`/tmp/epc_2026_sample/<cert>.json`). **Describe the cert
field-by-field first** (the user reproduces in Elmhurst; their repros are approximate — confirm
SAP matches lodged before pinning). Worksheets land under `sap worksheets/golden fixture
debugging/simulated case NN/` or `sap worksheets/additional with api 2/<cert>/`. Pin the cascade
to the P960 §3/§4/§9a/§10a line refs at abs=1e-4. **Caveat:** the user's repros often diverge
(wrong system / approximate inputs) — validate the BEHAVIOUR (e.g. λ, no-heat-loss) empirically
against the lodged SAP, don't blindly pin to a non-faithful repro.

## TOOLS & CONVENTIONS (non-negotiable)
- `scripts/eval_api_sap_accuracy.py` — headline + TOP-40 + `_results.csv`.
- `scripts/profile_api_error.py` — raw-API characteristic profiling (NEW, run first).
- `scripts/decompose_api_cost_error.py` — per-component cost decomposition (off-peak caveat: uses
  STANDARD elec price, mis-flags off-peak certs).
- ~1009 cached API JSONs at `/tmp/epc_2026_sample` (`EPC_SAMPLE_CACHE` overrides).
- **one cause = one slice = one commit**; **spec citation (page+line)** in the message; AAA test
  headers (`# Arrange/# Act/# Assert`); `abs(x-y)<=tol` not `pytest.approx`; **SAP 10.2 only**;
  **no tolerance-widening / xfail**; RdSAP is **deterministic** — every fix is a spec rule, not a
  population data-fit (the user is firm); pyright strict **net-zero** (baseline-compare via
  `git stash`); **stage files BY NAME** (tree carries unrelated `scripts/` + `sap worksheets/`
  changes — never `git add -A`); `Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>`.
- **REGRESSION after any calc/mapper change:** `tests/domain/sap10_calculator/`,
  `backend/documents_parser/tests/`, `datatypes/epc/`, golden fixtures (esp. **6035**).
- **Pre-existing failures to IGNORE** (fail on the stashed baseline too): `test_total_floor_area`
  and the 2 stone-wall U tests in `domain/sap10_ml/tests/test_rdsap_uvalues.py`.

## ARCHITECTURE NOTES (so you don't re-discover them)
- API path: `EpcPropertyDataMapper.from_api_response(doc)` → `cert_to_inputs(epc, prices=
  SAP_10_2_SPEC_PRICES)` → `calculate_sap_from_inputs(...).sap_score_continuous`.
- Cost path uses `inputs.fuel_cost` (Table-32/12a precompute); `_fuel_cost` returns a ZERO
  sentinel for off-peak → calculator falls back to the legacy scalar `_space_heating_fuel_cost_
  gbp_per_kwh` (which DOES carry the off-peak rate). SapResult fuel codes are RAW API enums —
  translate via `table_12.API_FUEL_TO_TABLE_12`.
- Heating efficiency: `_main_heating_detail_efficiency` → PCDB Table 105 winter eff (if PCDB
  index) else `seasonal_efficiency(code, cat, fuel)` (Table 4a/4b, in `domain/sap10_ml/
  sap_efficiencies.py`). Warm-air Table 4a code→eff map already covers 501-520.
- `sap10_ml/` is marked for eventual migration to `sap10_calculator/` but is still the live
  u-value/efficiency path.