junte/Model
1
0
Fork 0
mirror of https://github.com/Hestia-Homes/Model.git synced 2026-07-12 13:29:04 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
Model/docs/HANDOVER_API_PROFILING.md
Khalim Conn-Kowlessar dfba20babf docs: session-8 handover — API mechanical_ventilation enum → §2 MV-kind dispatch 
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 23:02:51 +00:00

30 KiB
Raw Blame History

Handover — API SAP accuracy (session 3): raises cleared, now profile-driven

Branch: feature/per-cert-mapper-validation (long-lived working branch — NEVER PR to main; the user pushes/PRs when ready). HEAD a8e5563a+ (the profiler commit), local-only ahead of origin.

READ ALSO: the auto-memory project_per_cert_mapper_validation_state (full slice log + deproven approaches + the meter/shower data-fidelity findings), and the earlier docs/HANDOVER_API_ACCURACY_S2.md (session-2 method).

THE GOAL (unchanged)

100% of API records with a lodged SAP compute within 0.5 SAP of the API's energy_rating_current. Headline gauge: PYTHONPATH=/workspaces/model python scripts/eval_api_sap_accuracy.py.

metric session-5 (19235d11) session-6 (a7761ea8) session-7 (3e05c95e) session-8 (71b378b9)
% |err| < 0.5 53.1% 53.14% 54.24% 55.01%
% |err| < 1.0 68.0% 67.99% 69.64% 70.08%
% |err| < 2.0 ~81% 81.85% 83.50% ~83.6%
mean |err| 1.329 1.312 1.248 1.233
median |err| 0.467 0.467 0.457 0.448
computed / raises 909 / 0 909 / 0 909 / 0 909 / 0
unsupported_schema 100 (deferred) 100 (deferred) 100 (deferred) 100 (deferred)

SESSION-8 — API mechanical_ventilation enum never mapped, 71b378b9

Re-profiling after the sheltering fix surfaced mechanical_ventilation=2 as a clean systematic over-rate (n=20, signed +1.90, 5% within 0.5, every cert positive). ROOT: from_api_response DROPPED the doc-level mechanical_ventilation field, so mechanical_ventilation_kind was always None and the §2 (24a..d) cascade defaulted to NATURAL — under-stating the air-change rate for every mechanical system (only the Elmhurst path mapped it). New _api_mechanical_ventilation_kind maps the RdSAP-Schema-21 enum → MechanicalVentilationKind: 0→NATURAL, 1 MV→"MV", 2/3 MEV + 6 PIV-outside → "EXTRACT_OR_PIV_OUTSIDE", 5 PIV-loft→NATURAL; code 4 (MVHR) DEFERRED (needs the lodged HR efficiency, PCDB Table 326, the API path doesn't plumb — stays NATURAL rather than mis-modelling as MV). Unmapped → UnmappedApiCode. Extract cohort +1.90→+0.9 median (within-0.5 5%→35%), 20 improved / 3 regressed. The +0.9 residual is the MEV FAN ELECTRICITY (§2.6.4 SFP, PCDB Table 322 decentralised-MEV — 9 of 20 carry mechanical_ventilation_index_number): a clean next slice. Goldens + regression green, pyright net-zero.

SESSION-7 — sheltered alternative walls (RdSAP Table 4 R=0.5), 3e05c95e

The headline-moving audit. User: 53% is poor enough to indicate a MAJOR error — audit again. The decisive diagnostic CHAIN (reusable):

  1. Error by dwelling_type → flats are the drag (houses 60-66% within 0.5, flats 28-47%; top-floor flat −1.19, mid-floor 28% within 0.5).
  2. Split flats by mains_gas → ELECTRIC flats are the killer (gas flats ~45%, electric flats 13-19%; top-floor electric mean|err| 3.62).
  3. Invert the SAP equation (ECF = 0.42·cost/(TFA+45) → sap_rating) to get our_cost / lodged_cost → electric flats over-cost ~3% median (houses 1.00), amplified by the 4× electric price + steep low-band log curve (+3% ≈ +1.5 SAP at band 40).
  4. Under-rate tracks space-heating kWh/m² precisely — accurate certs 14-110, under-rating certs 130-289 (cert 2021: 11 275 kWh for 39 m²) → over-stated FABRIC, not tariff (storage flats on the correct 5.5p rate under-rate just as much as room-heater flats at 13p).
  5. Field-by-field on the worst → cert 0340-2976 (band-A flat) computed wall 128 W/K though its main wall is a FILLED cavity (U 0.7); the excess was a SECOND u_wall call — a sap_alternative_wall_1 timber-frame sub-area at U=2.5 lodging sheltered_wall="Y".

ROOT: the gov-EPC API lodges sheltered_wall="Y" per alt-wall, but it was DROPPED by the schema + domain dataclasses, so _alt_wall_w_per_k billed the sub-area at its full exposed U. RdSAP 10 Table 4 (PDF p.22) "Sheltered": added external resistance R=0.5 m²K/W → U_sheltered = 1/(1/U + 0.5) — the SAME adjustment the MAIN wall already applies for gable_wall_type=2 (gable_wall_sheltered, _SHELTERED_GABLE_ADDED_RESISTANCE_M2K_W). Threaded end-to-end: schema (21.0.0/21.0.1) + domain SapAlternativeWall.is_sheltered (default False → Summary/Elmhurst path unchanged, goldens untouched) + from_api_response "Y"→True + _alt_wall_w_per_k applies the 0.5 resistance (lodged-U + basement alts return before it). 140 certs (15% of corpus) carry a sheltered alt-wall; they under-rated at median −0.82 / signed −1.33 / 23% within 0.5. Eval: 102 improved / 38 regressed (offsetting-error cases — applied spec-uniformly per the determinism principle); +10 net within-0.5. Goldens + full regression green, pyright net-zero. OPEN audit leftovers: electric-flat tail persists (2021 −3.9 genuine uninsulated solid brick = data-fidelity; top-floor "Flat, limited insulation" roofs); detached houses 45% (balanced bidirectional scatter); the 38 sheltering regressions = offsetting errors elsewhere; main-wall (non-gable) sheltering not audited; multi-element wall description joined-string-to-all-BPs (113 certs, mild).

SESSION-6 — community fuels 30/31/32 collide with electricity codes, a7761ea8

Picked up the deferred "fuel-collision part 2". The profiler's strongly-biased main_control=2306 bucket (n=11, signed −3.75, nearly uniform) was a PROXY: every cert in it is sap_main_heating_code=301 (community heating). ROOT: gov-API main_fuel_type codes 30=waste-combustion / 31=biomass / 32=biogas (all "(community)" in epc_codes.csv) collide in VALUE with the Table-32 electricity codes 30 (standard) / 31 (7h-low) / 32 (7h-high). All three sit in _ELECTRIC_FUEL_CODES, so is_electric_fuel_code flagged a community-scheme main as electric and _is_electric_main routed its cost through the off-peak electricity branch — BYPASSING _heat_network_factor_fuel_code. Cert 8536 (biomass community) billed 5.5 p grid electricity instead of the 4.24 p heat-network rate → −17.2 SAP. Per SAP 10.2 Table 12 the community waste/biomass/biogas rows are 42/43/44 (the same rows the backwards-compat enum codes 11/12/13 already map to). Added 30→42, 31→43, 32→44 to API_FUEL_TO_TABLE_12 + API_FUEL_TO_TABLE_32. CRITICAL — the remap is GATED, NOT global. A prototype putting 30/31/32 in canonical_fuel_code regressed certs 2211 (+16 SAP) and 3420 (+7): the cascade uses the bare Table-32 code 30 internally as _STANDARD_ELECTRICITY_FUEL_CODE (the RdSAP no-water-heating whc=999 immersion default writes water_heating_fuel=30), so a blanket remap mis-prices genuine grid electricity as community waste. New _heat_network_community_fuel_code(fuel, main) translates only when _is_heat_network_main(main) is true, wired into _main_fuel_code AND _water_heating_fuel_code (water gated via _water_heating_main(epc)). STRICT-RAISE (user-requested): a heat-network main lodging a colliding community fuel the table doesn't cover raises UnmappedSapCode rather than silently falling through to the same-numbered electricity code (currently can't fire — all of {30,31,32} are mapped — but guards future community fuels 56/57/58/99). Cert 8536 −17.25 → −6.51 (residual now flat fabric), 5036 −6.29 → +1.36; mean|err| 1.329→1.312, within-1.0/2.0 up, within-0.5 held. 3 AAA tests, regression green (only pre-existing 2 stone-wall U + a flaky historic-epc ordering test), pyright net-zero. STILL DEFERRED: dual-fuel API-9; the per-cert FABRIC tail (8536's −6.5 is now flat fabric, 2100 demand 110905); the 100 unsupported-schema certs.

SESSION-5 — fuel-code collision (anthracite/coal), 19235d11

The re-audit traced the cohort's WORST cert (2100 anthracite, −61) + the −20/−21 coal cluster to the fuel-code collision (reference_fuel_code_collision): the shared price/CO2/PE lookups check Table-32/12-code membership BEFORE translating the gov-API fuel enum, so API-5 (anthracite) priced at the bulk-LPG rate (12.19 p) and API-33 (coal) at the electricity-10h rate (7.5 p). KEY constraint (goldens caught it): code 33 is ALSO the electricity-10h TARIFF code used by the dual-rate CO2/PE split — so the fix CANNOT live in the shared table functions (breaks golden 000565). Instead canonical_fuel_code (table_32) normalises the colliding SOLID-fuel enums at the fuel-TYPE boundary (_main_fuel_code/_water_heating_fuel_code): 5→15 anthracite 3.64 p, 33→11 house coal 3.67 p; also fixes is_electric_fuel_code(33) mis-flagging coal as electric. Scoped to {5, 33} (unambiguous large mispricings). mean|err| 1.424→1.329 (2100 −61→−11, residual now FABRIC: 110 905 kWh demand = a separate area over-statement); within-0.5 flat at 53.1%. DEFERRED follow-ups: dual-fuel API-9 (0.45 p delta, net-neutral, shifts certs in an un-understood direction — needs its own look); community API-20/25/31 (route through the heat-network standing/CO2 path, NOT unit_price — cert 8536 fuel-31 still mis-prices at 5.5 p electricity → −17). Method: the decompose_api_cost_error.py heat:high/low tail + field-by-field audit of the worst certs surfaced it; is_electric_fuel_code collisions are the tell.

THE UNIFYING PRINCIPLE (user, load-bearing) — "unknown insulation → as-built, NOT uninsulated"

An EPC insulation field that is UNDETERMINED (thickness 'ND'/'AB'/absent → parsed None, or description "as built / (assumed)") must map to the age-band default ("as built"), which is INSULATED at newer bands — never to the uninsulated row. The recurring bug shape: a fixed uninsulated U (cavity Filled-row, roof Table-16 2.30) is MASKED at old bands (where the age default coincides with uninsulated) and only diverges (catastrophic under-rate) at newer bands. All three session-5 fixes are instances. Review status across elements (all now conform):

  • Flat roofs — FIXED 58cff932 (this slice). Pitched roofs — "unknown"→Table 18 (a64e857b); "no insulation" only appears at bands A/B where 2.30 IS the age default (verified, no new-band bug).
  • Cavity walls — FIXED 2e466ed1. System/timber/solid walls — already on the as-built age row (verified: bidirectional scatter, not a one-cause under-rate). Floors — undetermined thickness already routes to the Table 19 age default (I=25/J=75/K=100 mm; verified).
  • CAVEAT: do NOT broadly reroute PITCHED 'ND'/'NI'→Table 18 (the parsed-0 'NI' case) — that was empirically net-negative (pitched "no insulation" lodgements genuinely use 2.30 even at newer bands; the description is load-bearing for pitched lofts). The principle holds for flat roofs, cavity, floors; pitched lofts are the documented exception.

SESSION-5 UPDATE (HEAD 2e466ed1) — whc=903 immersion HW + as-built cavity-U both closed

Shipped (47.6 → 52.1%, two spec-grounded fixes):

(2) 2e466ed1 as-built "insulated (assumed)" cavity → Cavity-as-built row, not Filled cavity (48.6 → 52.1%, the bigger win). Robust-sweep lead: wall_desc="Cavity wall, as built, insulated (assumed)" median +0.26, n=145, but split by age band it was a CLEAN G/H signal (G +1.38 n37, H +1.61 n18; I-L neutral). RdSAP 10 Table 6 (England, p.41) "Filled cavity" row carries the † footnote ("assumed as built") ONLY at bands I-M, where it equals "Cavity as built"; at A-H the filled row is a GENUINE fill. An as-built cavity (type 4) must use "Cavity as built" at all bands (G/H 0.60 not 0.35). This was the SAME latent A-H bug slice S0380.210 fixed for "partial insulation (assumed)" but left for "insulated (assumed)" by a legacy convention. Retired _cavity_described_as_filled; genuine fills (wall_insulation_type=2) still hit the filled row. Per-band confirmation: I-M unchanged, G/H corrected exactly. Bucket within-0.5 47% → 66%; eval +32 net (36 improved, 4 regressed — offsetting-error electric-storage flats). 3 tests updated to the corrected behaviour (the legacy tests literally said "we follow the legacy convention for parity").

(1) 43d4c67d WHC-903 electric immersion off-peak HW → SAP 10.2 Table 13 high-rate fraction (47.6 → 48.6%). The session-4 robust-sweep whc=903 lead (median +0.87, n=84).

  • 43d4c67d WHC-903 electric immersion off-peak HW → SAP 10.2 Table 13 high-rate fraction. Was billing 100% at the off-peak low rate; Table 12a "Immersion water heater" row (p.191) routes the WH column to Table 13 (p.197). New tables/table_13.py evaluates the Note-2 equations (f of cylinder volume V, occupancy N, single/dual immersion), clamped [0,1]; 18-/24-hour use the 10-hour column (Note 1). Wired into _hot_water_fuel_cost_gbp_per_kwh (threads V / N / immersion-single from the caller; absent any → old 100%-low fallback, no regression). Off-peak WHC-903 cohort (n=57): within-0.5 16% → 33%, median |err| 1.56 → 0.86.
  • IMMERSION CODE MAPPING CORRECTED: immersion_heating_type 1 = DUAL, 2 = SINGLE. The session-3 handover lead #3's "(1=single, 2=dual)" was UNVERIFIED and BACKWARDS. Confirmed via RdSAP 10 §10.5 (p.54 — immersion "assumed dual" on a dual/off-peak meter) cross-checked with the cohort: code 1 sits 3.6:1 on dual meters (40 vs 11), code 2 on single meters (22 vs 16). Dual carries Table 13's small fraction → matches the over-rating direction; the single mapping overshot in a prototype (cohort within-0.5 16% → 14%). The description-vs-code-audit lesson again: skeptical of unverified handover code-semantics claims.
  • Next robust leads (post-BOTH-fixes sweep, ranked by net directional skew + MEDIAN): every top bucket is now an UNDER-rate cluster (negative median = fabric/flat scatter, per-cert not one-bug): property_type=2 flats med −0.39 (n=283, netDir +75), roof "(another dwelling above)" −0.46 (n=182), wall_desc="Solid brick, as built, no insulation" −0.22 (n=114). No clean OVER-rate single-cause bucket remains (cavity-insulated dropped to −0.13, main_heat_cat=7 to −0.31, whc=903 off the top — all addressed). The flats under-rate is the biggest front but DIFFUSE (fabric/tariff per-cert) — likely needs worksheets, not one rule. The 100 unsupported-schema certs remain the deferred big ticket. METHOD NOTE: the cavity win came from splitting the +0.26 bucket BY AGE BAND — the mild median hid a sharp G/H spike. When a description bucket has a modest median but a plausible single mechanism, re-split by age band / sub-field before dismissing it as scatter.

SESSION-4 shipped (45.1 → 47.6%): four spec-grounded fixes + closed one false lead. See the ## SESSION-4 … blocks below and the auto-memory for full detail. The systematic bias is gone; the winning method this session was the description-vs-code audit + an outlier-robust categorical sweep (rank by net directional skew + MEDIAN, not mean — the mean-based metric is fooled by multi-cause outliers). 47.6% is still the target's halfway point.

WHAT SHIPPED THIS SESSION (7 slices, all green, pyright net-zero)

  1. e41a0bc0 PCDB heat pump w/o SAP code → Table 12a ASHP_APP_N SH split (0.80 high-rate).
  2. 2bc73fb0 HP-DHW (WHC 901/902/914 + PCDB HP) → Table 12a WH 0.70 split. Together (1)+(2) killed the cat-4 heat-pump over-rating bias (+1.43 → +0.06).
  3. 449d8c5b direct-acting electric boiler (191) → zero primary circuit loss (SAP Table 3 p.160 zero list names it verbatim).
  4. f4048588 wall_insulation_thermal_conductivity ignored → §5.8 default λ=0.04. (See KEY INSIGHT below — the gov field is RdSAP output, not an input.)
  5. 1c5675a0 floor_heat_loss=8 → no floor heat loss (extension floor over a heated space; RdSAP §3, like code 6).
  6. a8e5563a main_heating_category=9 (warm air) → Table 11 secondary fraction 0.10. (4)(5)(6) cleared all 4 raises — eval now has zero raises.
  7. (profiler) scripts/profile_api_error.py — the new diagnostic (below).

SESSION-4 UPDATE (HEAD faf29942) — read before re-working the leads below

Shipped this session (45.1 → 47.6%)

  1. b40e0f67 exposed-floor-on-flats (floor_heat_loss=1) — §3.12; per-BP override of the dwelling-level flat suppression.
  2. 8741fbdf floor_heat_loss=3 → above partially heated space, U=0.7 (§3.12/§5.14) + re-pinned golden 7536 (its "irreducible residual" was THIS bug).
  3. 5e7ef5c7 boiler interlock for TRVs+bypass controls 2107/2111 (§9.4.11) — biggest single win (+1.6pts). The no-interlock set was keyed off the wrong signal (the "+0.6 °C" annotation); 2107/2111 lack a room thermostat → −5pp + Table 4f ×1.3 pump.
  4. faf29942 description-lodged secondary heating (§A.2.2/Table 11) — gas/oil boilers with an API-description-only secondary ("Portable electric heaters (assumed)", code field None) dropped the secondary (sec_kWh=0); now _has_lodged_secondary_description fires Table 11. Also added cat-8 (electric underfloor) Table-11 fraction 0.10.
  • 560c912c/d0f57a0e docs: roof_construction=8 lead CLOSED as data-fidelity (not a bug — see the roof-8 section below; user worksheet sim-case-29 proved we ≡ Elmhurst).

The two methods that worked (reuse these)

  • Description-vs-code audit: join each int code (floor_heat_loss, roof_construction, wall_construction, secondary type) to its authoritative …[].description, on single-element certs only (multi-element [] arrays are LOSSY). Mis-maps fall out (floor-3, secondary).
  • Outlier-robust categorical sweep (/tmp/cat_audit2.py): rank field-values by net directional skew (#under−0.5 minus #over+0.5) + MEDIAN error. The mean-based directionality metric (/tmp/cat_audit.py) gets FOOLED by multi-cause outliers (e.g. "Solid brick no insulation" looked systematic at mean −1.07 but median is −0.22 = scatter; 2100 −61/RR drove it).

Open robust leads (verify with /tmp/cat_audit2.py — they shift; check MEDIAN not mean)

  • whc=903 electric-immersion HW: median +0.87, n=84 — likely off-peak immersion handling (the handover noted WHC 903 raises NotImplementedError on the Table-12a off-peak-immersion row).
  • main_heat_cat=7 electric storage: median +1.05, n=41 — over-rate (tariff/cost; partly artifact).
  • immersion_type=2 dual: +1.50, n=43 — we OVER-credit (so §12 dual→off-peak would worsen it).
  • dwelling_type=Top-floor flat: median −1.24, n=99 — under-rate, mostly fabric scatter/artifacts.
  • Low-dir = SCATTER, do NOT single-fix: non-PCDB main / data_source=2 (n=242, 28% within-0.5), mains_gas=N electric (n=145), most flats. These are per-cert/data-fidelity, not one bug.

Resolved/closed this session (don't re-chase)

  • Lead #1 floor_codes=3 RESOLVED — the code IS authoritative. The diagnostic that cracked it: join each single-BP cert's floor_heat_loss code to its independent floors[].description (the multi-BP tally was contaminated because a cert's floors[] summary is LOSSY — it drops some BPs' descriptions). Single-BP gives a perfect 1:1 enum: code 1↔"To external air"(exposed), 2↔"To unheated space"(semi-exposed), 3↔"(other premises below)" (9/9), 6↔"(another dwelling below)"(party), 7↔Solid/Suspended(ground). Per RdSAP §3.12 (p.25) code 3 = "above a partially heated space" (non-domestic premises below) → §5.14 constant U=0.7 (NOT Table-20 semi-exposed, NOT ground). SHIPPED 8741fbdf.
  • SHIPPED b40e0f67: exposed-floor-on-flats (code 1) area fix — §3.12. A flat's code-1 floor was area-zeroed by _dwelling_exposure; now the per-BP is_exposed_floor overrides the flat suppression upward (mirrors the "another dwelling below" party override).
  • SHIPPED 8741fbdf: code 3 → is_above_partially_heated_space (U=0.7) + area override. RE-PINNED golden 7536-3827 — its Ext2(bp3) code-3 floor was mis-read as "ground U=1.12" by a prior agent (the lossy floors[] dropped its description), who declared the residual an "irreducible register-rounding artifact, DO NOT chase". It was this bug: U 1.12→0.70, PE/CO2 residuals moved toward 0. LESSON: "irreducible residual" golden notes are suspect — a real mapper bug can hide there. Eval (both slices): 45.1→45.3%, mean|err| 1.702→1.659, <1.0 59.5→60.2%. User is generating a fresh 0380-2087-8190-2996-3075 worksheet to independently confirm U=0.7 (0380 now −0.63) — validate when it lands.
  • Leads re-checked, NOT clean: immersion_type=2 (+1.86) is high-scatter (mean|err| 3.71, bidirectional). main_control=2107 (+1.63) is correctly mapped ("Programmer, TRVs and bypass" type 2 Table 4c(2)) — over-rate is diffuse gas-boiler/flat-fabric, not a dispatch bug. roof_codes=1 broad bucket is mean −0.15 (the −1.78 was top-floor-electric-flat outliers −29/−25). Remaining gains need per-cert worksheets (start code-3) or the unsupported-schema ticket.

SESSION-4 AUDIT — description-vs-code cross-check (other element types)

After floor-3, audited every API field that carries BOTH an integer code AND an independent …[].description (roofs, walls, floors, main_heating, controls), by joining code↔description on single-element certs (the multi-element [] summary is lossy).

  • Walls / heating / controls: CLEAN — codes map 1:1 to their description families; residual error is per-cert scatter, not mis-mapping.
  • Roof roof_construction=8: investigated hard, NOT a bug — DO NOT re-chase. It looked like a huge lead (n=57, ~189 |err|, mean −2.43) because code-8 sloping ceilings described "Pitched, insulated" with no numeric thickness compute U=2.30 (e.g. 7921-0052 roof_w 241 W/K, SAP 56.75 vs lodged 80). User worksheet (simulated case 29) settled it: at band C, RdSAP/Elmhurst assumes UNINSULATED for an "insulated (assumed)" roof+wall with no measured thickness → Elmhurst SAP 55, which MATCHES our 56.75. The lodged 80 needed real insulation the API record doesn't carry = data-fidelity artifact (meter-3 class). KEY LESSON (user): "insulated (assumed)" = the age-band DEFAULT insulation level, NOT "well insulated" — at old bands that default is ~uninsulated. Re-audit by band confirmed: old-band no-numeric = artifacts (we ≡ Elmhurst); newer-band code-8 already gets correct insulated U (2031 band I U≈0.18, 1436 band E 0.17, 0536 band B 0.09) — their under-rates are elsewhere; numeric- thickness = accurate (9884 +0.06). So the force-uninsulated-at-pre-1950 rule is CORRECT. A roof-8 "fix" would push us AWAY from Elmhurst's faithful 55 toward the unreproducible 80.
  • Roof roof_construction=3: latent mis-map (inert). gov desc "(another dwelling above)" (party roof) but we map to "Pitched, no access to loft"; masked by dwelling exposure (mean −0.06). Correct for robustness only if touching the roof mapper; not worth chasing.
  • Worksheet-gen constraints (user, for future repros): Elmhurst no longer lets you pick build form for a flat; and a band-C repro defaults to uninsulated walls+roof.

KEY INSIGHT (load-bearing, from the user)

The gov EPC API JSON is the published OUTPUT of RdSAP software (Elmhurst), not its input. So any API field Elmhurst doesn't expose as an input is register metadata the RdSAP10 method does not consume — route it to the spec default, don't try to "use" it. This is exactly why wall_insulation_thermal_conductivity (slice 4) → always λ=0.04. Apply the same lens to any new "extra" API field before wiring it.

THE NEW DIAGNOSTIC — scripts/profile_api_error.py (run this first)

PYTHONPATH=/workspaces/model python scripts/profile_api_error.py joins each computed cert's signed error with a rich feature set from its raw API JSON (not the mapped EpcPropertyData), and ranks (feature, value) buckets by error carried + by |mean signed| bias. This is how to find "silly API-path handling" gaps. --min-n N sets the bucket floor.

PRIORITISED LEADS (from the run at a8e5563a — verify with the profiler, they'll shift)

Cleanest "API-path handling" candidates first (small, biased buckets = likely a mapper/dispatch bug, not noise):

  1. floor_codes=3 → mean signed +5.37 (n=10). We map API floor_heat_loss=3 → "To unheated space" (same as code 2). The +5.37 over-rate says that's wrong — code 3 likely isn't "unheated space" (or its U is wrong). Pull the n=10 certs, check what code 3 really is (ask the user the Elmhurst floor dropdown — the API=output lens). Highest bias, smallest scope = start here.
  2. Control-code biases: main_control=2306 −2.96 (n=11), 2602 +2.49 (n=14), 2107 +1.65 (n=38), 2402 +1.14 (n=10), 2307 +0.74 (n=11). Several control codes carry systematic bias → Table 4c/4e control dispatch gaps. 2107/2602 are the biggest. Check _CONTROL_TYPE_BY_CODE + the Table 4c efficiency-adjustment / Table 4e control coverage.
  3. immersion_type=2 (dual immersion) → +2.00 (n=43, mean|err| 3.85). RdSAP §12 lists "dual electric immersion" as an off-peak trigger; the cascade does NOT consume immersion_heating_type for tariff (verified — only comments reference it). Wiring the §12 dual-immersion → off-peak rule for Unknown meters is a clean spec slice. (1=single, 2=dual per the Elmhurst Summary.)
  4. roof_codes=1 −1.78 (n=27) (flat roof under-rate) and roof_insulation_thickness=None −1.18 (n=52) — flat-roof / no-thickness roof handling.
  5. main_data_source=2 / has_pcdb_main=False → 28% within 0.5, mean|err| 3.17 (n≈242). Non-PCDB heating systems (SAP-table efficiency) are a big under-rating cluster. Likely Table 4b default-efficiency or fabric, but worth a look — it's 1/4 of the sample.

Big scattered segments (need worksheets, NOT clean single fixes)

  • whc=903 (electric immersion HW): 13% within 0.5, n=84 — looks like the worst bucket but it's the electric storage(cat-7)+room-heater(cat-10) segment compounding (worst certs span −29…+32, bidirectional). Not one bug.
  • mains_gas=N (electric): 21% within 0.5, mean|err| 4.27 (n=145) — the hardest segment; per-cert fabric/tariff scatter.
  • Flats (property_type=2): 31% within 0.5 (n=283) — still the worst dwelling type.
  • cat-7 storage (+0.75) / cat-10 room heaters (+0.75) — both net over-rate; bidirectional.

DEPROVEN — do NOT retry (empirically failed in earlier sessions; details in memory)

  • Routing roof 'ND' → Table 18 (description is load-bearing even with 'ND').
  • Broad "all Unknown(meter 3) electric → off-peak" (over-credits room heaters). NOTE: the meter-3 under-rate is partly an irreducible data-fidelity artifact — the register stores meter_type=3 ("Unknown") on certs whose lodged rating actually used an off-peak meter (cert 2474: lodged 78 needs 18-hour, but API says Unknown → spec-faithful ~68). Don't chase those to the lodged value.
  • RR shell U Table-17-50mm (golden 6035 disproves it).
  • Shower enum is settled (non-bug): API shower_outlet_type 1=non-electric(mixer)/2=electric (cohort 2636/0330 validate at 1e-4); types 3/4/5 are finer gov-output sub-types (type 3 is all on unsupported schema 19.1.0; type 4 already accurate). shower_wwhrs 1/2/3/4 = none / inst- WWHRS-1 / inst-WWHRS-2 / storage. Low headline value — not worth pursuing.

THE 100 unsupported_schema CERTS (deferred — bigger ticket)

SAP-Schema-19.1.0 (and other pre-21). The user is planning a separate big piece: map old schemas → new + predict missing fields from similar-looking properties (needs an EPC-prediction method). That needs its own grilling session — do NOT start it here.

WORKSHEET WORKFLOW (the user generates them on request)

For per-cert scatter that needs ground truth, ask the user to generate P960 + Summary worksheets from the cert's OWN API JSON (/tmp/epc_2026_sample/<cert>.json). Describe the cert field-by-field first (the user reproduces in Elmhurst; their repros are approximate — confirm SAP matches lodged before pinning). Worksheets land under sap worksheets/golden fixture debugging/simulated case NN/ or sap worksheets/additional with api 2/<cert>/. Pin the cascade to the P960 §3/§4/§9a/§10a line refs at abs=1e-4. Caveat: the user's repros often diverge (wrong system / approximate inputs) — validate the BEHAVIOUR (e.g. λ, no-heat-loss) empirically against the lodged SAP, don't blindly pin to a non-faithful repro.

TOOLS & CONVENTIONS (non-negotiable)

  • scripts/eval_api_sap_accuracy.py — headline + TOP-40 + _results.csv.
  • scripts/profile_api_error.py — raw-API characteristic profiling (NEW, run first).
  • scripts/decompose_api_cost_error.py — per-component cost decomposition (off-peak caveat: uses STANDARD elec price, mis-flags off-peak certs).
  • ~1009 cached API JSONs at /tmp/epc_2026_sample (EPC_SAMPLE_CACHE overrides).
  • one cause = one slice = one commit; spec citation (page+line) in the message; AAA test headers (# Arrange/# Act/# Assert); abs(x-y)<=tol not pytest.approx; SAP 10.2 only; no tolerance-widening / xfail; RdSAP is deterministic — every fix is a spec rule, not a population data-fit (the user is firm); pyright strict net-zero (baseline-compare via git stash); stage files BY NAME (tree carries unrelated scripts/ + sap worksheets/ changes — never git add -A); Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>.
  • REGRESSION after any calc/mapper change: tests/domain/sap10_calculator/, backend/documents_parser/tests/, datatypes/epc/, golden fixtures (esp. 6035).
  • Pre-existing failures to IGNORE (fail on the stashed baseline too): test_total_floor_area and the 2 stone-wall U tests in domain/sap10_ml/tests/test_rdsap_uvalues.py.

ARCHITECTURE NOTES (so you don't re-discover them)

  • API path: EpcPropertyDataMapper.from_api_response(doc) → cert_to_inputs(epc, prices= SAP_10_2_SPEC_PRICES) → calculate_sap_from_inputs(...).sap_score_continuous.
  • Cost path uses inputs.fuel_cost (Table-32/12a precompute); _fuel_cost returns a ZERO sentinel for off-peak → calculator falls back to the legacy scalar _space_heating_fuel_cost_ gbp_per_kwh (which DOES carry the off-peak rate). SapResult fuel codes are RAW API enums — translate via table_12.API_FUEL_TO_TABLE_12.
  • Heating efficiency: _main_heating_detail_efficiency → PCDB Table 105 winter eff (if PCDB index) else seasonal_efficiency(code, cat, fuel) (Table 4a/4b, in domain/sap10_ml/ sap_efficiencies.py). Warm-air Table 4a code→eff map already covers 501-520.
  • sap10_ml/ is marked for eventual migration to sap10_calculator/ but is still the live u-value/efficiency path.