Model/docs/HANDOVER_PASHUB_EXTRACTOR_ACCURACY.md

# PasHub extractor accuracy — handover (2026-07-14 late)

## Goal
Guinness GMCA 205-property cohort within 0.5 SAP of pashub's `pre_sap`
(PRD #1555). Khalim: "We can get to 100% within 0.5 for this cohort." The
bugs are **extraction** (survey detail not reaching the calculator), not the
calculator — proven repeatedly this session.

## Where we are (branch `feat/pashub-extractor-fixes-and-runner`, pushed)
Harness `backend/documents_parser/tests/test_pashub_sap_accuracy.py`:
**205/205 compute, MAE 1.551, within-0.5 18.5% (38/205), bias ≈ −1.0**
(from 12.4%/2.56 at session start). Ratchets `_MIN_WITHIN_HALF=0.18`,
`_MAX_SAP_MAE=1.56` — tighten after every fix, comment each move.

### Fixed this session (all TDD 🟥/🟩 pairs; see git log on the branch)
Issue **#1590** (all 7 subtasks done + 2 round-2 fixes; trajectory table in
its last comment): secondary fuel labels; Table 4e Group 3 control 2306;
roof "Insulation At: None"→explicit 0; **PV arrays extracted+mapped**
(`_pashub_pv_arrays`); `pv_connection`→int (1/2); **mechanical ventilation**
`_pashub_mechanical_ventilation_kind` (biggest win: 76 fixtures, MAE −0.8);
cylinder size Table 28; system-build/basement code-6 collision;
**room-in-roof** (survey `RoomInRoofDetail` → extractor `_parse_room_in_roof`
→ mapper `_pashub_room_in_roof`; commons are `stud_wall`, roof-space
insulation threads into slope/stud/flat surfaces); **community heating**
(extractor captures "Heating System (Other):" → `_pashub_community_heating`
→ `sap_main_heating_code=301` + Table 12 fuel 51 via
`_resolve_community_heating_fuel_code`).

Issue **#1589** (separate, unfixed): modelling `refetch_epc=True` violates
ADR-0001 recency — gov API overrides newer stored site notes.

## Ground truth — now verified
`pre_sap` in the manifest was hand-verified by Khalim in pashub for the 4
worst deviations (2026-07-14): manifest corrected for 499584755922 (F33, was
stale D67 — we were right) and 499617935574 (accredited E43, was preliminary
53). **Confirmed correct** (so our deviation = extraction bug to hunt):

| deal | address | pashub | ours | d |
|---|---|---|---|---|
| 499530733767 | 58 Hackle Street, M11 4WU | D58 | 65.8 | **+7.8** |
| 499617935574 | Brightholme, 14 North Road, M11 4WE | 43 | 50.3 | **+7.3** |
| 507644414148 | 16 Bingley Close, M11 3RF | E52 | 57.7 | **+5.7** |

Rule: don't tune to `pre_sap` blindly, but after this verification treat it
as trustworthy unless a survey PDF plainly contradicts it (then ask Khalim to
check pashub — give him the ADDRESS, not the deal id).

## The proven working loop (repeat until 100%)
1. Rank divergences: parse every manifest fixture (`parse_site_notes_pdf` →
   `Sap10Calculator().calculate`) vs `pre_sap_score`; list worst signed.
2. Fan out 3-5 Sonnet sub-agents, one per worst case, with THE RECIPE:
   dump extracted `EpcPropertyData` fields; dump PDF text (`fitz`); diff
   field-by-field against the survey; attribute via `dataclasses.replace`
   patch-and-rerun (record SAP delta per candidate fix); report the
   mis-extracted field + concrete fix + deltas. Investigation only, no edits.
3. TDD each confirmed fix (`/tdd` skill): RED commit 🟥 → GREEN commit 🟩,
   test in `datatypes/epc/domain/tests/test_from_site_notes.py` (mapper) and/or
   `backend/documents_parser/tests/test_extractor.py` (extractor; splice
   verbatim PDF lines into `load_text_fixture()`); update the golden
   `test_full_mapping` when field coding changes.
4. Re-run harness; ratchet; commit; push.

## Conventions (non-negotiable)
- `_pashub_*` helpers strict-raise `UnmappedPasHubLabel` on unknown non-empty
  labels; blank passes through/None. Mirror `_ELMHURST_*`/`_api_*` siblings.
- pyright: zero NEW errors (baselines: mapper.py 39, extractor.py 34,
  test_from_site_notes.py ≤13). Beware `Union[RoofSpaceDetail,
  ExtensionRoofSpace]` attribute access — add fields to BOTH.
- Survey dataclasses hold RAW labels; coding happens at the mapper boundary
  (ADR-0015). Ratchets never loosen at fixed coverage.

## Known leads for the remaining gap
- **Solid ground floors**: worst-biased category (n=107, mean −1.53; 5/7 of
  the −4..−5 Natural-ventilation cluster: 499529160912, 499538003156,
  499627236560, 499586692333, 507533541571, 499570197710, 499592219845).
  Dig the ISO 13370 solid-floor U/perimeter cascade vs what the surveys lodge.
- 16 Bingley (+5.7): community-heating residual — water heating is
  "Hot water only community scheme - boilers" (WHC 950 path?), community
  standing charge/tariff detail, or DLF "Unknown" distribution type.
- 58 Hackle (+7.8): undug — full recipe pass needed.
- Brightholme (+7.3): RIR dwelling; the accredited 43 reflects a QA
  correction ("no access to loft — hatch screwed shut") — possibly RIR
  insulation should NOT thread from the roof-space block when access was
  impossible, or slopes should default worse. Check `_pashub_room_in_roof`.
- Mechanical ventilation and PCDB efficiency are VERIFIED CORRECT — don't re-dig.

## Also outstanding
- **Open the review PR** for the branch (metrics table = #1590's last
  comment; ~26 commits, all green: 206 harness + ~294 unit tests).
- Portfolio 838 modelling: re-extract 4 properties in pashub (754772/9
  Philips Park Court, 754780/12 Seymour Rd S, 754844/16 Bingley — the
  community fix now makes it extractable, 754816/130 Stanton St has no site
  note at all); then `RUN_DRY=0 python scripts/run_pashub_modelling.py`
  (env-parametrised: RUN_PORTFOLIO/RUN_SCENARIO/RUN_PIDS/RUN_DRY).
- `effective_sap_score` == the pashub rating for unchanged lodged dwellings
  (rebaseliner pass-through) — NEVER validate the calculator against it.
- hubspot_deal_data still holds the stale pre_sap for the 2 corrected deals —
  re-running `scripts/build_pashub_accuracy_fixtures.py` will REVERT the
  manifest corrections unless the DB rows are fixed first.