Model/docs/HANDOVER_PASHUB_EXTRACTOR_ACCURACY.md

PasHub extractor accuracy — handover (2026-07-14 late)

Goal

Guinness GMCA 205-property cohort within 0.5 SAP of pashub's pre_sap (PRD #1555). Khalim: "We can get to 100% within 0.5 for this cohort." The bugs are extraction (survey detail not reaching the calculator), not the calculator — proven repeatedly this session.

Where we are (branch feat/pashub-extractor-fixes-and-runner, pushed)

Harness backend/documents_parser/tests/test_pashub_sap_accuracy.py: 205/205 compute, MAE 1.551, within-0.5 18.5% (38/205), bias ≈ −1.0 (from 12.4%/2.56 at session start). Ratchets _MIN_WITHIN_HALF=0.18, _MAX_SAP_MAE=1.56 — tighten after every fix, comment each move.

Fixed this session (all TDD 🟥/🟩 pairs; see git log on the branch)

Issue #1590 (all 7 subtasks done + 2 round-2 fixes; trajectory table in its last comment): secondary fuel labels; Table 4e Group 3 control 2306; roof "Insulation At: None"→explicit 0; PV arrays extracted+mapped (_pashub_pv_arrays); pv_connection→int (1/2); mechanical ventilation _pashub_mechanical_ventilation_kind (biggest win: 76 fixtures, MAE −0.8); cylinder size Table 28; system-build/basement code-6 collision; room-in-roof (survey RoomInRoofDetail → extractor _parse_room_in_roof → mapper _pashub_room_in_roof; commons are stud_wall, roof-space insulation threads into slope/stud/flat surfaces); community heating (extractor captures "Heating System (Other):" → _pashub_community_heating → sap_main_heating_code=301 + Table 12 fuel 51 via _resolve_community_heating_fuel_code).

Issue #1589 (separate, unfixed): modelling refetch_epc=True violates ADR-0001 recency — gov API overrides newer stored site notes.

Ground truth — now verified

pre_sap in the manifest was hand-verified by Khalim in pashub for the 4 worst deviations (2026-07-14): manifest corrected for 499584755922 (F33, was stale D67 — we were right) and 499617935574 (accredited E43, was preliminary 53). Confirmed correct (so our deviation = extraction bug to hunt):

deal	address	pashub	ours	d
499530733767	58 Hackle Street, M11 4WU	D58	65.8	+7.8
499617935574	Brightholme, 14 North Road, M11 4WE	43	50.3	+7.3
507644414148	16 Bingley Close, M11 3RF	E52	57.7	+5.7

Rule: don't tune to pre_sap blindly, but after this verification treat it as trustworthy unless a survey PDF plainly contradicts it (then ask Khalim to check pashub — give him the ADDRESS, not the deal id).

The proven working loop (repeat until 100%)

1. Rank divergences: parse every manifest fixture (parse_site_notes_pdf → Sap10Calculator().calculate) vs pre_sap_score; list worst signed.
2. Fan out 3-5 Sonnet sub-agents, one per worst case, with THE RECIPE: dump extracted EpcPropertyData fields; dump PDF text (fitz); diff field-by-field against the survey; attribute via dataclasses.replace patch-and-rerun (record SAP delta per candidate fix); report the mis-extracted field + concrete fix + deltas. Investigation only, no edits.
3. TDD each confirmed fix (/tdd skill): RED commit 🟥 → GREEN commit 🟩, test in datatypes/epc/domain/tests/test_from_site_notes.py (mapper) and/or backend/documents_parser/tests/test_extractor.py (extractor; splice verbatim PDF lines into load_text_fixture()); update the golden test_full_mapping when field coding changes.
4. Re-run harness; ratchet; commit; push.

Conventions (non-negotiable)

• _pashub_* helpers strict-raise UnmappedPasHubLabel on unknown non-empty labels; blank passes through/None. Mirror _ELMHURST_*/_api_* siblings.
• pyright: zero NEW errors (baselines: mapper.py 39, extractor.py 34, test_from_site_notes.py ≤13). Beware Union[RoofSpaceDetail, ExtensionRoofSpace] attribute access — add fields to BOTH.
• Survey dataclasses hold RAW labels; coding happens at the mapper boundary (ADR-0015). Ratchets never loosen at fixed coverage.

Known leads for the remaining gap

• Solid ground floors: worst-biased category (n=107, mean −1.53; 5/7 of the −4..−5 Natural-ventilation cluster: 499529160912, 499538003156, 499627236560, 499586692333, 507533541571, 499570197710, 499592219845). Dig the ISO 13370 solid-floor U/perimeter cascade vs what the surveys lodge.
• 16 Bingley (+5.7): community-heating residual — water heating is "Hot water only community scheme - boilers" (WHC 950 path?), community standing charge/tariff detail, or DLF "Unknown" distribution type.
• 58 Hackle (+7.8): undug — full recipe pass needed.
• Brightholme (+7.3): RIR dwelling; the accredited 43 reflects a QA correction ("no access to loft — hatch screwed shut") — possibly RIR insulation should NOT thread from the roof-space block when access was impossible, or slopes should default worse. Check _pashub_room_in_roof.
• Mechanical ventilation and PCDB efficiency are VERIFIED CORRECT — don't re-dig.

Also outstanding

• Open the review PR for the branch (metrics table = #1590's last comment; ~26 commits, all green: 206 harness + ~294 unit tests).
• Portfolio 838 modelling: re-extract 4 properties in pashub (754772/9 Philips Park Court, 754780/12 Seymour Rd S, 754844/16 Bingley — the community fix now makes it extractable, 754816/130 Stanton St has no site note at all); then RUN_DRY=0 python scripts/run_pashub_modelling.py (env-parametrised: RUN_PORTFOLIO/RUN_SCENARIO/RUN_PIDS/RUN_DRY).
• effective_sap_score == the pashub rating for unchanged lodged dwellings (rebaseliner pass-through) — NEVER validate the calculator against it.
• hubspot_deal_data still holds the stale pre_sap for the 2 corrected deals — re-running scripts/build_pashub_accuracy_fixtures.py will REVERT the manifest corrections unless the DB rows are fixed first.