# PasHub portfolio 838 — problem properties & validation (2026-07-14)

Cohort: **The Guinness Partnership GMCA** (205 PasHub site-notes properties),
portfolio **838**, scenario **1297**. Modelled locally through the real
`applications/modelling_e2e` lambda handler using the **stored** PasHub site
notes (`refetch_epc=False`) — see `scripts/run_pashub_modelling.py`.

## Headline: our SAP calculator diverges from pashub's rating by ~MAE 2.7 — real work remains

**Do not use `property_baseline_performance.effective_sap_score` to validate the
calculator.** For a SAP-10.2 lodged EPC whose physical state is unchanged, the
rebaseliner passes **Lodged Performance through as Effective**
(`domain/property_baseline/rebaseliner.py`), and Lodged Performance's `sap_score`
is `energy_rating_current` — the fetched pashub rating
(`domain/property_baseline/performance.py:63`). So `effective_sap_score` **is**
the pashub rating; comparing it to `energy_rating_current` compares the rating to
itself (circular — an earlier "99.5% match" was this mistake).

The real accuracy of our **calculator** is the `test_pashub_sap_accuracy`
harness: **~12.2% within-0.5, MAE ~2.7** vs `pre_sap` (which ≈ the pashub
rating, MAE 0.15 — same ground truth). Worked example: property **754881** — our
extraction is **byte-identical** to the stored ingestion (fuel 26, PCDB index
18119, wall 4/2, roof 200mm, party-wall 4, floor solid), yet our calculator
scores **71.9** where pashub lodged **85** — a genuine ~13-point gap to
root-cause. These are the extraction/calculator bugs to hunt.

**Image stripping does NOT lose data:** the fixture (stripped) and DB (original
PDF) parses of 754881 are identical field-for-field — so the harness fixtures are
faithful and the ~2.7 MAE is a true calculator gap, not a fixture artifact.
`parse_site_notes_pdf` never extracts the SAP rating from the PDF at all
(`energy_rating_current` is populated by Dan's separate pashub-API fetch,
`pashub_service.py:45` ← `preSapRating`), which is why the rating is `None` on a
raw parse but present in the DB — unrelated to stripping.

## Properties that did not run (need PasHub re-extraction — data, not code)

These 3 have **no fresh int-coded stored site note** (only pre-fix rows with a
string fuel), so modelling strict-raised and skipped them. Re-trigger PasHub
extraction so a current-mapper site note is stored, then they model like the
other 201.

| property_id | UPRN | address | stored site-note fuel | fix |
|---|---|---|---|---|
| 754772 | 77168847 | 9 Philips Park Court, Willdale Close, M11 4DH | `"Mains gas"` (string) | re-extract |
| 754780 | 77180607 | 12 Seymour Road South, Clayton, M11 4PG | `"Mains gas"` (string) | re-extract |
| 754844 | 77155031 | 16 Bingley Close, Beswick, M11 3RF | `""` (blank) | re-extract; if the survey genuinely lodges no main fuel, that is a separate blank/residual-fuel mapper gap |

## Properties that ran but not from their own PasHub survey

| property_id | UPRN | address | note |
|---|---|---|---|
| 754816 | 77180053 | 130 Stanton Street, Clayton, M11 4PX | **no PasHub site note exists at all** — modelled via the prediction path. Needs extraction so it models from its own survey. |
| 754778 | 77181049 | (M11) | winner lodged EPC is **not** a PasHub site note (gov EPC / other); modelled SAP 60 vs stored rating 57 (**d=+3**, the cohort's only >0.5 divergence). Check why its PasHub site note isn't the winner. |

## Accuracy outliers surfaced by closing the extractor gaps

Closing the last 4 `test_pashub_sap_accuracy` xfails (see the PR) makes them
**compute**; two then show a large gap **vs `pre_sap`** (not vs the stored
rating — verify `pre_sap` first, do not tune to it):

| fixture (deal) | ours | pre_sap | note |
|---|---|---|---|
| 499584755922 | 34.0 | 67 | main fuel extracted as **Bulk LPG (27)** + house-coal secondary → low SAP. Either a bad `pre_sap` or a main-fuel extraction issue; confirm the survey's main fuel. |
| 507644414148 | 70.2 | 52 | **community heating**. The PasHub path maps only the Table 4e Group 3 control code (2306); it does **not** yet set the full heat-network fuel/flags (main_fuel is 26, not a Table 12 community code). Deeper community-heating mapping gap. |

## Root-caused extractor bugs (issue #1590)

Five worst-divergence properties were deep-dived (extracted inputs diffed
against the PDF text; gaps attributed via patch-and-rerun). **7 distinct
extractor bugs** — PV arrays never extracted (−13.5), roof "Insulation At:
None" treated as unknown not zero (−7.6/−7.1, systematic), `pv_connection`
string passthrough (−6.9), ventilation kind never mapped (−4.7), room-in-roof
never built (−2.1), cylinder "No Access" passthrough (−0.6), system-build/
basement code-6 collision (latent) — are itemised with fixes and fixture deals
in **https://github.com/Hestia-Homes/Model/issues/1590**. Also there: the
ground-truth caveat that `pre_sap` is pashub's *preliminary* figure — the
accredited lodgement can differ downstream of the site notes (754917: 53 → 43).

## How to reproduce

```
python scripts/run_pashub_modelling.py           # dry-run, all 838 batches
RUN_DRY=0 python scripts/run_pashub_modelling.py  # real writes
RUN_PIDS="754772" RUN_DRY=0 python scripts/run_pashub_modelling.py  # one property
```
Validation sweep (ours vs stored rating) is in the PR description / this handover.