Commit graph

2603 commits

Author SHA1 Message Date
KhalimCK
206ac197ee
Merge pull request #1544 from Hestia-Homes/feat/calico-stock-condition-ingestion
Calico roof-covering stock-condition ingestion (ADR-0064)
2026-07-24 14:31:54 +01:00
KhalimCK
49a36581f0
Merge pull request #1546 from Hestia-Homes/feat/ara-scenario-export
ARA Scenario Export: branded sheet-per-scenario XLSX export (ADR-0065)
2026-07-23 09:34:51 +01:00
Daniel Roth
ae1da26558 Move the PasHub fetcher into the DDD layer structure 🟪
The service was the last one living wholly under backend/. It now follows
the same layering as abri and the other newer services:

  domain/pashub_fetcher/          core file classification, subfolders
  infrastructure/pashub_fetcher/  PasHub client, token getter, wire DTOs
  orchestration/                  PashubFetcherOrchestrator (was PashubService)
  applications/pashub_fetcher/    lambda handler, trigger request, dev tooling

core_files.py is split along the layer boundary: the domain module keeps the
filename/evidence-category classification rules and no longer imports
infrastructure.postgres, while the CoreFiles -> FileTypeEnum translation moves
to infrastructure/pashub_fetcher/core_file_types.py.

Tests move into the tests/ tree by layer. Note this puts them in the only
suite CI currently runs (unit_tests.yml is disabled), so these 73 tests now
execute on PRs for the first time; they were previously reachable only via
the legacy pytest.ini testpaths.

sharepoint_renamer's image now copies just domain/pashub_fetcher/ rather than
the whole service, since SharepointSubfolders is all it needed.

Behaviour is unchanged. tests/ goes 9927 -> 10000 passed (+73, exactly the
tests that moved in); the legacy suite keeps its same 17 pre-existing failures
and 11 errors. tests/test_lambda_packaging.py confirms both changed
Dockerfiles still copy their handler's full import closure.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 13:35:54 +00:00
Daniel Roth
8381972286 Type-annotate the empty evidence payload for strict pyright 🟪
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 13:01:12 +00:00
Daniel Roth
0b8eb628a1 Surface an expired PasHub token when listing evidence 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 12:59:23 +00:00
Daniel Roth
7b5a88e7ac Treat a missing totalItemCount as a complete evidence list 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 12:58:56 +00:00
Daniel Roth
cc3fa54688 Return the full evidence list when the response is complete 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 12:58:16 +00:00
Daniel Roth
c95d3b20dc Fail loudly when PasHub truncates the evidence list 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 12:57:25 +00:00
Daniel Roth
d8f673a31f Fail loudly when PasHub truncates the evidence list 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 12:56:55 +00:00
Daniel Roth
f54b420ae1 Request evidence list with explicit paging parameters 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 12:55:53 +00:00
Daniel Roth
0378166263 Request evidence list with explicit paging parameters 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 12:54:49 +00:00
Khalim Conn-Kowlessar
72f3e3a72d Add default-plan export selection + plan_name column (ADR-0065)
A second export selection alongside the per-Scenario one, chosen by
`plan_selection` on the export request:

- "scenario" (default, unchanged): one sheet per Scenario, freshest Plan per
  (property, scenario); scenario_ids required.
- "default" (new): a single "Default Plans" sheet of each home's is_default
  Plan (one-per-property across all Scenarios, ADR-0012/0017) — the portfolio's
  current state, one row per home; scenario_ids ignored.

Repository gains `default_rows_for(portfolio_id, property_ids)`: same read-model
and Effective-EPC join as `rows_for`, with the plan-selection WHERE swapped from
`scenario_id = :scenario_id` to `is_default = TRUE`. Orchestrator branches on
plan_selection; the router validates it and only requires scenario_ids for the
scenario selection. A recipe without plan_selection defaults to "scenario", so
the change is backward-compatible.

Both selections now surface the chosen Plan's `name` as a `plan_name` column.

Tests: repository default_rows_for (scenario-independent selection, plan_name,
non-default excluded); orchestrator default path (one "Default Plans" sheet;
empty selection is a recorded failure). 62 passed, pyright --strict clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 09:21:38 +00:00
KhalimCK
08dc81332c
Merge pull request #1650 from Hestia-Homes/fix/pashub-sitenotes-boiler-thermostat-gable
PasHub Wythenshawe: fix 6 from_site_notes defects + re-pin harness to DB oracle (#1649)
2026-07-20 12:04:16 +01:00
Daniel Roth
dc96cf5aba
Merge pull request #1648 from Hestia-Homes/feature/abri-api-resource-field
Sync third_party_surveyor_identifier deal field from HubSpot
2026-07-20 11:02:11 +01:00
Khalim Conn-Kowlessar
26e0584855 Record Elmhurst-validation provenance finding (#1649)
Two outputs of validating the Wythenshawe cohort against accredited Elmhurst:

1. Harness note: the DB-oracle residual tail is not engine defects. The two
   worst under-raters, 6 Barry Road (ours 60.12, oracle 62) and 87 Alderue
   (ours 61.80, oracle 63), were re-keyed into accredited Elmhurst on our exact
   inputs — Elmhurst returned 60 and 62, matching our engine and both below the
   lodged oracle. So our engine computes correct RdSAP; the lodged certs carried
   a lighter input set (dropped the surveyed 17.4 m2 solid-brick alternative
   wall, +1.7 SAP). It's input-provenance drift, not a calculation gap — don't
   chase these as engine bugs or tune the engine to close them. Corrects the
   earlier "spec-literal alt-wall" note.

2. epc-to-elmhurst skill gotcha: give Elmhurst the RAW surveyed room height, not
   the engine's storey height. The mapper already adds +0.25 m to upper floors
   (_UPPER_FLOOR_HEIGHT_ADD_M), so feeding sap_floor_dimensions.room_height_m
   makes Elmhurst double-add the joist void (~1 SAP too low — it under-scored
   6 Barry 60->59 until the first-floor height was corrected 2.76 -> raw 2.51).

Comment/doc only; no behaviour change (harness still 89.2%/0.297, pyright clean).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 09:16:54 +00:00
Khalim Conn-Kowlessar
1424e2110e Re-pin Wythenshawe accuracy harness to the DB oracle (#1649)
`test_pashub_sap_accuracy_wythenshawe` graded against the manifest `pre_sap`
(`hubspot_deal_data.pre_sap`) — surveyor hand-entered and frequently wrong,
which floated a misleading 79% / MAE 0.75. Grade instead against the DB
PasHub-assessment SAP (`epc_property_energy_performance.energy_rating_current`,
`ep.source='lodged'`), matched by UPRN or postcode+house-number.

`scripts/enrich_pashub_wythenshawe_oracle.py` resolves each fixture's oracle SAP
once (DB required, not run in CI) and bakes `oracle_sap` / `oracle_match` into
`manifest.json`, so the harness stays DB-free. 148/168 resolve (39 UPRN + 109
address); the 20 unmatched carry `oracle_sap` null and are excluded from the
aggregate until PasHub sources their SAP.

With the six from_site_notes fixes, the 148 matched score within-0.5 89.2%
(132/148), MAE 0.297 (was, vs pre_sap, 79.2% / 0.754). Ratchets re-baselined
0.75→0.88 floor, 0.85→0.31 ceiling. The residual tail is the integer-oracle
rounding floor (spec-correct dwellings) plus a couple of spec-literal gaps, not
engine defects.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 11:40:51 +00:00
Khalim Conn-Kowlessar
b7f127caec Fix six PasHub from_site_notes extractor/mapper defects (#1649)
All six are `from_site_notes`-only (gov-API RdSAP corpus unmoved at 78.9%),
graded against the DB PasHub-assessment oracle (energy_rating_current,
source='lodged'), not the surveyor-entered pre_sap.

A. Manual-Entry boiler efficiency. Extractor read PCDF-Search columns that a
   manual survey lacks; mapper returned None for all boilers → the generic
   0.80 gas default. Now read `Type of boiler:` / `Heating System (Boiler):`
   / `Is there an open flue?` and resolve the descriptor to its SAP 10.2
   Table 4b code. Moves all 7 Wythenshawe manual boilers to the oracle
   (4a Hollyhedge back boiler 63.6→53.4=53; group MAE 2.42→0.42).

B. Room-in-roof gable type None → strict-raise. A form variant omits the
   gable-type line (genuine field-absence); default None → party `gable_wall`
   (U=0.25), mirroring `_api_type_1_gable_kind(None)`. Present-but-unknown
   labels still strict-raise. Unblocks 8 & 34 Bucklow Drive.

C. Secondary `Closed room heater` unmapped → add Table 4a code 633.
   Unblocks 8 Brinkshaw Avenue.

D. Alternative-wall insulation thickness never extracted → parse
   `Wall insulation thickness:` and pass it through instead of the RdSAP 10
   §5.4 100 mm favourable default.

E. `Number of heated rooms?` label mismatch → the PDF lodges
   `Please enter the number of HEATED rooms:`.

F. HW cylinder thermostat dropped (NEW, systematic). `_map_sap_heating`
   never set `cylinder_thermostat`, so a surveyed thermostat defaulted None
   and the calculator applied the SAP 10.2 Table 2b Note a) ×1.3 penalty.
   Map it ("Y"/"N", gated on a lodged cylinder) as the Elmhurst path does.
   Fixes the two worst DB-oracle under-raters (52 Bucklow 59.4→61.8=62;
   13 Kerne Grove 65.0→67.1=67).

DB-oracle cohort (148 matched): within-0.5 83.8%→87.8%, MAE 0.427→0.303.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 11:30:00 +00:00
Daniel Roth
80113205c8 Sync third_party_surveyor_identifier deal field from HubSpot
Fetch the property from the HubSpot deals API, persist it on
hubspot_deal_data (column added by the frontend repo's drizzle
migration), and include it in the differ's FIELD_MAP so a change
to the field triggers a DB reload.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 12:53:26 +00:00
Daniel Roth
4438cbc4b3 Document LRHA WAVE 3 end-state coverage and residual blocks 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 16:41:32 +00:00
Daniel Roth
7c979d60f6 Rate gas room-heater main systems at Table 4a code 603 🟩
#1631: the label blocks in the main room-heater type table (Heating
System (Other) field), not the secondary table the issue cited — same
PCDB row either way. Wythenshawe 164→165 computed. The fixture scores
49.5 vs pre_sap 31: its own 2025 accredited cert
6890-0597-0722-0196-3943 lodges the identical codes (603/26/2601) at
band E agreeing with us, while pre_sap matches the dwelling's
superseded electric configuration (2017/2023 certs, band F) — accepted
divergence, MAE ceiling re-baselined 0.75→0.85 with the citation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 16:41:32 +00:00
Daniel Roth
be1f69ccd4 Suppress floor heat loss for internal floors over the same dwelling 🟩
#1625: 'Same dwelling below' → the "(another dwelling below)" party-
floor suppression string, mirroring the gov-API code-8 precedent
(RdSAP 10 §3: an internal floor between heated storeys has no floor
heat loss). Wythenshawe 151→161 computed; 7/10 newly-unblocked land
within 0.5 — ratchets re-baselined 0.76/0.72 → 0.75/0.75 for the two
tail entries (−3.3/−5.5).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 16:41:32 +00:00
Daniel Roth
01cacc579a Rate unthermostated room heaters via Table 4e Group 6 code 2601 🟩
#1624: Wythenshawe 149→151 computed. 500767954156 lands within 0.5;
500797465814 is a −8.8 tail entry — MAE ceiling re-baselined 0.67→0.72
per the coverage-growth convention (within-0.5 floor 0.76 holds).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 16:41:32 +00:00
Daniel Roth
3b2e6e348a Score heating-oil dwellings via SAP main-fuel code 28 🟩
#1632: LRHA WAVE 3 45→51 computed. Newly-unblocked oil configs land
outside 0.5 (coverage growth per module convention) — within-0.5 floor
re-baselined 0.46→0.41 at 41.2%/MAE 2.948.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 16:41:32 +00:00
Daniel Roth
d844bc6df2 PasHub SAP accuracy: pin Wythenshawe cohort (149/168 computable) 🟩
Add pashub_accuracy_wythenshawe fixtures + a sibling accuracy module,
built from project_code
[Wythenshawe WH:SHF RAs - WH:SF - 240426 - 494493718748] the same way as
LRHA WAVE 3 (#1620). First run: 149 computed / 19 blocked / 0 errored,
within-0.5 76.5%, MAE 0.665. Ratchets baselined to this cohort; the 19
blocks are separately-ticketed mapper-label gaps (floor 'Same dwelling
below' ×10 dominating), not tuning.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 14:49:17 +00:00
Daniel Roth
3e2509fb7d PasHub scores electric-storage & room-heater dwellings (Table 4e Group 4/6) 🟩
Teaches the pashub `from_site_notes` mapper the electric-storage-heater and
room-heater families, following the existing boiler/heat-network split exactly
(fix at the mapper boundary, ADR-0015 — never mis-rate, never strict-raise deep
in the calculator). Adds:

- Table 4e Group 4 storage charge-control codes (2401/2402/2404) and Group 6
  room-heater control codes (2602/2605), each gated on its own system group so a
  control label never takes a wrong-group 2xxx code.
- The Table 4a system code from the surveyed heater type lodged in the "Heating
  System (Other)" field (`community_heat_source`): Fan storage 404, slimline
  402, PCDF-Search product ⇒ HHR 409 (RdSAP 10 p.80), room heaters 691/694.
- A 2404⇔409 coherence guard: high-heat-retention controls on a non-HHR storage
  type is a surveyor contradiction and fails loud rather than rating a mismatch.

All codes taken from the SAP 10.2 spec (14-03-2025 PDF) and already present in
the calculator's inventories — no calculator change.

LRHA WAVE 3 (lincs rural) accuracy: 8 computed / 96 blocked → 45 computed / 59
blocked / 0 errored (within-0.5 46.7%, MAE 2.989); ratchets re-baselined to the
new coverage. Guinness cohort (208) + gov-API corpus unchanged; pyright --strict
adds zero new errors.

Closes #1619

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 13:30:07 +00:00
Daniel Roth
d557e1b0b4 make pashub accuracy test more generic and add files for lincs rural 2026-07-16 11:22:39 +00:00
Khalim Conn-Kowlessar
6c4f73cc91 PasHub SAP accuracy: refresh 5 Paxton fixture to the current V5 (MEV) re-survey 🟩
5 Paxton Place (deal 507639151845, uprn 10070864419) had a STALE fixture.
The same PAS Hub survey (report B74AA5A4-…, inspected 29 Jun 2026) has two
accredited revisions in S3: V2 (uploaded 6 Jul, ventilation "Natural",
SAP 80) and V5 (uploaded 14–15 Jul, ventilation "Mechanical Extract -
Centralised", SAP 76 — the current version). Our committed fixture was the
superseded V2; the portfolio-838 oracle tracks the latest V5 (76). Our
engine faithfully reproduces BOTH (V2 → 80.4, V5 → 75.6) — the +4.38 miss
vs the DB oracle was a fixture-version mismatch, NOT an engine over-rate
(a parallel over-credit sweep found no spec-correct from_site_notes lever;
every channel is faithful).

Refreshed the fixture to the V5 PDF (image-stripped, text layer preserved,
6.7MB→0.27MB, per scripts/build_pashub_accuracy_fixtures.py) and set the
manifest to C76. 5 Paxton flips from the cohort's largest miss to a
within-0.5 hit: DB-oracle gauge 86.9%→87.4% within-0.5, MAE 0.348→0.327.
Pinned via test_paxton_fixture_is_v5_resurvey_sap_76 (asserts a literal 76
against the V5 fixture) — a canary, because hubspot_deal_data.pre_sap is
still stale at C80 and a fixture-builder rerun would revert the manifest.

Confirmed against pashub: the current survey is SAP 76 (Khalim).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 00:52:33 +00:00
Khalim Conn-Kowlessar
d3f36d78ac PasHub SAP accuracy: pin 13 Walsden to its re-survey SAP 64, not the colliding 68 🟩
13 Walsden (deal 499630239947, uprn 77179782) has TWO distinct gov-EPC
assessments: the PAS Hub site-note re-survey we parse (26 May 2026,
accredited SAP 64, backed by rd_sap_site_note files) and a SEPARATE
later lodged cert (24 Jun 2026, SAP 68, registered, no site-note). The
portfolio-838 baseline latched onto the 24-Jun SAP-68 cert; our engine
models the survey it was given and reproduces its own accredited 64
(64.19). 6 of 7 Walsden Street homes have pbp oracle == PAS Hub survey
SAP exactly — #13 is the lone collision.

A durable regression pin (asserts our SAP within 0.5 of a literal 64,
not the manifest value the fixture-builder can revert to a colliding DB
row) documents the provenance so nobody "corrects" the target to 68.
13 Walsden is a known re-survey-diverges outlier, not an engine
under-rate; confirmed against pashub (Khalim, 2026-07-15e).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 00:52:33 +00:00
Khalim Conn-Kowlessar
4c0ccad882 PasHub SAP accuracy: semi-exposed-unheated floor → Table 20, not §5.14 🟩
`_PASHUB_FLOOR_EXPOSURE` mapped the surveyed "Semi Exposed (unheated)"
floor label to `is_above_partially_heated_space` (RdSAP §5.14, constant
U=0.7). A floor semi-exposed to an enclosed UNHEATED space belongs in
§5.13 Table 20 `u_exposed_floor` (band-B as-built U=1.20). §5.14's U=0.7
is reserved for a floor above a *partially* heated space. #1602
transposed the two branches; the too-low U under-counted floor loss on
the 9 label-carrying cohort fixtures → a shared over-credit.

Triangulated against all three siblings + the calculator docstring: the
gov-API code-2 "To unheated space" (mapper.py:4797) and Elmhurst
`_is_floor_exposed_to_unheated_space` (mapper.py:5970) both route to
Table 20; rdsap_uvalues.py:1416 documents Table 20 as collapsing exposed
and semi-exposed-to-unheated into one lookup. One-line data flip,
from_site_notes-only (gov-API RdSAP corpus untouched, still 78.9%).

Adjacent Ronald St mid-terrace pair (the clustered signal): 10 Ronald
60.35→59.02 (oracle 59), 12 Ronald 60.79→59.36 (oracle 59). Cohort vs
DB oracle (portfolio 838 effective_sap) 83.3%→86.9% within-0.5, MAE
0.377→0.348, no built form regresses, all 7 verified dwellings unchanged
(none carry the label). pashub harness 79.5%→82.4%, MAE 0.389→0.367;
ratchets tightened 0.78→0.82 / 0.40→0.37. pyright zero-new.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 00:52:33 +00:00
KhalimCK
04b7c26f67
PasHub SAP accuracy: infiltration inputs + immersion DHW + lights/lobby/heat-network (#1615)
* Electric-immersion DHW: bill at 100%, not the space boiler's efficiency 🟩

A separate electric immersion heater (SAP water-heating code 903) is 100%
efficient (SAP 10.2 Table 4a) and the space-heating boiler provides no water
heating, so Appendix D2.1 Eq D1 (the boiler seasonal-efficiency cascade) must
not apply to it. But `_water_heating_main` resolves DHW to the SPACE main — a
gas/oil boiler keeps its PCDB record — so three water-efficiency branches in
cert_to_inputs billed the immersion-heated cylinder at the boiler's ~87% summer
efficiency (a mongrel: electric fuel price x gas-boiler efficiency):

  1. the scalar `water_eff = water_pcdb_main.summer_efficiency_pct / 100`,
  2. the SAP §9.4.11 / Table 4c(2) -5pp no-interlock adjustment, and
  3. the Eq D1 (winter, summer) seasonal pair from `pcdb_main`.

Gate all three on `not dhw_is_electric_immersion`; the correct immersion path
(`_water_efficiency_with_category_inherit` -> 1.0, Eq D1 off) then applies,
mirroring the Table 3 zero-primary-loss gate already present for WHC 903.

General bug — fires for any WHC-903 dwelling whose space main is a PCDB gas/oil
boiler. It surfaced via the PasHub campaign's #1600 no-water-heating default
(WHC 999 -> 903) on a gas-combi dwelling: 58 Hackle St M11 4WU, SAP 55.30 ->
57.67 (pre_sap 58, verified 59).

Guardrails: the gov-API RdSAP corpus IMPROVES 78.8% -> 78.9% within-0.5, MAE
0.625 -> 0.622 (a handful of corpus certs with a boiler space main + electric
immersion move closer to accredited) — MAE ceiling ratcheted 0.626 -> 0.625.
Regression pinned in test_cert_to_inputs (RED before / GREEN after). pyright
0-new (cert_to_inputs baseline 30).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* PasHub SAP accuracy: thread low-energy lights + draught lobby (from_site_notes) 🟩

Two systematic silent-drops in the PAS Hub `from_site_notes` path, each a field
the gov-API/Elmhurst mappers thread but site-notes dropped, letting a favourable
default reach the SAP-10.2 calculator. Validated against the correct pashub
oracle (property_baseline_performance.effective_sap_score, portfolio 838):
cohort within-0.5 55.1% → 62.6%, MAE 0.599 → 0.521.

- Low-energy lights: when "exact LED/CFL known = No", PAS Hub lodges an aggregate
  "Number of fixed low energy lights?" count. Previously dropped (no dataclass
  field / extractor key / mapper thread) → calculator saw 0 low-energy bulbs and
  applied the pessimistic L5b/L8c no-data default, under-rating SAP. Now threaded
  into low_energy_fixed_lighting_bulbs_count, mirroring from_elmhurst_site_notes.
  ~6% of the cohort; resolves 58 Hackle (−0.43 vs oracle 59 with #1615).
- Draught lobby: _map_sap_ventilation set only the legacy `draught_lobby` field,
  never the canonical `has_draught_lobby` §2 (13) gate the cascade reads, so the
  surveyed lobby was ignored and infiltration over-stated. Now mirrors Elmhurst.

Two other audited levers were REJECTED against the pashub oracle (they matched
the gov-cert/RdSAP convention but pashub does not apply them): percent_draughtproofed
(cohort 30.8%) and the §A.2.2 assumed secondary heater (19.2%). A heat-network
control-code fix (2306→2303 for 16 Bingley) is HELD pending spec/Elmhurst
adjudication — it contradicts ADR-0053; see the NOTE on _PASHUB_HEAT_NETWORK_CONTROL_TO_SAP10.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* PasHub heat-network control: room-thermostat-only maps to 2308, not 2306 🟩

The Table 4e Group 3 label "charging linked to use of community heating, room
thermostat only" was mapped to 2306 — the linked-to-use *with-TRVs* code
(control type 3, space charging 1.00). The label explicitly excludes TRVs, so
it is control type 2 / space 1.05 = code 2308/2309 (linked-to-use → DHW 1.00).
2306 asserted TRVs the survey doesn't have and over-rated the cohort's sole
heat-network dwelling (16 Bingley Close 56.89 → 54.01 vs pashub oracle 52).

Verified against SAP 10.2 Table 4e as encoded in cert_to_inputs.py
(_CONTROL_TYPE_BY_CODE 2308→2, space-charging 2308→1.05, DHW 2308→1.00) and the
RdSAP control-label vocabulary in MainheatControlAttributes.py. 2303 (the earlier
audit's guess) is rejected: it is a flat-rate code (DHW 1.05) whose oracle match
was a coincidence of two offsetting spec violations. The residual +2.0 to oracle
52 is the documented SAP-10.2-engine-vs-lodged offset, not a fuel/flags gap
(those were threaded by the #1590 follow-up) — supersedes the ADR-0053 /
HANDOVER_838 "community fuel/flags" attribution.

16 Bingley is the only heat-network fixture in portfolio 838, so this moves one
fixture strictly toward its oracle. Harness ratcheted: within-0.5 0.51→0.58,
MAE 0.625→0.521 (observed 58.5% / 0.520 with all three 2026-07-15 levers).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* PasHub SAP accuracy: restore 3 dropped §2 infiltration inputs (from_site_notes)

`from_site_notes` uniquely dropped three ventilation inputs that the gov-API
and Elmhurst sibling mappers all set, causing a systematic cohort-wide SAP
under-rate vs the company's own accredited SAP-10.2 certs
(property_baseline_performance.effective_sap_score, portfolio 838):

1. percent_draughtproofed — never set, so §2(15) window infiltration was
   pinned at its 0.25-ACH worst case on every dwelling. Reconstructed as the
   area-weighted % of draught-proofed windows (mirrors Elmhurst
   `draught_proofing_percent`). Dominant term.
2. Upper-storey +0.25 m joist void — omitted; now added, matching
   `_UPPER_FLOOR_HEIGHT_ADD_M` on the gov-API/Elmhurst paths. This
   RE-ADJUDICATES #1601 ("keep raw"): that call was confounded by the
   then-present draught-proofing drop suppressing every verified dwelling.
3. sheltered_sides — left None → calculator's flat default of 2, which
   over-shelters end/semi/detached (RdSAP §S5 = 1/1/0). Now derived from
   built form.

The three are NON-ADDITIVE — each overshoots alone (which is why all three
were previously rejected individually) — but together they land all 7
verified ground-truth dwellings toward truth (none regress) and every built
form near zero. Cohort 62.1% -> 83.3% within-0.5, MAE 0.507 -> 0.377.

Guardrails: gov-API RdSAP corpus unchanged (78.9%, these are from_site_notes-
only helpers). pashub harness 58.5% -> 79.5% / MAE 0.520 -> 0.389; ratchets
tightened 0.58->0.78 and 0.521->0.40. pyright zero-new (mapper baseline 39).
New focused tests in TestFromSiteNotesInfiltrationFixes cover all three
fields incl. area-weighting and the built-form shelter map; goldens updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 00:30:55 +01:00
Khalim Conn-Kowlessar
6fd0393a4e Tighten pashub accuracy ratchets to 0.51 / 0.625 🟪
#1602 (floor exposure flags) + #1603 (alternative walls + extension floor
construction) move the Guinness 205 cohort within-0.5 50.2% -> 51.2% and MAE
0.653 -> 0.624. Lock the gain: floor 0.50 -> 0.51, ceiling 0.653 -> 0.625.
#1601 note updated from HELD to RESOLVED (keep-raw).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 15:46:06 +00:00
Khalim Conn-Kowlessar
5324ae0e57 PasHub floor exposure + alternative walls + extension floor 🟩
Resolve #1601 and land #1602/#1603 on the merged-#1609 tree, adjudicated
against Khalim's verified ground truth (not pre_sap).

#1601 (upper-floor +0.25 m joist void) — RESOLVED as keep-raw. The
gov-API/Elmhurst mappers add 0.25 m to upper-storey heights because their
lodged height is a clear internal ceiling. Against verified truth RAW beats
+0.25 on all 7 verified dwellings (16 Stillwater raw -0.35 within-band vs
+0.25 -0.75; every verified property is already under truth and +0.25 only
lowers SAP). pashub surveys internal dimensions (202/205 "Measurements
Location: Internal") and its upper-ground height gap is modal 0.00, so the
joist void is not differentially present and a blanket +0.25 is unsafe.
_map_floor_dimensions keeps the raw surveyed height with a RESOLVED note.

#1602 — semi-exposed / exposed floor exposure. The surveyed "Floor type"
"Semi Exposed (unheated)" now sets is_above_partially_heated_space (RdSAP
§5.14 U=0.7) and "Exposed Floor" sets is_exposed_floor (§5.13 Table 20), on
the lowest storey (floor==0) only, mirroring the gov-API/Elmhurst siblings.
New strict-raise _pashub_floor_exposure (ADR-0015).

#1603a — alternative walls. New AlternativeWall survey dataclass +
extract_alternative_walls (the "Alternative Wall"->"Windows" section) +
_map_pashub_alternative_walls -> sap_alternative_wall_1/2. The cascade
deducts each alt-wall area from the part's main opaque wall (no double
count), so a mixed-facade dwelling is billed at each sub-area's own
construction. The alt-wall block lodges "As Built" (capital B) — aliased to
the existing default insulation code.

#1603b — extension floor construction. ExtensionConstruction gains a floor
block (parsed like the main building) and _map_extension_building_part now
threads floor_type/construction/insulation/u_value_known, which the main
path already did and the extension silently dropped.

Cohort: within-0.5 50.2% -> 51.2%, MAE 0.652 -> 0.624, 0 strict-raise
errors across all 205. Verified-truth safe: none of the 7 verified
dwellings have alt walls, and #1602 moves the one it touches (2 Philips
exposed ext floor) by -0.00. pyright 0-new (mapper baseline 39). Goldens
test_full_building_construction + end_to_end EXTENSION_1 updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 15:46:06 +00:00
Khalim Conn-Kowlessar
644bc300a6 Merge remote-tracking branch 'origin/main' into feat/1596-pashub-two-clusters
# Conflicts:
#	backend/documents_parser/tests/test_pashub_sap_accuracy.py
2026-07-15 14:02:15 +00:00
Khalim Conn-Kowlessar
30b1728337 PasHub extraction: thread showers/baths, normalize floor label, code no-water-heating 🟩
Three silent-drop extraction fixes the gov-API/Elmhurst mappers already apply
but `from_site_notes` uniquely omitted — each let a favorable default reach the
SAP-10.2 calculator and over-rate the dwelling. Same calculator, so the whole
gap was extraction fidelity (gov-API corpus 78.6% within-0.5 vs pashub 23.9%).

#1598 — electric-shower + bath counts: `_map_sap_heating` built `shower_outlets`
  but never set `electric_shower_count`/`mixer_shower_count`/`number_baths`, so
  the cascade defaulted to 1 gas mixer / 0 electric, billing electric-shower hot
  water off the cheap main system. 100/205 fixtures lodge "Electric Shower".
#1599 — floor label: pashub lodges "Ground Floor" (capital F); the calculator's
  RdSAP §5(12) suspended-timber floor-infiltration gate is case-sensitive on
  "Ground floor", so the 0.2-ACH term was silently dropped. Normalized at the
  mapper boundary. 98/205 suspended-timber.
#1600 — water-heating "None": the RdSAP 10 §10.7 "no water heating system"
  signal was discarded and HW inherited the gas combi (58 Hackle 67.7 vs
  verified 59). Coded to WHC 999 so the existing electric-immersion + Table
  28/29 default fires (also folds in #1594: blank cylinder size is no cylinder,
  not a phantom one).

Guinness GMCA 205 cohort: within-0.5 23.9% -> 50.2% (103/205), MAE 1.410 ->
0.650. Ratchets tightened to 0.50 / 0.65. Verified vs Khalim's ground truth
(16 Stillwater 78.2 -> 75.7 vs 76). #1601 (upper-floor +0.25 m) held — it
empirically over-corrects this cohort (internal-vs-external dimension question,
under investigation). pyright: 0 new errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 13:52:13 +00:00
Daniel Roth
53ba1f1694 Re-baseline the pashub MAE ceiling to 1.415 for the water-only DHW fix 🟪
Second sanctioned exception to "ratchets never loosen" — needs @KhalimCK's
call, since the prior note expected to re-tighten here. Exactly one cohort
property is WHC 950 (16 Bingley Close, +5.946 -> +6.818 = the whole +0.0043);
within-0.5 unchanged at 23.9%. Its pre_sap is verified correct but its ~+6
residual is a known extraction bug, and this hybrid gauge cannot adjudicate a
calculator fix (#1592 Problem 2). The gov-API corpus, which gauges the
calculator alone, improves.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 10:57:16 +00:00
Khalim Conn-Kowlessar
bb7f05693c Update golden for the secondary-heating int-code + retighten pashub MAE post-#1592-revert
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 09:15:46 +00:00
Khalim Conn-Kowlessar
21674d5ed0 Re-baseline the pashub gauge to 23% within-0.5 / MAE 1.41 for the orientation fix
Khalim-authorised MAE loosening: coding window orientation exposes the known
SAP-10.2-vs-lodged offset the zeroed solar gains were masking; within-0.5 is
the north-star metric and jumps 20.0% -> 23.9%.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 08:51:52 +00:00
Khalim Conn-Kowlessar
b208234e62 Ratchet the pashub accuracy gauge to MAE 1.371 after the common-wall fix
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 08:40:39 +00:00
Khalim Conn-Kowlessar
7c2cb6b88a Ratchet the pashub accuracy gauge to MAE 1.37 after the common-wall fix
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 08:35:56 +00:00
Khalim Conn-Kowlessar
8a753822c8 Capture the Simplified RIR 'Roof room type' label from the survey 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 08:27:01 +00:00
Khalim Conn-Kowlessar
600262819c Capture the Simplified RIR 'Roof room type' label from the survey 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 08:25:11 +00:00
Khalim Conn-Kowlessar
948704d7b0 Record the secondary-heating-type coding fix in the accuracy ratchet note
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 00:28:40 +00:00
Khalim Conn-Kowlessar
390a818587 Ratchet the pashub accuracy gauge to 19% within-0.5 / MAE 1.39
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 00:05:13 +00:00
Khalim Conn-Kowlessar
483b69ff41 Capture the surveyed Wall Dry-Lined answer per building part 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 23:20:36 +00:00
Khalim Conn-Kowlessar
0b47798403 Capture the surveyed Wall Dry-Lined answer per building part 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 23:20:09 +00:00
Khalim Conn-Kowlessar
8e58f7e871 Capture the per-surface RIR insulation-known answer from the survey 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 23:12:08 +00:00
Khalim Conn-Kowlessar
d743f6fb3b Capture the per-surface RIR insulation-known answer from the survey 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 23:11:09 +00:00
Khalim Conn-Kowlessar
e0b189aa6d Correct two pashub-verified pre_sap values in the accuracy manifest
Manually verified in pashub (Khalim, 2026-07-14): deal 499584755922
(9 Council Houses, CW5 8AP) is F33, not the stale D67 — our computed 34.0
was right within a point; deal 499617935574 (Brightholme, 14 North Road)
is the accredited E43, not the preliminary 53. 58 Hackle Street (D58) and
16 Bingley Close (E52) were confirmed correct, so our +7.8/+5.7 deviations
there are extraction bugs to hunt next. Harness MAE 1.684 -> 1.551; ceiling
ratcheted to 1.56.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 22:42:03 +00:00
Khalim Conn-Kowlessar
f45651dbc0 Lodge the heat-network SAP code and Table 12 community fuel for PasHub community heating 🟩
The extractor now captures "Heating System (Other):" and the mapper resolves
it to the Table 4a code (301, community boilers) + Table 12 community fuel
(51 via the shared _resolve_community_heating_fuel_code), so
is_heat_network_main routes the DLF / standing-charge / 80%-efficiency
branch instead of pricing the dwelling as an ordinary mains-gas boiler
(fixture 507644414148: +14.3 over-rate closes by ~8.6). Harness MAE
1.726 -> 1.684; ceiling ratcheted to 1.70.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 22:34:50 +00:00
Khalim Conn-Kowlessar
4d60d79cc9 Extract the PasHub community heat-source label 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 22:33:08 +00:00