Commit graph

2577 commits

Author SHA1 Message Date
Daniel Roth
80113205c8 Sync third_party_surveyor_identifier deal field from HubSpot
Fetch the property from the HubSpot deals API, persist it on
hubspot_deal_data (column added by the frontend repo's drizzle
migration), and include it in the differ's FIELD_MAP so a change
to the field triggers a DB reload.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 12:53:26 +00:00
Daniel Roth
d844bc6df2 PasHub SAP accuracy: pin Wythenshawe cohort (149/168 computable) 🟩
Add pashub_accuracy_wythenshawe fixtures + a sibling accuracy module,
built from project_code
[Wythenshawe WH:SHF RAs - WH:SF - 240426 - 494493718748] the same way as
LRHA WAVE 3 (#1620). First run: 149 computed / 19 blocked / 0 errored,
within-0.5 76.5%, MAE 0.665. Ratchets baselined to this cohort; the 19
blocks are separately-ticketed mapper-label gaps (floor 'Same dwelling
below' ×10 dominating), not tuning.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 14:49:17 +00:00
Daniel Roth
3e2509fb7d PasHub scores electric-storage & room-heater dwellings (Table 4e Group 4/6) 🟩
Teaches the pashub `from_site_notes` mapper the electric-storage-heater and
room-heater families, following the existing boiler/heat-network split exactly
(fix at the mapper boundary, ADR-0015 — never mis-rate, never strict-raise deep
in the calculator). Adds:

- Table 4e Group 4 storage charge-control codes (2401/2402/2404) and Group 6
  room-heater control codes (2602/2605), each gated on its own system group so a
  control label never takes a wrong-group 2xxx code.
- The Table 4a system code from the surveyed heater type lodged in the "Heating
  System (Other)" field (`community_heat_source`): Fan storage 404, slimline
  402, PCDF-Search product ⇒ HHR 409 (RdSAP 10 p.80), room heaters 691/694.
- A 2404⇔409 coherence guard: high-heat-retention controls on a non-HHR storage
  type is a surveyor contradiction and fails loud rather than rating a mismatch.

All codes taken from the SAP 10.2 spec (14-03-2025 PDF) and already present in
the calculator's inventories — no calculator change.

LRHA WAVE 3 (lincs rural) accuracy: 8 computed / 96 blocked → 45 computed / 59
blocked / 0 errored (within-0.5 46.7%, MAE 2.989); ratchets re-baselined to the
new coverage. Guinness cohort (208) + gov-API corpus unchanged; pyright --strict
adds zero new errors.

Closes #1619

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 13:30:07 +00:00
Daniel Roth
d557e1b0b4 make pashub accuracy test more generic and add files for lincs rural 2026-07-16 11:22:39 +00:00
Khalim Conn-Kowlessar
6c4f73cc91 PasHub SAP accuracy: refresh 5 Paxton fixture to the current V5 (MEV) re-survey 🟩
5 Paxton Place (deal 507639151845, uprn 10070864419) had a STALE fixture.
The same PAS Hub survey (report B74AA5A4-…, inspected 29 Jun 2026) has two
accredited revisions in S3: V2 (uploaded 6 Jul, ventilation "Natural",
SAP 80) and V5 (uploaded 14–15 Jul, ventilation "Mechanical Extract -
Centralised", SAP 76 — the current version). Our committed fixture was the
superseded V2; the portfolio-838 oracle tracks the latest V5 (76). Our
engine faithfully reproduces BOTH (V2 → 80.4, V5 → 75.6) — the +4.38 miss
vs the DB oracle was a fixture-version mismatch, NOT an engine over-rate
(a parallel over-credit sweep found no spec-correct from_site_notes lever;
every channel is faithful).

Refreshed the fixture to the V5 PDF (image-stripped, text layer preserved,
6.7MB→0.27MB, per scripts/build_pashub_accuracy_fixtures.py) and set the
manifest to C76. 5 Paxton flips from the cohort's largest miss to a
within-0.5 hit: DB-oracle gauge 86.9%→87.4% within-0.5, MAE 0.348→0.327.
Pinned via test_paxton_fixture_is_v5_resurvey_sap_76 (asserts a literal 76
against the V5 fixture) — a canary, because hubspot_deal_data.pre_sap is
still stale at C80 and a fixture-builder rerun would revert the manifest.

Confirmed against pashub: the current survey is SAP 76 (Khalim).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 00:52:33 +00:00
Khalim Conn-Kowlessar
d3f36d78ac PasHub SAP accuracy: pin 13 Walsden to its re-survey SAP 64, not the colliding 68 🟩
13 Walsden (deal 499630239947, uprn 77179782) has TWO distinct gov-EPC
assessments: the PAS Hub site-note re-survey we parse (26 May 2026,
accredited SAP 64, backed by rd_sap_site_note files) and a SEPARATE
later lodged cert (24 Jun 2026, SAP 68, registered, no site-note). The
portfolio-838 baseline latched onto the 24-Jun SAP-68 cert; our engine
models the survey it was given and reproduces its own accredited 64
(64.19). 6 of 7 Walsden Street homes have pbp oracle == PAS Hub survey
SAP exactly — #13 is the lone collision.

A durable regression pin (asserts our SAP within 0.5 of a literal 64,
not the manifest value the fixture-builder can revert to a colliding DB
row) documents the provenance so nobody "corrects" the target to 68.
13 Walsden is a known re-survey-diverges outlier, not an engine
under-rate; confirmed against pashub (Khalim, 2026-07-15e).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 00:52:33 +00:00
Khalim Conn-Kowlessar
4c0ccad882 PasHub SAP accuracy: semi-exposed-unheated floor → Table 20, not §5.14 🟩
`_PASHUB_FLOOR_EXPOSURE` mapped the surveyed "Semi Exposed (unheated)"
floor label to `is_above_partially_heated_space` (RdSAP §5.14, constant
U=0.7). A floor semi-exposed to an enclosed UNHEATED space belongs in
§5.13 Table 20 `u_exposed_floor` (band-B as-built U=1.20). §5.14's U=0.7
is reserved for a floor above a *partially* heated space. #1602
transposed the two branches; the too-low U under-counted floor loss on
the 9 label-carrying cohort fixtures → a shared over-credit.

Triangulated against all three siblings + the calculator docstring: the
gov-API code-2 "To unheated space" (mapper.py:4797) and Elmhurst
`_is_floor_exposed_to_unheated_space` (mapper.py:5970) both route to
Table 20; rdsap_uvalues.py:1416 documents Table 20 as collapsing exposed
and semi-exposed-to-unheated into one lookup. One-line data flip,
from_site_notes-only (gov-API RdSAP corpus untouched, still 78.9%).

Adjacent Ronald St mid-terrace pair (the clustered signal): 10 Ronald
60.35→59.02 (oracle 59), 12 Ronald 60.79→59.36 (oracle 59). Cohort vs
DB oracle (portfolio 838 effective_sap) 83.3%→86.9% within-0.5, MAE
0.377→0.348, no built form regresses, all 7 verified dwellings unchanged
(none carry the label). pashub harness 79.5%→82.4%, MAE 0.389→0.367;
ratchets tightened 0.78→0.82 / 0.40→0.37. pyright zero-new.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 00:52:33 +00:00
KhalimCK
04b7c26f67
PasHub SAP accuracy: infiltration inputs + immersion DHW + lights/lobby/heat-network (#1615)
* Electric-immersion DHW: bill at 100%, not the space boiler's efficiency 🟩

A separate electric immersion heater (SAP water-heating code 903) is 100%
efficient (SAP 10.2 Table 4a) and the space-heating boiler provides no water
heating, so Appendix D2.1 Eq D1 (the boiler seasonal-efficiency cascade) must
not apply to it. But `_water_heating_main` resolves DHW to the SPACE main — a
gas/oil boiler keeps its PCDB record — so three water-efficiency branches in
cert_to_inputs billed the immersion-heated cylinder at the boiler's ~87% summer
efficiency (a mongrel: electric fuel price x gas-boiler efficiency):

  1. the scalar `water_eff = water_pcdb_main.summer_efficiency_pct / 100`,
  2. the SAP §9.4.11 / Table 4c(2) -5pp no-interlock adjustment, and
  3. the Eq D1 (winter, summer) seasonal pair from `pcdb_main`.

Gate all three on `not dhw_is_electric_immersion`; the correct immersion path
(`_water_efficiency_with_category_inherit` -> 1.0, Eq D1 off) then applies,
mirroring the Table 3 zero-primary-loss gate already present for WHC 903.

General bug — fires for any WHC-903 dwelling whose space main is a PCDB gas/oil
boiler. It surfaced via the PasHub campaign's #1600 no-water-heating default
(WHC 999 -> 903) on a gas-combi dwelling: 58 Hackle St M11 4WU, SAP 55.30 ->
57.67 (pre_sap 58, verified 59).

Guardrails: the gov-API RdSAP corpus IMPROVES 78.8% -> 78.9% within-0.5, MAE
0.625 -> 0.622 (a handful of corpus certs with a boiler space main + electric
immersion move closer to accredited) — MAE ceiling ratcheted 0.626 -> 0.625.
Regression pinned in test_cert_to_inputs (RED before / GREEN after). pyright
0-new (cert_to_inputs baseline 30).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* PasHub SAP accuracy: thread low-energy lights + draught lobby (from_site_notes) 🟩

Two systematic silent-drops in the PAS Hub `from_site_notes` path, each a field
the gov-API/Elmhurst mappers thread but site-notes dropped, letting a favourable
default reach the SAP-10.2 calculator. Validated against the correct pashub
oracle (property_baseline_performance.effective_sap_score, portfolio 838):
cohort within-0.5 55.1% → 62.6%, MAE 0.599 → 0.521.

- Low-energy lights: when "exact LED/CFL known = No", PAS Hub lodges an aggregate
  "Number of fixed low energy lights?" count. Previously dropped (no dataclass
  field / extractor key / mapper thread) → calculator saw 0 low-energy bulbs and
  applied the pessimistic L5b/L8c no-data default, under-rating SAP. Now threaded
  into low_energy_fixed_lighting_bulbs_count, mirroring from_elmhurst_site_notes.
  ~6% of the cohort; resolves 58 Hackle (−0.43 vs oracle 59 with #1615).
- Draught lobby: _map_sap_ventilation set only the legacy `draught_lobby` field,
  never the canonical `has_draught_lobby` §2 (13) gate the cascade reads, so the
  surveyed lobby was ignored and infiltration over-stated. Now mirrors Elmhurst.

Two other audited levers were REJECTED against the pashub oracle (they matched
the gov-cert/RdSAP convention but pashub does not apply them): percent_draughtproofed
(cohort 30.8%) and the §A.2.2 assumed secondary heater (19.2%). A heat-network
control-code fix (2306→2303 for 16 Bingley) is HELD pending spec/Elmhurst
adjudication — it contradicts ADR-0053; see the NOTE on _PASHUB_HEAT_NETWORK_CONTROL_TO_SAP10.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* PasHub heat-network control: room-thermostat-only maps to 2308, not 2306 🟩

The Table 4e Group 3 label "charging linked to use of community heating, room
thermostat only" was mapped to 2306 — the linked-to-use *with-TRVs* code
(control type 3, space charging 1.00). The label explicitly excludes TRVs, so
it is control type 2 / space 1.05 = code 2308/2309 (linked-to-use → DHW 1.00).
2306 asserted TRVs the survey doesn't have and over-rated the cohort's sole
heat-network dwelling (16 Bingley Close 56.89 → 54.01 vs pashub oracle 52).

Verified against SAP 10.2 Table 4e as encoded in cert_to_inputs.py
(_CONTROL_TYPE_BY_CODE 2308→2, space-charging 2308→1.05, DHW 2308→1.00) and the
RdSAP control-label vocabulary in MainheatControlAttributes.py. 2303 (the earlier
audit's guess) is rejected: it is a flat-rate code (DHW 1.05) whose oracle match
was a coincidence of two offsetting spec violations. The residual +2.0 to oracle
52 is the documented SAP-10.2-engine-vs-lodged offset, not a fuel/flags gap
(those were threaded by the #1590 follow-up) — supersedes the ADR-0053 /
HANDOVER_838 "community fuel/flags" attribution.

16 Bingley is the only heat-network fixture in portfolio 838, so this moves one
fixture strictly toward its oracle. Harness ratcheted: within-0.5 0.51→0.58,
MAE 0.625→0.521 (observed 58.5% / 0.520 with all three 2026-07-15 levers).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* PasHub SAP accuracy: restore 3 dropped §2 infiltration inputs (from_site_notes)

`from_site_notes` uniquely dropped three ventilation inputs that the gov-API
and Elmhurst sibling mappers all set, causing a systematic cohort-wide SAP
under-rate vs the company's own accredited SAP-10.2 certs
(property_baseline_performance.effective_sap_score, portfolio 838):

1. percent_draughtproofed — never set, so §2(15) window infiltration was
   pinned at its 0.25-ACH worst case on every dwelling. Reconstructed as the
   area-weighted % of draught-proofed windows (mirrors Elmhurst
   `draught_proofing_percent`). Dominant term.
2. Upper-storey +0.25 m joist void — omitted; now added, matching
   `_UPPER_FLOOR_HEIGHT_ADD_M` on the gov-API/Elmhurst paths. This
   RE-ADJUDICATES #1601 ("keep raw"): that call was confounded by the
   then-present draught-proofing drop suppressing every verified dwelling.
3. sheltered_sides — left None → calculator's flat default of 2, which
   over-shelters end/semi/detached (RdSAP §S5 = 1/1/0). Now derived from
   built form.

The three are NON-ADDITIVE — each overshoots alone (which is why all three
were previously rejected individually) — but together they land all 7
verified ground-truth dwellings toward truth (none regress) and every built
form near zero. Cohort 62.1% -> 83.3% within-0.5, MAE 0.507 -> 0.377.

Guardrails: gov-API RdSAP corpus unchanged (78.9%, these are from_site_notes-
only helpers). pashub harness 58.5% -> 79.5% / MAE 0.520 -> 0.389; ratchets
tightened 0.58->0.78 and 0.521->0.40. pyright zero-new (mapper baseline 39).
New focused tests in TestFromSiteNotesInfiltrationFixes cover all three
fields incl. area-weighting and the built-form shelter map; goldens updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 00:30:55 +01:00
Khalim Conn-Kowlessar
6fd0393a4e Tighten pashub accuracy ratchets to 0.51 / 0.625 🟪
#1602 (floor exposure flags) + #1603 (alternative walls + extension floor
construction) move the Guinness 205 cohort within-0.5 50.2% -> 51.2% and MAE
0.653 -> 0.624. Lock the gain: floor 0.50 -> 0.51, ceiling 0.653 -> 0.625.
#1601 note updated from HELD to RESOLVED (keep-raw).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 15:46:06 +00:00
Khalim Conn-Kowlessar
5324ae0e57 PasHub floor exposure + alternative walls + extension floor 🟩
Resolve #1601 and land #1602/#1603 on the merged-#1609 tree, adjudicated
against Khalim's verified ground truth (not pre_sap).

#1601 (upper-floor +0.25 m joist void) — RESOLVED as keep-raw. The
gov-API/Elmhurst mappers add 0.25 m to upper-storey heights because their
lodged height is a clear internal ceiling. Against verified truth RAW beats
+0.25 on all 7 verified dwellings (16 Stillwater raw -0.35 within-band vs
+0.25 -0.75; every verified property is already under truth and +0.25 only
lowers SAP). pashub surveys internal dimensions (202/205 "Measurements
Location: Internal") and its upper-ground height gap is modal 0.00, so the
joist void is not differentially present and a blanket +0.25 is unsafe.
_map_floor_dimensions keeps the raw surveyed height with a RESOLVED note.

#1602 — semi-exposed / exposed floor exposure. The surveyed "Floor type"
"Semi Exposed (unheated)" now sets is_above_partially_heated_space (RdSAP
§5.14 U=0.7) and "Exposed Floor" sets is_exposed_floor (§5.13 Table 20), on
the lowest storey (floor==0) only, mirroring the gov-API/Elmhurst siblings.
New strict-raise _pashub_floor_exposure (ADR-0015).

#1603a — alternative walls. New AlternativeWall survey dataclass +
extract_alternative_walls (the "Alternative Wall"->"Windows" section) +
_map_pashub_alternative_walls -> sap_alternative_wall_1/2. The cascade
deducts each alt-wall area from the part's main opaque wall (no double
count), so a mixed-facade dwelling is billed at each sub-area's own
construction. The alt-wall block lodges "As Built" (capital B) — aliased to
the existing default insulation code.

#1603b — extension floor construction. ExtensionConstruction gains a floor
block (parsed like the main building) and _map_extension_building_part now
threads floor_type/construction/insulation/u_value_known, which the main
path already did and the extension silently dropped.

Cohort: within-0.5 50.2% -> 51.2%, MAE 0.652 -> 0.624, 0 strict-raise
errors across all 205. Verified-truth safe: none of the 7 verified
dwellings have alt walls, and #1602 moves the one it touches (2 Philips
exposed ext floor) by -0.00. pyright 0-new (mapper baseline 39). Goldens
test_full_building_construction + end_to_end EXTENSION_1 updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 15:46:06 +00:00
Khalim Conn-Kowlessar
644bc300a6 Merge remote-tracking branch 'origin/main' into feat/1596-pashub-two-clusters
# Conflicts:
#	backend/documents_parser/tests/test_pashub_sap_accuracy.py
2026-07-15 14:02:15 +00:00
Khalim Conn-Kowlessar
30b1728337 PasHub extraction: thread showers/baths, normalize floor label, code no-water-heating 🟩
Three silent-drop extraction fixes the gov-API/Elmhurst mappers already apply
but `from_site_notes` uniquely omitted — each let a favorable default reach the
SAP-10.2 calculator and over-rate the dwelling. Same calculator, so the whole
gap was extraction fidelity (gov-API corpus 78.6% within-0.5 vs pashub 23.9%).

#1598 — electric-shower + bath counts: `_map_sap_heating` built `shower_outlets`
  but never set `electric_shower_count`/`mixer_shower_count`/`number_baths`, so
  the cascade defaulted to 1 gas mixer / 0 electric, billing electric-shower hot
  water off the cheap main system. 100/205 fixtures lodge "Electric Shower".
#1599 — floor label: pashub lodges "Ground Floor" (capital F); the calculator's
  RdSAP §5(12) suspended-timber floor-infiltration gate is case-sensitive on
  "Ground floor", so the 0.2-ACH term was silently dropped. Normalized at the
  mapper boundary. 98/205 suspended-timber.
#1600 — water-heating "None": the RdSAP 10 §10.7 "no water heating system"
  signal was discarded and HW inherited the gas combi (58 Hackle 67.7 vs
  verified 59). Coded to WHC 999 so the existing electric-immersion + Table
  28/29 default fires (also folds in #1594: blank cylinder size is no cylinder,
  not a phantom one).

Guinness GMCA 205 cohort: within-0.5 23.9% -> 50.2% (103/205), MAE 1.410 ->
0.650. Ratchets tightened to 0.50 / 0.65. Verified vs Khalim's ground truth
(16 Stillwater 78.2 -> 75.7 vs 76). #1601 (upper-floor +0.25 m) held — it
empirically over-corrects this cohort (internal-vs-external dimension question,
under investigation). pyright: 0 new errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 13:52:13 +00:00
Daniel Roth
53ba1f1694 Re-baseline the pashub MAE ceiling to 1.415 for the water-only DHW fix 🟪
Second sanctioned exception to "ratchets never loosen" — needs @KhalimCK's
call, since the prior note expected to re-tighten here. Exactly one cohort
property is WHC 950 (16 Bingley Close, +5.946 -> +6.818 = the whole +0.0043);
within-0.5 unchanged at 23.9%. Its pre_sap is verified correct but its ~+6
residual is a known extraction bug, and this hybrid gauge cannot adjudicate a
calculator fix (#1592 Problem 2). The gov-API corpus, which gauges the
calculator alone, improves.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 10:57:16 +00:00
Khalim Conn-Kowlessar
bb7f05693c Update golden for the secondary-heating int-code + retighten pashub MAE post-#1592-revert
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 09:15:46 +00:00
Khalim Conn-Kowlessar
21674d5ed0 Re-baseline the pashub gauge to 23% within-0.5 / MAE 1.41 for the orientation fix
Khalim-authorised MAE loosening: coding window orientation exposes the known
SAP-10.2-vs-lodged offset the zeroed solar gains were masking; within-0.5 is
the north-star metric and jumps 20.0% -> 23.9%.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 08:51:52 +00:00
Khalim Conn-Kowlessar
b208234e62 Ratchet the pashub accuracy gauge to MAE 1.371 after the common-wall fix
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 08:40:39 +00:00
Khalim Conn-Kowlessar
7c2cb6b88a Ratchet the pashub accuracy gauge to MAE 1.37 after the common-wall fix
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 08:35:56 +00:00
Khalim Conn-Kowlessar
8a753822c8 Capture the Simplified RIR 'Roof room type' label from the survey 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 08:27:01 +00:00
Khalim Conn-Kowlessar
600262819c Capture the Simplified RIR 'Roof room type' label from the survey 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 08:25:11 +00:00
Khalim Conn-Kowlessar
948704d7b0 Record the secondary-heating-type coding fix in the accuracy ratchet note
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 00:28:40 +00:00
Khalim Conn-Kowlessar
390a818587 Ratchet the pashub accuracy gauge to 19% within-0.5 / MAE 1.39
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 00:05:13 +00:00
Khalim Conn-Kowlessar
483b69ff41 Capture the surveyed Wall Dry-Lined answer per building part 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 23:20:36 +00:00
Khalim Conn-Kowlessar
0b47798403 Capture the surveyed Wall Dry-Lined answer per building part 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 23:20:09 +00:00
Khalim Conn-Kowlessar
8e58f7e871 Capture the per-surface RIR insulation-known answer from the survey 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 23:12:08 +00:00
Khalim Conn-Kowlessar
d743f6fb3b Capture the per-surface RIR insulation-known answer from the survey 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 23:11:09 +00:00
Khalim Conn-Kowlessar
e0b189aa6d Correct two pashub-verified pre_sap values in the accuracy manifest
Manually verified in pashub (Khalim, 2026-07-14): deal 499584755922
(9 Council Houses, CW5 8AP) is F33, not the stale D67 — our computed 34.0
was right within a point; deal 499617935574 (Brightholme, 14 North Road)
is the accredited E43, not the preliminary 53. 58 Hackle Street (D58) and
16 Bingley Close (E52) were confirmed correct, so our +7.8/+5.7 deviations
there are extraction bugs to hunt next. Harness MAE 1.684 -> 1.551; ceiling
ratcheted to 1.56.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 22:42:03 +00:00
Khalim Conn-Kowlessar
f45651dbc0 Lodge the heat-network SAP code and Table 12 community fuel for PasHub community heating 🟩
The extractor now captures "Heating System (Other):" and the mapper resolves
it to the Table 4a code (301, community boilers) + Table 12 community fuel
(51 via the shared _resolve_community_heating_fuel_code), so
is_heat_network_main routes the DLF / standing-charge / 80%-efficiency
branch instead of pricing the dwelling as an ordinary mains-gas boiler
(fixture 507644414148: +14.3 over-rate closes by ~8.6). Harness MAE
1.726 -> 1.684; ceiling ratcheted to 1.70.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 22:34:50 +00:00
Khalim Conn-Kowlessar
4d60d79cc9 Extract the PasHub community heat-source label 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 22:33:08 +00:00
Khalim Conn-Kowlessar
955b4dc1e8 Extract the PasHub detailed room-in-roof block 🟩
_parse_room_in_roof captures the RdSAP §3.10 block (age range, floor area,
gables with Exposed/Party type, slopes, common walls, flat ceiling as
length x height) on main and extension roof spaces; both part builders now
carry it to sap_room_in_roof (issue #1590 bug 5 — closes the last #1590
subtask). Harness MAE 1.748 (754917 corrects to pashub's own pre-SAP; RIR
surfaces now billed per-surface instead of as insulated loft).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 22:09:44 +00:00
Khalim Conn-Kowlessar
518c682793 Extract the PasHub detailed room-in-roof block 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 22:05:31 +00:00
Khalim Conn-Kowlessar
090e16ddc2 Dispatch PasHub mechanical ventilation into the §2 ACH cascade 🟩
The surveyed "Ventilation type" label never reached
mechanical_ventilation_kind, so ventilation_from_cert silently defaulted every
mechanical system to NATURAL (24d), dropping its ACH heat loss — 76/205
cohort fixtures lodge dMEV/MEV/PIV (issue #1590 bug 4). Mapping mirrors the
gov-API _API_MECHANICAL_VENTILATION_TO_KIND table (PIV-from-loft is
"as natural"). Harness: within-0.5 12.2% -> 18.5%, MAE 2.538 -> 1.730 — the
campaign's largest single move; floors ratcheted to 0.18 / 1.75.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:45:24 +00:00
Khalim Conn-Kowlessar
56b36ce7f0 Resolve the PasHub PV Connection label to the gov-API credit-gate int 🟩
"Not connected to electricity meter" -> 1 (zero Appendix M credit), "Connected
to dwellings electricity meter" -> 2; the raw string fell through the credit
gate's non-int branch to True, crediting a separately-metered array (issue
#1590 bug 3, fixture 499516101839 -6.9 SAP). Harness MAE 2.571 -> 2.538;
ceiling ratcheted to 2.55.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:41:04 +00:00
Khalim Conn-Kowlessar
f92d4f26b8 Extract the PasHub per-system PV block and diverter flag 🟩
"Photovoltaic array kWp Known? Yes" lodges a per-system block (kWp, pitch,
orientation, overshading) instead of the percent-roof estimate; the extractor
only read the estimate, so measured PV never reached the mapper and the
Appendix M credit was silently zero (issue #1590 bug 1). Harness MAE
2.632 -> 2.571 (fixture 507639151843's 13.5-SAP under-rate closes to +0.5);
ceiling ratcheted to 2.58.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:38:07 +00:00
Khalim Conn-Kowlessar
80de9b0c04 Extract the PasHub per-system PV block and diverter flag 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:35:34 +00:00
Khalim Conn-Kowlessar
bc58cf4930 Lodge explicit zero roof insulation for a surveyed "Insulation At: None" 🟩
"Insulation At: None" is the assessor recording zero loft insulation, not an
unknown — with thickness left None, u_roof fell through to the age-band
"assumed insulated" default (0.16 W/m²K at band J/K) instead of the Table 16
uninsulated row (2.30), understating roof loss ~14x (issue #1590 bug 2).
Harness MAE 2.701 -> 2.632 (the two ~7-SAP roof over-raters correct exactly);
ceiling ratcheted to 2.64.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:28:25 +00:00
Khalim Conn-Kowlessar
999008f9d0 Refresh pashub accuracy manifest pre_sap from the corrected DB rows
Two hubspot_deal_data corrections: 499539910905 D58->B82 (confirmed bad row)
and one C73->C75. Rebuilt via scripts/build_pashub_accuracy_fixtures.py
(no --force; PDFs unchanged).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:24:37 +00:00
Khalim Conn-Kowlessar
cab7e3d613 Close the last PasHub mapper gaps: secondary fuel + Group 3 heat-network control 🟩
Secondary fuel 'Mains gas'/'Mains Gas'->26 and 'House Coal'->11 (Table 32);
main-heating control resolution becomes group-aware with a Table 4e Group 3
map ('Charging system linked to use of community heating, room thermostat
only' -> 2306, the code the accredited Elmhurst certs lodge for this cohort;
all four linked-to-use codes are calculation-identical). All 205 Guinness
fixtures now compute (was 201/4 xfails); ratchet re-baselined to the
full-coverage aggregate (12.2% within-0.5 floor, MAE ceiling 2.71) with the
two newly-computable outliers documented for verification.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:24:36 +00:00
Daniel Roth
ecb113b52e Re-ratchet PasHub SAP-accuracy floor to the current integrated aggregate (12.4% within-0.5) 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 17:57:34 +00:00
Daniel Roth
df3fce0a3f
Merge branch 'main' into feature/pashub-map-main-heating-control-1557 2026-07-14 18:50:08 +01:00
Daniel Roth
98aa0127a7 Keep swallowing the control gap's boundary form in the accuracy harness 🟪
The control gap surfaces as UnmappedSapCode on main and as UnmappedPasHubLabel
once the #1557 control mapper lands (unmapped community-heating control), so the
harness must swallow both; only the closed fuel-specific MissingMainFuelType is
dropped as a fuel-drop tripwire.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 17:40:50 +00:00
Daniel Roth
88f221b86e Stop swallowing closed PAS Hub main-fuel gaps in the accuracy harness 🟪
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 17:24:15 +00:00
Daniel Roth
8529c71ff2 Capture PAS Hub main-heating fuel from the colon 'Fuel:' label variant 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 17:19:12 +00:00
Daniel Roth
2783c2c0c6 Capture PAS Hub main-heating fuel from the colon 'Fuel:' label variant 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 17:18:46 +00:00
Daniel Roth
aa69462b5b Raise PasHub SAP-accuracy floor to 13.4% within-0.5 on the merged fabric stack 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 16:55:03 +00:00
Daniel Roth
5da621c7a5 Ratchet PasHub SAP-accuracy floor to 10.9% within-0.5 (control-code fix unblocks 201/205) 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 16:43:54 +00:00
Khalim Conn-Kowlessar
e5fee3896a Add PAS Hub SAP-accuracy regression harness (Guinness 205 cohort)
Foundation for PRD #1555: runs each PAS Hub site-note PDF through the
extractor -> EpcPropertyData -> Sap10Calculator and gauges the computed
SAP against pashub's own SAP-10.2 `pre_sap` (from hubspot_deal_data).

- test_pashub_sap_accuracy.py: hybrid gate. Per-fixture "must compute"
  (xfail on the known in-progress mapper gaps MissingMainFuelType /
  UnmappedSapCode / UnmappedPasHubLabel) + aggregate within-0.5 ratchet
  floor, mirroring test_sap_accuracy_corpus.py.
- 205 image-stripped site-note PDFs + manifest.json. Images stripped so
  the repo footprint stays ~52MB while the text layer the extractor reads
  is byte-identical.
- build_pashub_accuracy_fixtures.py: provenance/rebuild from S3 +
  hubspot_deal_data.

All 206 currently xfail on the known heating-string mapper gaps; each fix
(#1556-1568) flips its fixtures to computing and ratchets the floor.

Refs #1555 #1568

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 15:32:18 +00:00
Daniel Roth
442352c378 Drop the unused DownloadedFile import from the client tests 🟪
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 14:07:23 +00:00
Daniel Roth
38cae15f4d Preserve SharePoint distribution when the site-note save fails loudly 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 14:06:45 +00:00
Daniel Roth
e31d2ad2d1 Preserve SharePoint distribution when the site-note save fails loudly 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 14:06:45 +00:00
Daniel Roth
62c0151469 Fill partial summaries best-effort, fail loudly, skip evidence-only jobs, and retry on Coordination Hub 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 14:06:45 +00:00