Commit graph

337 commits

Author SHA1 Message Date
Khalim Conn-Kowlessar
d3f36d78ac PasHub SAP accuracy: pin 13 Walsden to its re-survey SAP 64, not the colliding 68 🟩
13 Walsden (deal 499630239947, uprn 77179782) has TWO distinct gov-EPC
assessments: the PAS Hub site-note re-survey we parse (26 May 2026,
accredited SAP 64, backed by rd_sap_site_note files) and a SEPARATE
later lodged cert (24 Jun 2026, SAP 68, registered, no site-note). The
portfolio-838 baseline latched onto the 24-Jun SAP-68 cert; our engine
models the survey it was given and reproduces its own accredited 64
(64.19). 6 of 7 Walsden Street homes have pbp oracle == PAS Hub survey
SAP exactly — #13 is the lone collision.

A durable regression pin (asserts our SAP within 0.5 of a literal 64,
not the manifest value the fixture-builder can revert to a colliding DB
row) documents the provenance so nobody "corrects" the target to 68.
13 Walsden is a known re-survey-diverges outlier, not an engine
under-rate; confirmed against pashub (Khalim, 2026-07-15e).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 00:52:33 +00:00
Khalim Conn-Kowlessar
4c0ccad882 PasHub SAP accuracy: semi-exposed-unheated floor → Table 20, not §5.14 🟩
`_PASHUB_FLOOR_EXPOSURE` mapped the surveyed "Semi Exposed (unheated)"
floor label to `is_above_partially_heated_space` (RdSAP §5.14, constant
U=0.7). A floor semi-exposed to an enclosed UNHEATED space belongs in
§5.13 Table 20 `u_exposed_floor` (band-B as-built U=1.20). §5.14's U=0.7
is reserved for a floor above a *partially* heated space. #1602
transposed the two branches; the too-low U under-counted floor loss on
the 9 label-carrying cohort fixtures → a shared over-credit.

Triangulated against all three siblings + the calculator docstring: the
gov-API code-2 "To unheated space" (mapper.py:4797) and Elmhurst
`_is_floor_exposed_to_unheated_space` (mapper.py:5970) both route to
Table 20; rdsap_uvalues.py:1416 documents Table 20 as collapsing exposed
and semi-exposed-to-unheated into one lookup. One-line data flip,
from_site_notes-only (gov-API RdSAP corpus untouched, still 78.9%).

Adjacent Ronald St mid-terrace pair (the clustered signal): 10 Ronald
60.35→59.02 (oracle 59), 12 Ronald 60.79→59.36 (oracle 59). Cohort vs
DB oracle (portfolio 838 effective_sap) 83.3%→86.9% within-0.5, MAE
0.377→0.348, no built form regresses, all 7 verified dwellings unchanged
(none carry the label). pashub harness 79.5%→82.4%, MAE 0.389→0.367;
ratchets tightened 0.78→0.82 / 0.40→0.37. pyright zero-new.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 00:52:33 +00:00
KhalimCK
04b7c26f67
PasHub SAP accuracy: infiltration inputs + immersion DHW + lights/lobby/heat-network (#1615)
* Electric-immersion DHW: bill at 100%, not the space boiler's efficiency 🟩

A separate electric immersion heater (SAP water-heating code 903) is 100%
efficient (SAP 10.2 Table 4a) and the space-heating boiler provides no water
heating, so Appendix D2.1 Eq D1 (the boiler seasonal-efficiency cascade) must
not apply to it. But `_water_heating_main` resolves DHW to the SPACE main — a
gas/oil boiler keeps its PCDB record — so three water-efficiency branches in
cert_to_inputs billed the immersion-heated cylinder at the boiler's ~87% summer
efficiency (a mongrel: electric fuel price x gas-boiler efficiency):

  1. the scalar `water_eff = water_pcdb_main.summer_efficiency_pct / 100`,
  2. the SAP §9.4.11 / Table 4c(2) -5pp no-interlock adjustment, and
  3. the Eq D1 (winter, summer) seasonal pair from `pcdb_main`.

Gate all three on `not dhw_is_electric_immersion`; the correct immersion path
(`_water_efficiency_with_category_inherit` -> 1.0, Eq D1 off) then applies,
mirroring the Table 3 zero-primary-loss gate already present for WHC 903.

General bug — fires for any WHC-903 dwelling whose space main is a PCDB gas/oil
boiler. It surfaced via the PasHub campaign's #1600 no-water-heating default
(WHC 999 -> 903) on a gas-combi dwelling: 58 Hackle St M11 4WU, SAP 55.30 ->
57.67 (pre_sap 58, verified 59).

Guardrails: the gov-API RdSAP corpus IMPROVES 78.8% -> 78.9% within-0.5, MAE
0.625 -> 0.622 (a handful of corpus certs with a boiler space main + electric
immersion move closer to accredited) — MAE ceiling ratcheted 0.626 -> 0.625.
Regression pinned in test_cert_to_inputs (RED before / GREEN after). pyright
0-new (cert_to_inputs baseline 30).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* PasHub SAP accuracy: thread low-energy lights + draught lobby (from_site_notes) 🟩

Two systematic silent-drops in the PAS Hub `from_site_notes` path, each a field
the gov-API/Elmhurst mappers thread but site-notes dropped, letting a favourable
default reach the SAP-10.2 calculator. Validated against the correct pashub
oracle (property_baseline_performance.effective_sap_score, portfolio 838):
cohort within-0.5 55.1% → 62.6%, MAE 0.599 → 0.521.

- Low-energy lights: when "exact LED/CFL known = No", PAS Hub lodges an aggregate
  "Number of fixed low energy lights?" count. Previously dropped (no dataclass
  field / extractor key / mapper thread) → calculator saw 0 low-energy bulbs and
  applied the pessimistic L5b/L8c no-data default, under-rating SAP. Now threaded
  into low_energy_fixed_lighting_bulbs_count, mirroring from_elmhurst_site_notes.
  ~6% of the cohort; resolves 58 Hackle (−0.43 vs oracle 59 with #1615).
- Draught lobby: _map_sap_ventilation set only the legacy `draught_lobby` field,
  never the canonical `has_draught_lobby` §2 (13) gate the cascade reads, so the
  surveyed lobby was ignored and infiltration over-stated. Now mirrors Elmhurst.

Two other audited levers were REJECTED against the pashub oracle (they matched
the gov-cert/RdSAP convention but pashub does not apply them): percent_draughtproofed
(cohort 30.8%) and the §A.2.2 assumed secondary heater (19.2%). A heat-network
control-code fix (2306→2303 for 16 Bingley) is HELD pending spec/Elmhurst
adjudication — it contradicts ADR-0053; see the NOTE on _PASHUB_HEAT_NETWORK_CONTROL_TO_SAP10.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* PasHub heat-network control: room-thermostat-only maps to 2308, not 2306 🟩

The Table 4e Group 3 label "charging linked to use of community heating, room
thermostat only" was mapped to 2306 — the linked-to-use *with-TRVs* code
(control type 3, space charging 1.00). The label explicitly excludes TRVs, so
it is control type 2 / space 1.05 = code 2308/2309 (linked-to-use → DHW 1.00).
2306 asserted TRVs the survey doesn't have and over-rated the cohort's sole
heat-network dwelling (16 Bingley Close 56.89 → 54.01 vs pashub oracle 52).

Verified against SAP 10.2 Table 4e as encoded in cert_to_inputs.py
(_CONTROL_TYPE_BY_CODE 2308→2, space-charging 2308→1.05, DHW 2308→1.00) and the
RdSAP control-label vocabulary in MainheatControlAttributes.py. 2303 (the earlier
audit's guess) is rejected: it is a flat-rate code (DHW 1.05) whose oracle match
was a coincidence of two offsetting spec violations. The residual +2.0 to oracle
52 is the documented SAP-10.2-engine-vs-lodged offset, not a fuel/flags gap
(those were threaded by the #1590 follow-up) — supersedes the ADR-0053 /
HANDOVER_838 "community fuel/flags" attribution.

16 Bingley is the only heat-network fixture in portfolio 838, so this moves one
fixture strictly toward its oracle. Harness ratcheted: within-0.5 0.51→0.58,
MAE 0.625→0.521 (observed 58.5% / 0.520 with all three 2026-07-15 levers).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* PasHub SAP accuracy: restore 3 dropped §2 infiltration inputs (from_site_notes)

`from_site_notes` uniquely dropped three ventilation inputs that the gov-API
and Elmhurst sibling mappers all set, causing a systematic cohort-wide SAP
under-rate vs the company's own accredited SAP-10.2 certs
(property_baseline_performance.effective_sap_score, portfolio 838):

1. percent_draughtproofed — never set, so §2(15) window infiltration was
   pinned at its 0.25-ACH worst case on every dwelling. Reconstructed as the
   area-weighted % of draught-proofed windows (mirrors Elmhurst
   `draught_proofing_percent`). Dominant term.
2. Upper-storey +0.25 m joist void — omitted; now added, matching
   `_UPPER_FLOOR_HEIGHT_ADD_M` on the gov-API/Elmhurst paths. This
   RE-ADJUDICATES #1601 ("keep raw"): that call was confounded by the
   then-present draught-proofing drop suppressing every verified dwelling.
3. sheltered_sides — left None → calculator's flat default of 2, which
   over-shelters end/semi/detached (RdSAP §S5 = 1/1/0). Now derived from
   built form.

The three are NON-ADDITIVE — each overshoots alone (which is why all three
were previously rejected individually) — but together they land all 7
verified ground-truth dwellings toward truth (none regress) and every built
form near zero. Cohort 62.1% -> 83.3% within-0.5, MAE 0.507 -> 0.377.

Guardrails: gov-API RdSAP corpus unchanged (78.9%, these are from_site_notes-
only helpers). pashub harness 58.5% -> 79.5% / MAE 0.520 -> 0.389; ratchets
tightened 0.58->0.78 and 0.521->0.40. pyright zero-new (mapper baseline 39).
New focused tests in TestFromSiteNotesInfiltrationFixes cover all three
fields incl. area-weighting and the built-form shelter map; goldens updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 00:30:55 +01:00
Khalim Conn-Kowlessar
6fd0393a4e Tighten pashub accuracy ratchets to 0.51 / 0.625 🟪
#1602 (floor exposure flags) + #1603 (alternative walls + extension floor
construction) move the Guinness 205 cohort within-0.5 50.2% -> 51.2% and MAE
0.653 -> 0.624. Lock the gain: floor 0.50 -> 0.51, ceiling 0.653 -> 0.625.
#1601 note updated from HELD to RESOLVED (keep-raw).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 15:46:06 +00:00
Khalim Conn-Kowlessar
5324ae0e57 PasHub floor exposure + alternative walls + extension floor 🟩
Resolve #1601 and land #1602/#1603 on the merged-#1609 tree, adjudicated
against Khalim's verified ground truth (not pre_sap).

#1601 (upper-floor +0.25 m joist void) — RESOLVED as keep-raw. The
gov-API/Elmhurst mappers add 0.25 m to upper-storey heights because their
lodged height is a clear internal ceiling. Against verified truth RAW beats
+0.25 on all 7 verified dwellings (16 Stillwater raw -0.35 within-band vs
+0.25 -0.75; every verified property is already under truth and +0.25 only
lowers SAP). pashub surveys internal dimensions (202/205 "Measurements
Location: Internal") and its upper-ground height gap is modal 0.00, so the
joist void is not differentially present and a blanket +0.25 is unsafe.
_map_floor_dimensions keeps the raw surveyed height with a RESOLVED note.

#1602 — semi-exposed / exposed floor exposure. The surveyed "Floor type"
"Semi Exposed (unheated)" now sets is_above_partially_heated_space (RdSAP
§5.14 U=0.7) and "Exposed Floor" sets is_exposed_floor (§5.13 Table 20), on
the lowest storey (floor==0) only, mirroring the gov-API/Elmhurst siblings.
New strict-raise _pashub_floor_exposure (ADR-0015).

#1603a — alternative walls. New AlternativeWall survey dataclass +
extract_alternative_walls (the "Alternative Wall"->"Windows" section) +
_map_pashub_alternative_walls -> sap_alternative_wall_1/2. The cascade
deducts each alt-wall area from the part's main opaque wall (no double
count), so a mixed-facade dwelling is billed at each sub-area's own
construction. The alt-wall block lodges "As Built" (capital B) — aliased to
the existing default insulation code.

#1603b — extension floor construction. ExtensionConstruction gains a floor
block (parsed like the main building) and _map_extension_building_part now
threads floor_type/construction/insulation/u_value_known, which the main
path already did and the extension silently dropped.

Cohort: within-0.5 50.2% -> 51.2%, MAE 0.652 -> 0.624, 0 strict-raise
errors across all 205. Verified-truth safe: none of the 7 verified
dwellings have alt walls, and #1602 moves the one it touches (2 Philips
exposed ext floor) by -0.00. pyright 0-new (mapper baseline 39). Goldens
test_full_building_construction + end_to_end EXTENSION_1 updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 15:46:06 +00:00
Khalim Conn-Kowlessar
644bc300a6 Merge remote-tracking branch 'origin/main' into feat/1596-pashub-two-clusters
# Conflicts:
#	backend/documents_parser/tests/test_pashub_sap_accuracy.py
2026-07-15 14:02:15 +00:00
Khalim Conn-Kowlessar
30b1728337 PasHub extraction: thread showers/baths, normalize floor label, code no-water-heating 🟩
Three silent-drop extraction fixes the gov-API/Elmhurst mappers already apply
but `from_site_notes` uniquely omitted — each let a favorable default reach the
SAP-10.2 calculator and over-rate the dwelling. Same calculator, so the whole
gap was extraction fidelity (gov-API corpus 78.6% within-0.5 vs pashub 23.9%).

#1598 — electric-shower + bath counts: `_map_sap_heating` built `shower_outlets`
  but never set `electric_shower_count`/`mixer_shower_count`/`number_baths`, so
  the cascade defaulted to 1 gas mixer / 0 electric, billing electric-shower hot
  water off the cheap main system. 100/205 fixtures lodge "Electric Shower".
#1599 — floor label: pashub lodges "Ground Floor" (capital F); the calculator's
  RdSAP §5(12) suspended-timber floor-infiltration gate is case-sensitive on
  "Ground floor", so the 0.2-ACH term was silently dropped. Normalized at the
  mapper boundary. 98/205 suspended-timber.
#1600 — water-heating "None": the RdSAP 10 §10.7 "no water heating system"
  signal was discarded and HW inherited the gas combi (58 Hackle 67.7 vs
  verified 59). Coded to WHC 999 so the existing electric-immersion + Table
  28/29 default fires (also folds in #1594: blank cylinder size is no cylinder,
  not a phantom one).

Guinness GMCA 205 cohort: within-0.5 23.9% -> 50.2% (103/205), MAE 1.410 ->
0.650. Ratchets tightened to 0.50 / 0.65. Verified vs Khalim's ground truth
(16 Stillwater 78.2 -> 75.7 vs 76). #1601 (upper-floor +0.25 m) held — it
empirically over-corrects this cohort (internal-vs-external dimension question,
under investigation). pyright: 0 new errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 13:52:13 +00:00
Daniel Roth
53ba1f1694 Re-baseline the pashub MAE ceiling to 1.415 for the water-only DHW fix 🟪
Second sanctioned exception to "ratchets never loosen" — needs @KhalimCK's
call, since the prior note expected to re-tighten here. Exactly one cohort
property is WHC 950 (16 Bingley Close, +5.946 -> +6.818 = the whole +0.0043);
within-0.5 unchanged at 23.9%. Its pre_sap is verified correct but its ~+6
residual is a known extraction bug, and this hybrid gauge cannot adjudicate a
calculator fix (#1592 Problem 2). The gov-API corpus, which gauges the
calculator alone, improves.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 10:57:16 +00:00
Khalim Conn-Kowlessar
bb7f05693c Update golden for the secondary-heating int-code + retighten pashub MAE post-#1592-revert
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 09:15:46 +00:00
Khalim Conn-Kowlessar
21674d5ed0 Re-baseline the pashub gauge to 23% within-0.5 / MAE 1.41 for the orientation fix
Khalim-authorised MAE loosening: coding window orientation exposes the known
SAP-10.2-vs-lodged offset the zeroed solar gains were masking; within-0.5 is
the north-star metric and jumps 20.0% -> 23.9%.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 08:51:52 +00:00
Khalim Conn-Kowlessar
b208234e62 Ratchet the pashub accuracy gauge to MAE 1.371 after the common-wall fix
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 08:40:39 +00:00
Khalim Conn-Kowlessar
7c2cb6b88a Ratchet the pashub accuracy gauge to MAE 1.37 after the common-wall fix
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 08:35:56 +00:00
Khalim Conn-Kowlessar
8a753822c8 Capture the Simplified RIR 'Roof room type' label from the survey 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 08:27:01 +00:00
Khalim Conn-Kowlessar
600262819c Capture the Simplified RIR 'Roof room type' label from the survey 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 08:25:11 +00:00
Khalim Conn-Kowlessar
948704d7b0 Record the secondary-heating-type coding fix in the accuracy ratchet note
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 00:28:40 +00:00
Khalim Conn-Kowlessar
390a818587 Ratchet the pashub accuracy gauge to 19% within-0.5 / MAE 1.39
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 00:05:13 +00:00
Khalim Conn-Kowlessar
483b69ff41 Capture the surveyed Wall Dry-Lined answer per building part 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 23:20:36 +00:00
Khalim Conn-Kowlessar
0b47798403 Capture the surveyed Wall Dry-Lined answer per building part 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 23:20:09 +00:00
Khalim Conn-Kowlessar
8e58f7e871 Capture the per-surface RIR insulation-known answer from the survey 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 23:12:08 +00:00
Khalim Conn-Kowlessar
d743f6fb3b Capture the per-surface RIR insulation-known answer from the survey 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 23:11:09 +00:00
Khalim Conn-Kowlessar
e0b189aa6d Correct two pashub-verified pre_sap values in the accuracy manifest
Manually verified in pashub (Khalim, 2026-07-14): deal 499584755922
(9 Council Houses, CW5 8AP) is F33, not the stale D67 — our computed 34.0
was right within a point; deal 499617935574 (Brightholme, 14 North Road)
is the accredited E43, not the preliminary 53. 58 Hackle Street (D58) and
16 Bingley Close (E52) were confirmed correct, so our +7.8/+5.7 deviations
there are extraction bugs to hunt next. Harness MAE 1.684 -> 1.551; ceiling
ratcheted to 1.56.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 22:42:03 +00:00
Khalim Conn-Kowlessar
f45651dbc0 Lodge the heat-network SAP code and Table 12 community fuel for PasHub community heating 🟩
The extractor now captures "Heating System (Other):" and the mapper resolves
it to the Table 4a code (301, community boilers) + Table 12 community fuel
(51 via the shared _resolve_community_heating_fuel_code), so
is_heat_network_main routes the DLF / standing-charge / 80%-efficiency
branch instead of pricing the dwelling as an ordinary mains-gas boiler
(fixture 507644414148: +14.3 over-rate closes by ~8.6). Harness MAE
1.726 -> 1.684; ceiling ratcheted to 1.70.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 22:34:50 +00:00
Khalim Conn-Kowlessar
4d60d79cc9 Extract the PasHub community heat-source label 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 22:33:08 +00:00
Khalim Conn-Kowlessar
955b4dc1e8 Extract the PasHub detailed room-in-roof block 🟩
_parse_room_in_roof captures the RdSAP §3.10 block (age range, floor area,
gables with Exposed/Party type, slopes, common walls, flat ceiling as
length x height) on main and extension roof spaces; both part builders now
carry it to sap_room_in_roof (issue #1590 bug 5 — closes the last #1590
subtask). Harness MAE 1.748 (754917 corrects to pashub's own pre-SAP; RIR
surfaces now billed per-surface instead of as insulated loft).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 22:09:44 +00:00
Khalim Conn-Kowlessar
518c682793 Extract the PasHub detailed room-in-roof block 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 22:05:31 +00:00
Khalim Conn-Kowlessar
090e16ddc2 Dispatch PasHub mechanical ventilation into the §2 ACH cascade 🟩
The surveyed "Ventilation type" label never reached
mechanical_ventilation_kind, so ventilation_from_cert silently defaulted every
mechanical system to NATURAL (24d), dropping its ACH heat loss — 76/205
cohort fixtures lodge dMEV/MEV/PIV (issue #1590 bug 4). Mapping mirrors the
gov-API _API_MECHANICAL_VENTILATION_TO_KIND table (PIV-from-loft is
"as natural"). Harness: within-0.5 12.2% -> 18.5%, MAE 2.538 -> 1.730 — the
campaign's largest single move; floors ratcheted to 0.18 / 1.75.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:45:24 +00:00
Khalim Conn-Kowlessar
56b36ce7f0 Resolve the PasHub PV Connection label to the gov-API credit-gate int 🟩
"Not connected to electricity meter" -> 1 (zero Appendix M credit), "Connected
to dwellings electricity meter" -> 2; the raw string fell through the credit
gate's non-int branch to True, crediting a separately-metered array (issue
#1590 bug 3, fixture 499516101839 -6.9 SAP). Harness MAE 2.571 -> 2.538;
ceiling ratcheted to 2.55.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:41:04 +00:00
Khalim Conn-Kowlessar
f92d4f26b8 Extract the PasHub per-system PV block and diverter flag 🟩
"Photovoltaic array kWp Known? Yes" lodges a per-system block (kWp, pitch,
orientation, overshading) instead of the percent-roof estimate; the extractor
only read the estimate, so measured PV never reached the mapper and the
Appendix M credit was silently zero (issue #1590 bug 1). Harness MAE
2.632 -> 2.571 (fixture 507639151843's 13.5-SAP under-rate closes to +0.5);
ceiling ratcheted to 2.58.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:38:07 +00:00
Khalim Conn-Kowlessar
80de9b0c04 Extract the PasHub per-system PV block and diverter flag 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:35:34 +00:00
Khalim Conn-Kowlessar
bc58cf4930 Lodge explicit zero roof insulation for a surveyed "Insulation At: None" 🟩
"Insulation At: None" is the assessor recording zero loft insulation, not an
unknown — with thickness left None, u_roof fell through to the age-band
"assumed insulated" default (0.16 W/m²K at band J/K) instead of the Table 16
uninsulated row (2.30), understating roof loss ~14x (issue #1590 bug 2).
Harness MAE 2.701 -> 2.632 (the two ~7-SAP roof over-raters correct exactly);
ceiling ratcheted to 2.64.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:28:25 +00:00
Khalim Conn-Kowlessar
999008f9d0 Refresh pashub accuracy manifest pre_sap from the corrected DB rows
Two hubspot_deal_data corrections: 499539910905 D58->B82 (confirmed bad row)
and one C73->C75. Rebuilt via scripts/build_pashub_accuracy_fixtures.py
(no --force; PDFs unchanged).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:24:37 +00:00
Khalim Conn-Kowlessar
cab7e3d613 Close the last PasHub mapper gaps: secondary fuel + Group 3 heat-network control 🟩
Secondary fuel 'Mains gas'/'Mains Gas'->26 and 'House Coal'->11 (Table 32);
main-heating control resolution becomes group-aware with a Table 4e Group 3
map ('Charging system linked to use of community heating, room thermostat
only' -> 2306, the code the accredited Elmhurst certs lodge for this cohort;
all four linked-to-use codes are calculation-identical). All 205 Guinness
fixtures now compute (was 201/4 xfails); ratchet re-baselined to the
full-coverage aggregate (12.2% within-0.5 floor, MAE ceiling 2.71) with the
two newly-computable outliers documented for verification.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 21:24:36 +00:00
Daniel Roth
ecb113b52e Re-ratchet PasHub SAP-accuracy floor to the current integrated aggregate (12.4% within-0.5) 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 17:57:34 +00:00
Daniel Roth
df3fce0a3f
Merge branch 'main' into feature/pashub-map-main-heating-control-1557 2026-07-14 18:50:08 +01:00
Daniel Roth
98aa0127a7 Keep swallowing the control gap's boundary form in the accuracy harness 🟪
The control gap surfaces as UnmappedSapCode on main and as UnmappedPasHubLabel
once the #1557 control mapper lands (unmapped community-heating control), so the
harness must swallow both; only the closed fuel-specific MissingMainFuelType is
dropped as a fuel-drop tripwire.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 17:40:50 +00:00
Daniel Roth
88f221b86e Stop swallowing closed PAS Hub main-fuel gaps in the accuracy harness 🟪
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 17:24:15 +00:00
Daniel Roth
8529c71ff2 Capture PAS Hub main-heating fuel from the colon 'Fuel:' label variant 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 17:19:12 +00:00
Daniel Roth
2783c2c0c6 Capture PAS Hub main-heating fuel from the colon 'Fuel:' label variant 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 17:18:46 +00:00
Daniel Roth
aa69462b5b Raise PasHub SAP-accuracy floor to 13.4% within-0.5 on the merged fabric stack 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 16:55:03 +00:00
Daniel Roth
5da621c7a5 Ratchet PasHub SAP-accuracy floor to 10.9% within-0.5 (control-code fix unblocks 201/205) 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 16:43:54 +00:00
Khalim Conn-Kowlessar
e5fee3896a Add PAS Hub SAP-accuracy regression harness (Guinness 205 cohort)
Foundation for PRD #1555: runs each PAS Hub site-note PDF through the
extractor -> EpcPropertyData -> Sap10Calculator and gauges the computed
SAP against pashub's own SAP-10.2 `pre_sap` (from hubspot_deal_data).

- test_pashub_sap_accuracy.py: hybrid gate. Per-fixture "must compute"
  (xfail on the known in-progress mapper gaps MissingMainFuelType /
  UnmappedSapCode / UnmappedPasHubLabel) + aggregate within-0.5 ratchet
  floor, mirroring test_sap_accuracy_corpus.py.
- 205 image-stripped site-note PDFs + manifest.json. Images stripped so
  the repo footprint stays ~52MB while the text layer the extractor reads
  is byte-identical.
- build_pashub_accuracy_fixtures.py: provenance/rebuild from S3 +
  hubspot_deal_data.

All 206 currently xfail on the known heating-string mapper gaps; each fix
(#1556-1568) flips its fixtures to computing and ratchets the floor.

Refs #1555 #1568

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 15:32:18 +00:00
Daniel Roth
b5c05166db Persist resolved UPRN on PasHub site-notes rows (#1537)
Thread the UPRN the PashubService has already resolved for a job into the
PasHub site-notes mapper so the EpcPropertyData aggregate is born with its
uprn set, and the existing save_epc_property_data path persists it.

- EpcPropertyDataMapper.from_site_notes gains uprn: Optional[int] = None and
  sets it unconditionally (site notes never carry a UPRN natively).
- parse_site_notes_pdf / _parse_pashub forward the uprn; the Elmhurst branch
  is untouched.
- PashubService coerces uprn str -> int in one place, carries it on the
  internal upload record, and passes it into parse_site_notes_pdf.
- No new lookups; jobs with no known UPRN still persist null, unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 13:19:28 +00:00
Daniel Roth
a174301136 rename mains_gas to gas_connection_available 2026-07-01 10:50:58 +00:00
Khalim Conn-Kowlessar
4e2f2bdcc7 test(worksheet): pin simulated case 50 — MVHR + dual-immersion all-electric
Adds the mapper-driven e2e cascade pin for "simulated case 50" (000565 semi,
electric storage main SAP 402 + portable electric secondary + MVHR + whc-903
DUAL electric immersion + 160 L cylinder, Economy-7). Routes the Summary PDF
through extractor + mapper + calculator like the other 000565 fixtures.

Locks in two off-peak fixes this case ground-truthed:
- the Table 13 HW high/low split applied to CO2/PE (commit 39ae2cf0), and
- the Table 12a Grid 2 MVHR fan fraction 0.71/0.58 (commit cd5113ab).

All 11 SAP-result fields reconcile to the U985 worksheet EXACTLY, including
the (272) rating CO2 2397.1237 — SAP 38.8426 (=39), cost £1317.0116, water
1668.0788 kWh, fans 315.6384 kWh.

Summary mirrored to the tracked fixtures dir so the test doesn't depend on
the unstaged `sap worksheets/` workspace.

pyright strict gate not run locally (pyright not installed in this container).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 09:07:16 +00:00
Khalim Conn-Kowlessar
f3e3494bf7 test(worksheet): pin simulated case 52 — regular gas boiler + cylinder
Adds the mapper-driven e2e cascade pin for "simulated case 52" (000565
semi + regular non-combi mains-gas boiler SAP 102 + 160 L foam cylinder
heated from the main, no cylinder stat, uninsulated primary pipework,
standard tariff). Routes the Summary PDF through extractor + mapper +
calculator like the other 000565 / 001431_case* fixtures.

This closes the last untested branch of the cylinder/water chain: the
SAP 10.2 §4 cylinder storage loss (Table 2/2a/2b lines 51-55) + the
Table 3 PRIMARY circuit loss (59, uninsulated pipework + no stat) that
combi/immersion fixtures don't reach. All 11 SAP-result fields reconcile
to the U985 worksheet EXACTLY with no calculator change — SAP 57.2904
(=57), cost £911.1973, water 3929.7635 kWh — confirming the cylinder-loss
derivation is correct.

Summary mirrored to the tracked fixtures dir so the test doesn't depend
on the unstaged `sap worksheets/` workspace.

pyright strict gate not run locally (pyright not installed in this container).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 08:50:19 +00:00
Khalim Conn-Kowlessar
fc7c4d2d3b fix(climate): compute EPC CO2/PE on the postcode demand cascade (SAP 10.2 Appendix U p.124)
The SAP/EI rating is computed on UK-average weather (Appendix U Tables
U1-U3 region 0) so ratings are nationally comparable, but Appendix U
paragraph 1 (PDF p.124) requires that "other calculations (such as for
energy use and costs on EPCs) are done using local weather. Weather data
for each postcode district are taken from the PCDB". `Sap10Calculator.
calculate` ran ONE cascade (UK-average) and fed it to SAP, CO2 AND primary
energy, so every cert's EPC-displayed CO2/PE were computed on the wrong
climate. Because most of England is warmer than the UK-average, this
systematically OVER-counted heating demand on the emissions/PE outputs.

The two cascades (`cert_to_inputs` rating, `cert_to_demand_inputs`
postcode) already existed; this wires the demand cascade into the
production entry point and grafts its CO2/PE onto the rating result (SAP
unchanged). The corpus gauge's longstanding +5% CO2/PE over-estimate was
mostly this climate bug, NOT (as previously diagnosed) per-cert mapper
fidelity:
  CO2 MAE 0.26 -> 0.12 t/yr  (bias +0.18 -> +0.04)
  PE  MAE 13.6 -> 3.8 kWh/m2 (bias +9.0  -> +0.24)
  SAP within-0.5 = 69.7% (rating cascade, unchanged)

Worksheet-validated to 1e-4 on simulated case 45 (heat-pump ground-floor
flat, postcode W6): the P960 prints the current dwelling twice — Block 1
on UK-average weather (SAP 60.5318, CO2 692.13) and Block 2 on postcode
weather (CO2 626.78, PE 6581.59). Both reproduce exactly. Added a tracked
case-45 Summary fixture + two-cascade cascade pin as a permanent guard,
and ratcheted the corpus CO2/PE ceilings to 0.13 / 4.2. The e2e Elmhurst
suite (Block-1 line refs) now pins the rating cascade directly; the two
Vaillant overlay snapshots refreshed to demand-cascade CO2/PE.

pyright not installed in this codespace (strict gate not run locally);
change is type-trivial (dataclasses.replace over SapResult).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 14:15:34 +00:00
Khalim Conn-Kowlessar
fe3bf4eaed fix(ventilation): read Blower Door AP50 pressure test (Summary)
SAP 10.2 §2 (17)-(18): a measured/design air permeability at 50 Pa from a
Blower Door test routes infiltration via `(18) = AP50/20 + (8)`, in
preference to the components-based (16) estimate. The Elmhurst extractor
read only the AP4 ("Pulse") column of §12.2, so a Blower Door result
(§12.2 "Pressure Test Result (AP50)") fell through to the structural-
infiltration default — over-counting ventilation heat loss.

Surfaced by simulated case 44 (AP50 4.50): effective air change rate was
0.81 vs the worksheet's 0.58 (+38% ventilation loss). The cascade already
supports `air_permeability_ap50` (preferred over AP4); this wires the read
end to end (extractor → ElmhurstSiteNotes → SapVentilation → cert_to_inputs).

Pinned against the case-44 P960 §2 at abs=1e-4: (18) infiltration 0.3417
(= 4.5/20 + 0.1167) and (25) Jan effective ach 0.5812. Worksheet harness
stays 47/47 0-raised.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 23:18:17 +00:00
Khalim Conn-Kowlessar
d4d2b222fc feat(conservatory): §6.1 fabric cascade (27/27a/28a + TFA/volume)
Wire the non-separated conservatory into the §3 heat-transmission +
§1 dimensions cascade per RdSAP 10 §6.1 (PDF p.49) + Table 25 (p.51):

  "The floor area and volume of a non-separated conservatory are added to
   the total floor area and volume of the dwelling. Its roof area is taken
   as its floor area divided by cos(20°), and wall area is taken as the
   product of its exposed perimeter and its height. ... The conservatory
   walls and roof are taken as fully glazed ... Glazed walls are taken as
   windows, glazed roof as rooflight."

New `worksheet/conservatory.py` derives the geometry:
  - height from the equivalent storey count (§6.1: 1 storey → ground-floor
    room height; 1½ → ground + 0.25 + 0.5×first; etc.);
  - glazed WALL → window (27) at Table 25 U (double 3.1 / single 4.8) with
    the §3.2 curtain resistance (R=0.04) → U_eff 2.758;
  - glazed ROOF → rooflight (27a) at Table 25 roof U (double 3.4 / single
    5.3) + curtain → U_eff 2.993;
  - FLOOR → (28a) via BS EN ISO 13370 as an uninsulated SOLID ground floor
    with 300 mm walls (§5.12, spec p.43), exposed perimeter = glazed
    perimeter → U 0.89;
  - glazed wall + roof + floor areas join (31)/(36); the fully-glazed
    structure walls/roof add nothing (the glazing IS the window/rooflight).

`dimensions_from_cert` adds the conservatory floor area to TFA (4) and
floor area × height to volume (5) (feeds ventilation (8)), without making
it a storey (avg storey height for §2 infiltration is unchanged).

Pinned against the simulated case-44 P960 §3 at abs=1e-4 — every line ref
EXACT: (4) 95.3800, (5) 257.1630, (27) 96.1169, (27a) 38.2201, (28a)
21.4164, (29a) 35.5852, (30) 7.4688, (31) 294.2900, (33) 207.3274,
(36) 23.5432. The remaining whole-dwelling SAP/CO2 gap is the §6 solar
gains, closed in the next slice. Worksheet harness stays 47/47 0-raised.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 15:59:26 +00:00
Khalim Conn-Kowlessar
fa131cca0b feat(conservatory): read §6.1 geometry through extractor + mapper
RdSAP 10 §6.1 (PDF p.49) models a non-separated (heated) conservatory as
part of the dwelling. Until now the Summary §5 block was reduced to an
inert `has_conservatory` bool and the geometry (floor area, glazed
perimeter, glazing, storey height) was dropped on both paths.

Plumbing only — no cascade consumer yet (Slices B/C/D wire §3/§6):
- ElmhurstSiteNotesExtractor reads the §5 Conservatory block into a new
  `Conservatory` site-notes record (scoped to §5 so the generic
  "Floor Area"/"Room Height" labels can't collide with §4 dimensions);
- domain gains a frozen `SapConservatory` (floor area, glazed perimeter,
  double/single glazing, thermally-separated guard, equivalent storey
  count) on `EpcPropertyData.sap_conservatory`;
- the Elmhurst mapper threads it through, dropping SEPARATED
  conservatories per §6.2 ("A separated conservatory ... is disregarded").

Verified against the simulated case-44 Summary (RefNo 001431): extracts
floor_area=12.0, glazed_perimeter=9.0, double_glazed=True, 1 storey.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 15:37:05 +00:00
Khalim Conn-Kowlessar
419e340477 test(worksheet): pin simulated case 43 at 1e-4 (RR + dry-line + mixed roof)
Golden regression fixture for the multi-feature dwelling that surfaced the
two Elmhurst-extractor bugs in a33707f8. case 43 is a 2-BP mid-terrace with
a DETAILED room-in-roof (two slopes, two flat ceilings, party + exposed
gables, two common walls), a MIXED-insulation multi-section roof (Main
insulated + Extension uninsulated 2.30), a DRY-LINED extension solid wall,
a mains-gas boiler (102 / control 2106) and a House-coal solid-fuel
secondary (633).

Routes the Summary PDF through the WHOLE extractor + mapper + calculator
pipeline (no hand-built EpcPropertyData) and pins the §3 fabric + SAP-rating
block at abs=1e-4: (29a) walls 74.5800, (30) roof 38.5008, (33) fabric
172.7844, continuous SAP 73.2332 = (258), CO2 3518.3037 = (272). Guards the
detailed-RR slope/common_wall surfaces, the dry-lining R=0.17 adjustment,
and the per-part mixed-roof billing together. Summary mirrored to
backend/documents_parser/tests/fixtures/Summary_001431_case43.pdf; provider
module mirrors the _case6/_case21 pattern, assertion in
test_section_cascade_pins. Harness 47/47; regression = the 3 pre-existing
fails; pyright net-zero.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 08:26:05 +00:00