Commit graph

2360 commits

Author SHA1 Message Date
Daniel Roth
f95b6bdd7d get_evidence_files_by_job_id returns DownloadedFiles with empty other when include_other=False 🟩 2026-06-04 15:40:25 +00:00
Daniel Roth
665dc69ad5 get_evidence_files_by_job_id returns DownloadedFiles with empty other when include_other=False 🟥 2026-06-04 15:40:25 +00:00
Daniel Roth
e7c679e0db Group evidence into core and other via _group_into_core_and_other_files 🟪 2026-06-04 15:40:25 +00:00
Daniel Roth
99229844b5 _select_other_files returns non-core evidence files 🟩 2026-06-04 15:40:25 +00:00
Daniel Roth
db796747d9 _select_other_files returns non-core evidence files 🟥 2026-06-04 15:40:25 +00:00
Daniel Roth
6cb6c8c756 allow for missing deal stage column when triggering sqs from file 2026-06-04 15:40:25 +00:00
Daniel Roth
790e430aff rename local handler trigger script 2026-06-04 15:40:25 +00:00
Jun-te Kim
dfd05ba28b tests files 2026-06-04 11:47:42 +00:00
Daniel Roth
df0c8f69b1 Merge branch 'main' into feature/handle-new-magicplan-response-structure 2026-06-04 10:52:32 +00:00
Daniel Roth
db2cbc9c57 SharePoint upload is skipped when client is None 🟩 2026-06-04 09:33:07 +00:00
Daniel Roth
416dac0512 SharePoint upload is skipped when client is None 🟥 2026-06-04 09:29:24 +00:00
Daniel Roth
eb72b0223f add ECO_SHAREPOINT_ID env var 2026-06-04 09:11:08 +00:00
Khalim Conn-Kowlessar
fe59c4d8a2 S0380.208: case 7 combi e2e fixture — condensing-oil-combi path validated exact
Adds simulated case 7: case 6 (P960-0001-001431) with the heating swapped
to a CONDENSING OIL COMBI (SAP code 130, Table 4b 82/73) and the cylinder
removed — combi instantaneous DHW (WHC 901), Table 3a keep-hot combi loss
(61) = 600 kWh/yr, no primary/storage loss, boiler interlock PRESENT (no
−5pp). This is the heating archetype golden cert 0240-0200-5706-2365-8010
uses, which case 6 (SAP code 127, a *regular* condensing oil boiler +
cylinder) never exercised.

The cascade reproduces the case-7 worksheet EXACTLY at abs=1e-4 on every
top-level SapResult output with ZERO calculator changes:
  (211) 7865.4304  (213) 7556.9821  (219) 3496.8121  (98c) 12646.3783
  (255) 1123.3372  (257) 1.9631     (272) 5738.9315  (258) 73
This validates the SAP 10.2 Appendix D Eq D1 combi efficiency blend +
Table 3a keep-hot combi loss + Table 4b code 130 (82/73) path, and
exonerates the combi mechanism as the source of 0240's API-path residual
— which therefore lives in 0240's fabric/demand or the API mapper.

Test-only slice (no impl change). New fixture file: 0 pyright errors.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 17:57:22 +00:00
Daniel Roth
174ef26075 refactor magicplan in ddd structure 2026-06-03 17:20:20 +00:00
Khalim Conn-Kowlessar
2b1afa7339 S0380.204: extract Main Heating2's own emitter + control (§14.1)
Prerequisite for the SAP 10.2 p.186 two-systems-different-parts MIT.
When two main systems heat different parts of a dwelling, §14.1 Main
Heating2 lodges its OWN "Heat Emitter" + "Main Heating Controls Sap"
(simulated case 6: Main 1 radiators / control 2106 serving the living
area, Main 2 underfloor / control 2110 serving elsewhere). The extractor
+ mapper dropped both — `MainHeatingDetail.heat_emitter_type` and
`main_heating_control` came through as empty-string sentinels, so the
cascade saw system 2 as having no responsiveness (defaulted R=1.0) and no
control type.

- `MainHeating2` datatype gains `heat_emitter` + `heating_controls_sap`.
- The extractor reads them from the §14.1 block.
- `_map_elmhurst_main_heating_2` maps them via the same helpers as Main 1
  (`_elmhurst_heat_emitter_int` → underfloor-in-screed = emitter 2, Table
  4d R=0.75; `_elmhurst_sap_control_code` → 2110, Table 4e type 3),
  threading the dwelling floor + age band for the underfloor subtype.

Empty-string fallback preserved for the legacy DHW-only Main 2 (cert
000565 §14.1 omits emitter/control). No cascade output changes yet — the
MIT consumer lands in S0380.205. Full suite 2358 pass + 0 fail.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 15:53:32 +00:00
Daniel Roth
d31d0683f5 get triggers working for abri address list 2026-06-03 14:23:08 +00:00
Khalim Conn-Kowlessar
2b1f90a7de S0380.199: site-notes "Roof of Room" windows → roof windows (cross-mapper parity with S0380.198)
The Elmhurst extractor crashed parsing simulated-case-6's room-in-roof
window rows: the §11 "Location" cell "Roof of Room in Roof" wraps across
the layout prefix/suffix blocks and leaked into the glazing-type phrase
("Double between 2002 Roof of Room and 2021 in Roof" → UnmappedElmhurst-
Label). Fix (`_parse_window_from_anchors`): detect the roof-of-room
location tokens, strip them from the before/after blocks so the glazing
phrase reconstructs cleanly, and set location="Roof of Room".

Mapper: `_is_elmhurst_roof_window` gains a "Roof of Room" location branch
(highest-confidence rooflight signal, above the BP-roof-type / U>3.0
gates); `_ELMHURST_ROOF_WINDOW_U_BY_GLAZING` gains "Double between 2002
and 2021" → 2.30 (case 6 lodges the already-inclined roof-window U, so
the +0.30 inclination adjustment must not double-apply).

This is the site-notes mirror of S0380.198 (API window_wall_type=4):
both paths now route room-in-roof rooflights to (27a) at the inclined U.
Validated against the case-6 P960 worksheet at abs=1e-4:
  (27)  Windows      = 22.7408 (cascade 22.7407)
  (27a) Roof Windows = 13.0375 (cascade 13.0375, EXACT)
  (31)  ext area     = 336.13

Case 6 is pinned only on the §3 window line refs (new standalone test,
not added to the section-pin `_FIXTURES`) because its DUAL main heating
(51% rads + 49% underfloor, oil) makes the §10/§12 per-system lines
non-comparable to SapResult's aggregated fields — documented in the
fixture module. Summary mirrored to Summary_001431_case6.pdf.

Suite: 2355 passed, 1 skipped. New code: 0 pyright errors.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 12:46:18 +00:00
Jun-te Kim
c614ff6388 save local changes 2026-06-03 12:41:56 +00:00
Khalim Conn-Kowlessar
570df83459 S0380.197: simulated case 5 e2e fixture — detached sandstone RR validates S0380.196 (RdSAP 10 §3.9.1 + Table 4 p.22)
Promotes user-simulated "case 5" (detached, sandstone-walled, room-in-roof
cousin of golden cert 0240) to an e2e worksheet fixture pinning the WHOLE
extractor → mapper → calculator pipeline at abs=1e-4 on all 11 Block-1
line refs. Its worksheet prints the exact RR-gable routing S0380.196
implements, validating that fix against ground truth:

  Roof room Main Gable Wall 1  15.68  U=0.35  (29a)  Exposed → walls @ main-wall U
  Roof room Main remaining area 61.73  U=0.30  (30)  A_RR shell − Σ gables
  External roof Main           14.52  U=0.11  (30)  loft residual
  Roof room Main Gable Wall 2  15.68  U=0.25  (32)  Party → party @ 0.25

gable area = 6.40 × 2.45 (§3.9.1 default RR storey height); A_RR remaining
= 12.5√(83.2/1.5) − 2×15.68 = 93.09 − 31.36 = 61.73 (RdSAP 10 §3.9.1(e)).
Confirms a DETACHED dwelling can lodge a Party RR gable (Table 4 p.22
row 2) — so my S0380.196 mapping (gable_wall_type 0=Party, 1=Exposed) is
correct; do not flip it.

Two extractor/mapper gaps surfaced and fixed (case 5 is the forcing test):
- Sandstone wall label "SS Stone: sandstone or limestone" had no
  `_ELMHURST_WALL_CODE_TO_SAP10` entry (raised UnmappedElmhurstLabel).
  Added "SS" → 2 (WALL_STONE_SANDSTONE), matching 0240's API
  wall_construction=2 (cross-mapper parity).
- Roof "Insulation Thickness 400+ mm" was silently dropped: the four
  thickness parsers used `.split()[0].isdigit()`, which rejects the
  trailing "+" → None → u_roof fell back to the age-J default 0.16
  instead of 0.11 (+1.09 W/K roof, the whole 0.12 SAP gap). Added
  `_parse_thickness_mm` (strips to leading digits) and applied it at all
  four sites (walls / alt-wall / roof / floor). The only existing fixture
  with "400+ mm" (000565 Stud Wall) routes via the RIR regex, unaffected.

Result: case 5 cascade ≡ worksheet at 1e-4 on SAP/ECF/cost/CO2 + every
energy stream. Neither gap affects 0240 (its API path captures both the
sandstone code and "400mm+"); 0240's residual is therefore non-fabric.

Suite: 2353 passed, 1 skipped. New code: 0 pyright errors.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 11:41:16 +00:00
Khalim Conn-Kowlessar
4a21717de6 S0380.195: pin sim case 4 (6035 floor geometry) e2e at 1e-4 — 6035 +19 PE is lodged divergence
Adds the user-simulated case-4 worksheet as e2e fixture `001431_6035` —
reproduces golden cert 6035's full floor geometry (Main ground-floor HLP
15.99 + first-floor HLP 8.32, the asymmetric upper storey) and 8 windows.
All 11 Block-1 line refs pin at abs=1e-4 against the worksheet (SAP 68,
ECF 2.2802, cost 937.2341, CO2 4682.3494, space 15745.3260, main fuel
18744.4357).

This is the 4th independent 1e-4 confirmation across the 6035 archetype
(sim cases 1-4). Case 4 matches 6035 on floors + window areas; the
residual ~50 kWh / £11 cascade delta vs 6035 is two lodged inputs only
(largest window orientation N vs S; meter type "Dual" vs API 2), not
calculator behaviour.

Conclusion: the cascade reproduces the spec engine exactly for 6035's
geometry, so 6035's +19 PE vs the lodged register is lodged-register
divergence (the gov.uk register's rounded value vs the spec-exact
worksheet), NOT a calculator gap. 6035 is a "pin-forever" lodged-only
cert. Bugs surfaced + fixed along the way: S0380.192 (Simplified-RR
remaining area) and S0380.193 (suspended-floor sealed rule).

2341 passed (+11), 0 failed; pyright net-zero.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 09:56:39 +00:00
Khalim Conn-Kowlessar
e7a0c9885e S0380.194: pin sim case 3 (near-exact 6035 replica) e2e at 1e-4
Adds the user-simulated case-3 worksheet as e2e fixture `001431_rr8` —
Main + Extension + Simplified room-in-roof with 8 windows (≈14.15 m²,
reproducing golden cert 6035's glazing) and Main ground-floor HLP 15.99.
All 11 Block-1 line refs pin at abs=1e-4 against the worksheet (SAP 68,
cost 951.3425, CO2 4767.4862, space 16086.3557, main fuel 19150.4235,
HW 3307.2639, lighting 262.0885).

This is the third independent 1e-4 confirmation that the cascade
reproduces the spec engine for the 6035 archetype (after S0380.192
Simplified-RR + S0380.193 suspended-floor). It differs from 6035 in one
input only — the Main first-floor HLP (15.99 here vs 6035's 8.32) — so
6035's +19 PE vs the lodged register is lodged-register divergence, not
a calculator gap. A byte-identical 6035 replica (first-floor HLP 8.32)
would let 6035 itself be pinned directly to close that out.

2330 passed (+11), 0 failed; pyright net-zero.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 09:46:56 +00:00
Khalim Conn-Kowlessar
62fc27a5cc S0380.193: suspended-floor (12) sealed rule fires only on a SUPPLIED U-value
RdSAP 10 §5 (PDF p.29) "Floor infiltration (suspended timber ground
floor only)", age band A-E, splits on whether a floor U-value is
supplied:
  a) [U-value supplied] if floor U-value < 0.5 → "sealed", (12) = 0.1
  b) [no U-value supplied] retro-fitted insulation → "sealed" 0.1;
     otherwise "unsealed", (12) = 0.2

`_has_suspended_timber_floor_per_spec` fed the cascade's COMPUTED default
U into rule (a), so an as-built/uninsulated suspended-timber floor whose
default U happens to be < 0.5 was marked "sealed" (0.1) where Elmhurst
uses "unsealed" (0.2). That dropped (18) infiltration 0.85 → 0.75, (25)
effective ACH, HTC, and understated space heating ~450 kWh.

Fix: gate rule (a) on `floor_u_value_known` — a computed default U is not
a supplied value, so it falls through to (b). Verified against the
cert 001431 sim-case-2 worksheet: floor "As built", U=0.43 (matches the
worksheet's (28a) 0.4300 exactly), (12)=0.2 unsealed. Golden cert 6035
(also a suspended uninsulated floor) is unaffected — its U=0.63 ≥ 0.5
already routed to unsealed.

Promotes sim case 2 to the e2e harness as `001431_rr` (Main + Extension
+ Simplified room-in-roof — the 6035 archetype). All 11 Block-1 line
refs pin at abs=1e-4, locking BOTH this fix and S0380.192 (Simplified-RR
remaining area) end-to-end: SAP 69, cost 920.5046, CO2 4566.7090, space
15269.8593, main fuel 18178.4039. 2319 passed (+11), 0 failed; pyright
net-zero.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 09:16:25 +00:00
Daniel Roth
19e40ff049 rename mcs_certificate to mcs_compliance_certificate to match existing db enum value 2026-06-03 09:00:01 +00:00
Daniel Roth
36f50c3bef Merge branch 'main' into pashub-fetcher-all-files 2026-06-03 08:57:31 +00:00
Daniel Roth
7984a6ded8 All downloaded PasHub files uploaded to SharePoint property folder 🟩 2026-06-03 08:52:30 +00:00
Daniel Roth
39fa1ccfa0 All downloaded PasHub files uploaded to SharePoint property folder 🟥 2026-06-03 08:49:37 +00:00
Daniel Roth
37ec409553 tweak local trigger 2026-06-03 08:21:54 +00:00
Khalim Conn-Kowlessar
ec9ef0e8bb fix(extractor): drop windows-table header remnant from first window glazing type
Summary PDFs preprocessed from `pdftotext -layout` wrap the windows-table
header across several lines. The third header line's tail ("U value / g
value / Draught Proofed / Permanent Shutters") tokenises to "value value
Proofed Shutters" and lands directly above the FIRST window's data row.

Because the first window in a building part has `before_start = 0`, its
prefix block reaches back into that header remnant. The remnant is
neither an orientation nor a building-part fragment, so it survived the
pops in `_compose_window_descriptors` and leaked into glazing_type as
"value value Proofed Shutters Double between 2002 and 2021" (windows 2-3,
whose prefix starts after the previous window's manufacturer line, were
clean).

Fix: the glazing-type phrase always starts with a glazing-start word
(Single/Double/Triple/Secondary), so trim any prefix fragments preceding
that word before joining the glazing type. Orientation/bp pops still run
on the full prefix, so they are unaffected.

Reproduced from `sap worksheets/Recommendations Elmhurst Files/
cavity_wall_insulation - main wall/before/Summary_001431.pdf`. Added a
regression test driving the real `_extract_windows_from_layout` path with
the verbatim tokenised header+rows. 2306 passed (+4), pyright net-zero.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 22:54:49 +00:00
Khalim Conn-Kowlessar
896b5740c3 S0380.191: pin simulated 001431 gas-combi end-to-end at 1e-4 (e2e harness)
Adds the user-simulated 001431 case (the cert that drove S0380.189/.190)
as an Elmhurst-only e2e fixture: Summary PDF → extractor → mapper →
calculator, every Block-1 SapResult field pinned against the
P960-0001-001431 worksheet at abs=1e-4. All 11 pins pass with zero
residual — the case is clean, confirming the S0380.190 gas-combi fuel
derivation closes the Summary path natively.

Verified the handover's flagged "+0.0007 SAP" was a target artifact, not
a cascade gap: the worksheet displays ECF (257) rounded to 1.6047 and
integer SAP (258)=78; the cascade's continuous SAP is computed from the
UNROUNDED ECF = (255)*(256)/((4)+45) = 660.9750*0.4200/173.0, giving
77.6147 — which matches the worksheet's own unrounded value. Pinning the
continuous SAP from the display-rounded ECF (→ 77.6144) was the wrong
target. Block-1 line refs all match exactly: (211) 10699.7225, (219)
3327.1592, (231) 86.0, (232) 283.2229, (255) 660.9750, (272) 3000.1664,
Σ(98) 8987.7669.

Summary mirrored into the tracked fixtures dir as
Summary_001431_gas_combi.pdf (distinct name — the corpus reuses cert
001431 across every heating variant); source Summary + worksheet tracked
under sap worksheets/golden fixture debugging/ as the pin ground truth.

2302 passed (+11), 0 failed; pyright net-zero on new/changed files.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 22:44:32 +00:00
Khalim Conn-Kowlessar
57241322ea S0380.185: record CH6 pin-forever proof — distribution-loss is a Summary-export gap
CH6's P960 worksheet input lodges Distribution Loss = "Two adjoining
dwellings sharing a single heating system" → (306) DLF = 1.0000, vs CH4's
"Calculated" → 1.5 → (306) = 1.4500. That DLF choice swings SAP/cost/CO2/PE
materially, but it is NOT present in the Summary PDF that the corpus pipeline
consumes (Summary → ElmhurstSiteNotesExtractor → mapper → calculator).

Proven empirically with a user-supplied controlled pair (CH adjoined
dwellings/Summary_001431 (1) vs (2)): the two Summaries are byte-identical
across every RdSAP INPUT field, differing only in the derived header
(SAP 80 vs 75, bill £954 vs £1237, emissions 5.407 vs 7.394 t). A
case-insensitive scan of the CH6 Summary for "distribution"/"adjoin" returns
0 hits. Since CH4/CH6 Summaries are themselves identical bar fuel type, no
Summary-derivable rule can yield CH4=1.45 AND CH6=1.0.

Doc-only change (comment in _EXPECTATIONS); 20/20 community-heating corpus
tests pass. Closes the CH6 re-litigation: pin held.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 19:21:28 +00:00
Khalim Conn-Kowlessar
82f7315f8d S0380.184: community electric-HP network CO2/PE uses monthly Table 12d/12e — closes CH3
SAP 10.2 worksheet block 12b/13b (367)/(467) for a community heating
electric heat pump (Table 4a code 304 → Table 12 fuel 41 "heat from
electric heat pump"). The HP meters grid electricity, so per Table 12
note (s)/(t) + block 12b/13b footnote (a) its emission/PE factor is the
MONTHLY Table 12d/12e cascade (fuel 41 = standard-electricity profile),
weighted by the network heat profile, then × 1/heat-source-eff (1/COP):

  (367)/(467) = [(307)+(310)] / COP × Σ((307+310)_m × factor_m)/Σ(...)

Per-line walk of CH3 (the displayed (367) 0.1535 / (467) 1.5717 are PDF
artifacts; the (373)/(473) totals reconcile only with):
  CO2 factor = 0.15040 (monthly Table 12d wtd) vs cascade annual 0.136
  PE  factor = 1.55692 (monthly Table 12e wtd) vs cascade annual 1.501

Pre-slice the cascade routed code 304 through the non-electric branch
(`_co2_factor_kg_per_kwh(main) × 1/COP` = annual × scaling). New
`_is_heat_network_electric_main` (heat-network main whose fuel has a
Table 12d monthly set — i.e. fuel 41) routes all four factor helpers
(main + HW, CO2 + PE) through the monthly cascade × 1/COP. Non-electric
heat networks (gas 51 / oil 53 / coal 54) have no monthly set → annual
path unchanged (CH1, CH6 untouched).

Closure (CH3 was already SAP+cost EXACT):
  CH3 (HP/Elec)  CO2 −75.32→+0.0000 (= [(307+310)/3]×(0.1504−0.136)),
                 PE −249.32→−0.0000 (× (1.5569−1.501))  — FULLY EXACT

Corpus now 40/41 EXACT on all four metrics. Only CH6 remains: its
worksheet lodges a manual DLF=1.0 ("two adjoining dwellings") absent
from the Summary PDF (byte-identical to CH4 bar fuel type) — an
architectural limit, not a cascade gap. 2226 pass + 1 skip + 0 fail
(tolerances 1e-4 all metrics); pyright net-zero 43→43.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 18:43:16 +00:00
Khalim Conn-Kowlessar
803da062a2 S0380.183: community-heating HW bills at heat-network rate (§10b) — closes CH2/CH4 fully
SAP 10.2 §10b: hot water for a community-heating dwelling bills at the
heat-network rate, not the cert-lodged fuel. Elmhurst §15.0 lodges
`water_heating_fuel_type = "Mains gas"` (3.48 p/kWh) as a placeholder on
community certs; the worksheet (342) Water-heating cost = (310) × the
S0380.171 CHP heat-fraction blend — the SAME rate as space heating (340).

Per-line walk of the CH2 block 10b:
  (340) space   = 11837.83 × 0.037955 = 449.3047  (cascade EXACT)
  (342) water   =  3854.12 × 0.037955 = 146.2830  (cascade billed
                  3854.12 × 0.0348 = 134.12 → −£12.16, the whole residual)
  (350) lighting + (351) standing → (355) 754.1502.

`_hot_water_fuel_cost_gbp_per_kwh`'s `inherit_main_for_community_heating`
path already routes HW cost through `_fuel_cost_gbp_per_kwh(main)` (the
CHP blend), but its gate `_is_community_heating_hw_from_main` excluded
code 302. S0380.182 wired the 302 CO2/PE credit via
`_heat_network_code_302_effective_factor`, which intercepts the HW
CO2/PE helpers ABOVE this predicate's branch — so extending the
predicate to include 302 now affects ONLY the cost path.

Closures:
  CH2 (CHP/Gas)  SAP +0.5277→−0.0000, cost −£12.16→−£0.00  — FULLY EXACT
  CH4 (CHP/Oil)  SAP +0.5277→−0.0000, cost −£12.16→−£0.00  — FULLY EXACT
  CH6 (CHP/Coal) SAP −7.49→−8.02, cost +£172.68→+£184.84 — its HW now
                 also bills the blend, compounding the DLF=1.0 quirk
                 (cascade DLF=1.45); same separate CH6 DLF front.

Corpus now 39 variants EXACT on all four metrics (CH2/CH4 join). Open:
CH3 CO2/PE (code-304 community-HP COP), CH6 all-metric (DLF=1.0 manual
override the Summary doesn't carry). 2225 pass + 1 skip + 0 fail
(tolerances 1e-4 all metrics); pyright net-zero 32→32.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 18:29:57 +00:00
Khalim Conn-Kowlessar
8e86de2257 S0380.182: community-heating CHP+boilers CO2/PE credit (§12b/13b) — closes CH2/CH4 CO2+PE
SAP 10.2 worksheet block 12b (CO2) / 13b (PE) for community heating
"CHP and boilers" (SAP code 302). Per unit of network heat fuel
H = (307)+(310) the effective generation factor is:

  chp×100/(362)×f_fuel − chp×(361)/(362)×f_disp + (1−chp)×100/(367)×f_fuel

  (363)/(463) CHP fuel      = chp_frac × 100/heat_eff × f_fuel
  (364)/(464) less credit   = −chp_frac × elec_eff/heat_eff × f_disp
  (368)/(468) boiler fuel   = (1−chp_frac) × 100/boiler_eff × f_fuel

f_fuel = Table 12 heat-network fuel factor (the CHP unit and the back-up
boilers burn the same community fuel — verified vs CH2 gas / CH4 oil /
CH6 coal worksheets (363)/(368)); f_disp = Table 12f (PDF p.196) credit
for the CHP-generated electricity. RdSAP 10 §C (p.58) defaults: heat eff
50% (362), electrical eff 25% (361), boiler eff 80% (367); CHP heat frac
0.35 per-cert via community_heating_chp_fraction.

New `_heat_network_code_302_effective_factor` + Table 12f flexible
constants (0.420 CO2 / 2.369 PE) + RdSAP §C efficiency constants, wired
into all four factor helpers (main + HW, CO2 + PE) ahead of the existing
single-fuel / 1-over-heat-source-eff path. The worksheet (368)/(468)
boiler emissions DISPLAY rounded/mis-aligned in the PDF, but the
(373)/(473)/(386)/(486) totals reconcile only with the boiler at the
full Table 12 factor — verified EXACT.

Two spec citations applied:
- Table 12f flexible-operation default for RdSAP community CHP is an
  Elmhurst engine choice (Table 12f notes make "standard" the default);
  mirrored per [[feedback-software-no-special-handling]] and documented
  in SAP_CALCULATOR.md §8.3.
- Table 12 heat-network oil/biodiesel CO2 (codes 53/56) corrected
  0.298 → 0.335 per Table 12 (p.189) "assumes 'gas oil'"; the code-302
  oil cascade (CH4) was the first to exercise it. PE 1.180 was already
  correct. No other variant uses these codes (no regression).

Closures (CO2 + PE only — the CHP credit does not touch cost/SAP):
  CH2 (CHP/Gas)  CO2 −1411.49→+0.0000, PE +1331.23→+0.0000  EXACT
  CH4 (CHP/Oil)  CO2 −4378.24→−0.0000, PE  +319.81→−0.0000  EXACT
  CH6 (CHP/Coal) CO2/PE re-pinned (+2411.54 / +5023.48) — its worksheet
                 lodges a manual DLF=1.0 the Summary doesn't carry, so
                 cascade DLF=1.45 over-scales H; same root as the CH6
                 SAP −7.49 / cost +£172 (separate DLF front).

CH2/CH4 are now CO2+PE-exact but still carry the heat-network cost/SAP
residual (+0.5277 SAP / −£12.16 cost, exposed by S0380.175 — cost-side,
untouched here). CH3 unchanged (code 304 community-HP COP front).

Corpus state: 37 variants EXACT on all four metrics (incl. CH1);
remaining residuals are CH2/CH4 cost+SAP, CH3 CO2+PE (HP COP), CH6
all-metric (DLF quirk). 2223 pass + 1 skip + 0 fail (tolerances 1e-4 all
metrics per S0380.181); pyright net-zero 43→43.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 18:23:17 +00:00
Khalim Conn-Kowlessar
02a89bcb39 S0380.181: tighten heat-systems corpus residual tolerances to 1e-4 (all metrics)
The corpus residual-pin tolerances had drifted looser than the comment
above them claimed ("pin at 1e-4 relative to lodged precision"): SAP was
1e-3, cost ±£0.01, CO2 ±0.1 kg, PE ±0.1 kWh. A ±0.1 kg CO2 band could
silently mask a ~0.09 kg drift on a variant we report as EXACT.

The worksheet pins are extracted from the P960 PDF text, which prints
4 d.p., so the hard residual floor is ~5e-5 (half a unit in the last
printed digit) regardless of cascade precision. 1e-4 sits just above
that floor. All 41 variants hold at uniform 1e-4 on continuous SAP,
cost, CO2 AND PE — confirming the 37 EXACT variants are genuinely exact
to PDF print-rounding and the looser bands were masking nothing.

Aligns the guard with [[feedback-zero-error-strict]] /
[[feedback-continuous-sap-tolerance]] (basically zero error across all
four metrics). Test-only change; no cascade behaviour touched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 18:11:28 +00:00
Khalim Conn-Kowlessar
8452cf9e2d S0380.180: heat-network distribution pumping electricity (§C3.2) — closes CH1
SAP 10.2 Appendix C §C3.2 (PDF p.51), verbatim: "CO2 emissions and
Primary Energy associated with the electricity used for pumping water
through the distribution system are allowed for by adding electrical
energy equal to 1% of the energy required for space and water heating."

Worksheet line (313) = 0.01 × [(307)+(310)]; its CO2 (372) and PE (472)
bill on the Table 12d/12e monthly factors for fuel code 50 ("electricity
for pumping in distribution network"), weighted by the monthly heat
profile per worksheet footnote (a). (307)m/(310)m = (space_demand +
hw_output) / efficiency (the cascade models a heat network's generator
efficiency as 1/DLF).

This un-defers the (372)/(472) front the post-S0380.179 handover flagged
"don't guess until the factor source is identified": the source is
§C3.2 + Table 12d/12e code 50, NOT an empirical constant. The apparent
0.1994/0.2114 "factor" is an Elmhurst DISPLAY artifact — the worksheet
shows the (372) energy column as 0.01×(307) (space only) while computing
emissions on 0.01×(307+310) per the §C3.2 text. Verified EXACT line-by-
line against the CH2 corpus worksheet: (372)=23.6007 CO2 (rating),
(472)=208.2267 PE (demand).

New `_heat_network_distribution_electricity` helper (gated on
`_is_heat_network_main`) precomputes the energy + effective CO2/PE
factors; three new CalculatorInputs fields + calculator.py CO2/PE
summation terms (0.0/None → no-op for individually-heated certs).

Closures:
  CH1 (Boilers/Gas)  CO2 −23.60→−0.00, PE −208.23→+0.00  — FULLY EXACT
  CH3 (HP/Elec)      CO2 −98.92→−75.32, PE −457.54→−249.32 (distribution
                     component closed; code-304 community-HP COP remains)
  CH2/CH4/CH6        gain their (372)/(472) component (CO2 +23.6, PE
                     +208.2); dominant CHP displaced-electricity credit
                     residual (Table 12f + block 12b/13b) is next slice.

No regression on the other 36 corpus variants (helper returns None off
heat-network mains) + golden + U985 fixtures. 2223 pass + 1 skip + 0
fail; pyright net-zero 43→43.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 18:04:16 +00:00
Khalim Conn-Kowlessar
d7d5084f90 Move sap10_calculator tests to tests/domain/sap10_calculator/ for CI
The calculator tests lived under domain/sap10_calculator/{tests,worksheet/
tests,rdsap/tests,climate/tests,validation/tests}, none of which are in
pytest.ini testpaths — so CI (which collects tests/) never ran them. Relocate
all five dirs to tests/domain/sap10_calculator/{,worksheet,rdsap,climate,
validation}, mirroring the tests/domain/property_baseline/ convention, so the
cascade-pin / golden / e2e conformance suites run in CI.

Mechanics:
- git mv preserves history (110 files).
- Flattening the trailing /tests keeps each file's depth-to-repo-root
  identical, so all 16 repo-root parents[4] fixture refs stay valid. Only
  test_pcdb_etl.py's parents[1] (→ pcdb data) and one hardcoded absolute
  golden-fixture path in test_cert_to_inputs.py needed rebasing.
- Cross-imports rewritten domain.sap10_calculator.worksheet.tests →
  tests.domain.sap10_calculator.worksheet (21 files incl. the external
  importer backend/documents_parser/tests/test_summary_pdf_mapper_chain.py).
- Golden-fixture path strings in test_summary_pdf_mapper_chain.py +
  scripts/fetch_cohort2_api_jsons.py updated to the new location (the JSONs
  moved with the rdsap tests).

load_cells / gitignored worksheet xlsx: the xlsx-pinned tests (test_dimensions
/ ventilation / water_heating) read 2026-05-19-17-18 RdSap10Worksheet.xlsx,
which is gitignored (.gitignore `*.xlsx`) and so absent in CI. _xlsx_loader.
load_cells now pytest.skip()s when the file is absent, so those tests run
locally and skip cleanly in CI instead of erroring — no new CI failures from
the move, and the gitignore policy is respected.

Verified: tests/domain/sap10_calculator + backend/documents_parser +
tests/domain/property_baseline = 2248 pass, 1 skipped; pyright resolves the
new import paths with zero import-resolution errors.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 16:58:00 +00:00
Khalim Conn-Kowlessar
69995edec8 Merge branch 'main' of https://github.com/Hestia-Homes/Model into feature/per-cert-mapper-validation 2026-06-02 16:10:41 +00:00
Jun-te Kim
4e02eb7c77 more tests to ensure we don't deploy something that is brokern 2026-06-02 15:03:20 +00:00
Khalim Conn-Kowlessar
f2062a2fbe Slice S0380.179: RdSAP §10.7 electric-immersion default for no-system certs
Closes the "no system" corpus variant fully (ΔSAP +1.18 → <1e-4 on all
four metrics).

The cert lodges §15.0 "Water Heating Code: NON / SapCode 999" and §15.1
"Hot Water Cylinder Present: No". Per RdSAP 10 §10.7 (PDF p.55) "No
water heating system" verbatim: "the calculation is done for an
electric immersion heater. If the electric meter is dual the immersion
heater is also dual, but is a single immersion otherwise... for a
cylinder defined by the first row of Table 28 (110 litres) and the
first row of Table 29." Table 29 row 1 gives age-band cylinder
insulation (age G -> 25 mm foam) and assumes a cylinder thermostat
present for immersion-heated DHW.

The BRE-approved Elmhurst engine confirms the substitution: the P960
worksheet header lodges "WHS: 903 Electric immersion, Single", a 110 L
cylinder, and storage loss (56) = 594.32 kWh/yr, so HW (64) = (45)
1935.37 + 594.32 = 2529.6927.

Pre-slice the cascade trusted the lodged "no cylinder" -> added no
storage loss and a spurious Table 3a keep-hot combi loss; the wrong HW
heat-gains also propagated through §5/§7, over-stating the base MIT by
+0.25 K and space fuel by +228 kWh. New
`_apply_rdsap_no_water_heating_system_default(epc)` rebinds the epc at
the top of cert_to_inputs (the demand cascade delegates here too) when
water_heating_code == 999, injecting WHC 903 + electricity fuel +
110 L cylinder + Table 29 insulation + assumed cylinder thermostat.
This closes HW fuel AND the downstream space residual in one move.

Age bands A-F (12 mm loose jacket) raise UnmappedSapCode — no corpus
member exercises that and the Table 2 loss-factor dispatch only has the
factory-foam path plumbed. Gate is keyed on code 999, unique to "no
system" in the corpus; 40 other variants + 858 section pins + 6 U985
fixtures unchanged. 936 pass; pyright net-zero 32 -> 32.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 14:44:45 +00:00
Daniel Roth
223c1a259b adjust comment about evidence type 2026-06-02 14:32:13 +00:00
Daniel Roth
1a1804399e Service persists mcs_certificate file_type when evidence_category is MCS cert 🟩 2026-06-02 14:29:52 +00:00
Daniel Roth
bdc35573b5 Downloaded files carry evidence_category as DownloadedFile 🟩 2026-06-02 14:27:54 +00:00
Daniel Roth
d68cbb44f8 Downloaded files carry evidence_category as DownloadedFile 🟥 2026-06-02 14:24:24 +00:00
Daniel Roth
98eccda0ba MCS cert identified by evidence_category in get_core_file_type 🟩 2026-06-02 14:21:58 +00:00
Daniel Roth
ca224515f9 MCS cert identified by evidence_category in get_core_file_type 🟥 2026-06-02 14:20:35 +00:00
Khalim Conn-Kowlessar
c054d71284 Slice S0380.178: oil 6 circulation pump x1.3 for absent room thermostat
Closes the residual S0380.177 exposed on oil 6. The cascade's central
heating pump used the bare Table 4f age default (41 kWh for "2013 or
later") but the worksheet (230c) = 53.3 kWh.

SAP 10.2 Table 4f (PDF p.175) footnote a) on the "Circulation pump"
rows reads verbatim: "Multiply by a factor of 1.3 if room thermostat
is absent." oil 6 lodges control code 2101 ("No time or thermostatic
control of room temperature") = no room thermostat, so 41 x 1.3 = 53.3
= ws (230c) EXACTLY; pumps/fans (231) = 53.3 + 100 (liquid-fuel boiler
flue fan/pump) = 153.3 EXACT. Same root cause (absent room thermostat)
as the S0380.177 Table 4c(2) interlock fix — both keyed on the new
`_BOILER_NO_ROOM_THERMOSTAT_CONTROL_CODES = {2101, 2102}`.

`_table_4f_circulation_pump_kwh` now multiplies the resolved pump kWh
by `_TABLE_4F_NO_ROOM_THERMOSTAT_PUMP_MULTIPLIER = 1.3` when the main's
control code is in that set.

oil 6 now FULLY EXACT on all four metrics (ΔSAP/cost/CO2/PE < 1e-4).
The sibling oil 5 (same "2013 or later" pump age but control 2106 WITH
a room thermostat) keeps the bare 41 kWh and is unaffected — as do the
other 39 corpus variants (2101/2102 appear only on oil 6). 935 pass;
pyright net-zero 32 -> 32.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 14:20:18 +00:00
Khalim Conn-Kowlessar
5276282d8c Slice S0380.177: oil 6 boiler interlock from room thermostat absence
oil 6 (B30K standard liquid-fuel boiler, Table 4b code 126 winter 80 /
summer 68) lodges Main Heating Controls Sap code 2101 ("No time or
thermostatic control of room temperature") WITH a cylinder thermostat.
The cascade's `no_interlock` gate only checked the cylinder thermostat,
so oil 6 kept raw efficiency despite the P960 worksheet header lodging
"Boiler Interlock: No".

Per RdSAP 10 §3 (PDF p.57): boiler interlock is "assumed present if
there is a room thermostat and (for stored hot water systems heated by
the boiler) a cylinder thermostat. Otherwise not interlocked." Control
code 2101 (and 2102 "Programmer, no room thermostat") provides no room
thermostat — the two Table 4e Group 1 rows carrying the "+0.6 °C /
Table 4c(2)" annotation — so the boiler is NOT interlocked regardless
of the cylinderstat. SAP 10.2 Table 4c(2) (PDF p.169) "No thermostatic
control of room temperature – regular boiler" then deducts 5pp from
BOTH the Space and DHW seasonal efficiency.

Three changes in cert_to_inputs.py:
- new `_BOILER_NO_ROOM_THERMOSTAT_CONTROL_CODES = {2101, 2102}`;
- `no_interlock` now ORs room-thermostat absence with the existing
  stored-HW cylinderstat-absence test (the RdSAP §3 conjunction);
- the Space -5pp leg fires for Table 4b non-PCDB boilers (code
  101-141), not only PCDB-record boilers; the DHW leg is gated on a
  cylinder being present (Table 4c(2) combi DHW = 0).

Result for oil 6: space fuel (211) = 13446.3457 EXACT, HW fuel (219) =
4099.5872 EXACT. ΔSAP +3.0518 → +0.0782, Δcost -£69.79 → -£1.68,
ΔCO2 -240.66 → -1.71, ΔPE -1112.66 → -18.61.

The spec-correct fix exposes a single residual cause (per
[[feedback-software-no-special-handling]]): the central heating pump
(230c) — cascade reads pump_age=2 → Table 4f 41 kWh but ws (230c) =
53.3 kWh. The 12.3 kWh gap fully accounts for the residual across all
three metrics; pinned as the S0380.178 forcing function.

All other 40 corpus variants + 858 section pins + 6 U985 fixtures
unchanged (2101/2102 boiler codes appear only on oil 6). Pyright
net-zero.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 14:13:51 +00:00
Khalim Conn-Kowlessar
326066ee12 Slice S0380.176: Table 4b combi sub-row dispatch for (61)m
SAP 10.2 §4 line 7702 (PDF p.137) defines (61)m as "Combi loss for
each month from Table 3a, 3b or 3c (enter '0' if not a combi
boiler)". Table 4b sub-rows 128 / 129 / 130 are explicit combi sub-
rows per the spec row names:
    128: Combi oil boiler, pre-1998
    129: Combi oil boiler, 1998 or later
    130: Condensing combi oil boiler

Pre-slice `_table_3a_combi_loss_default_applies` gated only on
`main_heating_category ∈ {1, 2, 3, 6}`. The Elmhurst mapper leaves
`main_heating_category=None` on Table 4b liquid-fuel boilers (FAME,
HVO, B30K) — the cascade fell through to (61)m=0 despite the lodged
SAP code being a combi sub-row, under-counting (62)m by 600 kWh/yr
for FAME combi certs.

Extended the helper with a `_TABLE_4B_COMBI_OR_CPSU_CODES` fall-
through (set already exists for the symmetric `_primary_loss_
applies` Table 4b non-combi branch — see S0380.146). The set carries
the canonical combi + CPSU sub-row codes (103/104/107/108/112/113/
118/120-123/128-130). For cylinder-lodged certs the existing
`if epc.has_hot_water_cylinder: combi_loss_override = zero_monthly`
guard in `_water_heating_worksheet_and_gains` still pre-empts the
combi-loss fall-through correctly — non-combi codes with cylinders
remain (61)m=0.

Closures (heating-systems corpus 001431):
  oil 3 (code 128, FAME, no cylinder) ALL EXACT (±0.0000):
    ΔSAP_c +2.5863 → -0.0000
    Δcost  -£61.89 → -£0.00
    ΔCO2   -14.58  → +0.00
    ΔPE    -967.10 → +0.00
  oil 4 (code 129, FAME, no cylinder) ALL EXACT (±0.0000):
    ΔSAP_c +2.5603 → +0.0000
    Δcost  -£56.66 → +£0.00
    ΔCO2   -13.35  → +0.00
    ΔPE    -884.90 → +0.00

Oil 6 (code 126, NOT a combi, with cylinder) unchanged — the fix
is gated on the combi sub-row set. Cohort moves from 9 pinned
residuals to 7.

933 pass + 0 fail (+1 new mapper test). Pyright net-zero on cert_
to_inputs.py + tests.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-02 13:36:17 +00:00
Khalim Conn-Kowlessar
eda07d12dc Slice S0380.175: Community heating main_heating_control extraction
SAP 10.2 Table 4e Group 3 (PDF p.173) — heat-network control codes
2301-2314 dispatch to control_type 1, 2, or 3. Code 2306 = "Charging
system linked to use of heating, programmer and TRVs" →
control_type=3, temperature_adjustment=0. Per Table 9 the elsewhere-
zone off-hours depend on control_type: type 1/2 → (7, 8); type 3 →
(9, 8). The two extra off-hours change the §7 (90) T_rest mean by
~0.6 K → (92) MIT by ~0.4 K → (98) SH demand by ~390 kWh/yr.

Pre-slice diagnosis: cascade defaulted `main_heating_control=2`
(modal RdSAP) when the §14.0 "Main Heating Controls Sap" field was
empty. The 5 community heating corpus variants ALL lodge the SAP
code in §14.1 Community Heating "Heating Controls SAP" instead
(format: bare 4-digit integer, e.g. "2306"). The extractor was
storing this in `CommunityHeating.heating_controls_sap` but the
mapper only read `mh.heating_controls_sap` (§14.0).

Two changes:

1. `_elmhurst_sap_control_code` extended to accept bare 4-digit form
   ("2306") in addition to the §14.0 narrative form ("SAP code 2106,
   Programmer, room thermostat and TRVs"). Empty-string returns None
   instead of swallowing through the original `re.match` regex.

2. `_map_elmhurst_sap_heating` falls through to
   `mh.community_heating.heating_controls_sap` when the §14.0 main
   block leaves `heating_controls_sap` empty.

Closures (heating-systems corpus 001431):
  CH1 ΔSAP_c -1.0572 → +0.0000  EXACT
      Δcost  +£24.36 → -£0.00   EXACT
  CH3 ΔSAP_c -1.0572 → +0.0000  EXACT
      Δcost  +£24.36 → -£0.00   EXACT
  CH2/CH4 SAP-side flip ±0.42 → ±0.53 (CHP-split blend reacts to
        the now-lower SH demand × CHP rate)
  CH6 ΔSAP_c -8.4406 → -7.4942 (DLF=1.0 P960 quirk untouched)

Remaining CH1/CH3 ΔCO2 -23.60 / ΔPE -208.23 is the §13a (372)
"Electrical energy for heat distribution" line (118.38 kWh × electric
factors 0.1993 CO2 / 1.760 PE). Cascade doesn't currently meter this
electricity overhead separately from heat-network heat — next slice.

932 pass + 0 fail (+5 new mapper tests). No regressions on the other
36 corpus variants — the mapper change is gated on `mh.community_
heating is not None` and only fires when §14.0 leaves the control
field empty. Pyright net-zero on mapper.py + corpus test.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-02 13:07:59 +00:00
Daniel Roth
5df874b978 evidence categories plus typehinting 2026-06-02 12:54:12 +00:00