Commit graph

6152 commits

Author SHA1 Message Date
Khalim Conn-Kowlessar
8507728237 Slice S0380.157: SAP 10.2 Table 2b note b) WHC=903 electric-immersion guard
SAP 10.2 Table 2b note b) (PDF p.159) — verbatim:

  Multiply Temperature Factor by 0.9 if there is separate time
  control of domestic hot water (boiler systems, warm air systems
  and heat pump systems).

The parenthetical list restricts the rule to systems where the heat
generator (boiler / warm-air / HP) is the device heating the
cylinder. Electric immersion is NOT in that list because the
immersion isn't a heat-generator system feeding DHW — it sits inside
the cylinder. The ×0.9 multiplier reflects shorter cylinder-heating
periods when a boiler / HP / warm-air operates on a separate timer
for DHW vs SH; if the heat generator doesn't feed the cylinder at all
(because the immersion does), there's no such timing effect.

Pre-slice `_separately_timed_dhw` returned True for any Cat 4 HP
main BEFORE consulting WHC (line 3872 `if main.main_heating_category
== 4: return True`). For electric 2 (sap_main_heating_code=524 Cat 5
warm-air ASHP, main_heating_category=4 per Elmhurst mapper, WHC=903
electric immersion + cylinder + cylinder thermostat lodged), the
cat-4 branch fired before the existing `_is_electric_water` check
could route the cert to False. The cascade applied ×0.9 to the
Temperature Factor (53), pulling (55) from 1.2294 → 1.1064 → cascade
annual (56) = 403.87 vs worksheet (56) annual = 448.73.

Same WHC=903 principle as the prior slice S0380.156 (Table 3 zero-
loss list for electric immersion): when HW is independent of the
main heating, main-heating-specific DHW rules don't apply — even
when the main happens to be a HP / boiler / warm-air system.

Fix: new top-of-function `if epc.sap_heating.water_heating_code ==
_WHC_ELECTRIC_IMMERSION: return False` guard in
`_separately_timed_dhw`. Reuses the constant introduced in S0380.156.

Closures electric 2:
  Cylinder (56) storage loss annual 403.87 → 448.73 (matches
  worksheet 1.2294 × 365 = 448.73 EXACT within rounding)
  HW kWh demand 2339.24 → 2384.12 (matches worksheet (62)/(64) =
  2384.116 EXACT)
  ΔSAP +0.8118 → +0.7002
  Δcost −£18.71 → −£16.14
  ΔCO2 −7.21 → −2.37 kg
  ΔPE −161.68 → −108.58 kWh

The remaining +0.70 SAP residual is a separate upstream gap (likely
warm-air-HP SH cascade or Table 4a SH efficiency for code 524) —
follow-up slice.

No regressions on the other 24 cohort variants. Cohort-1 ASHP certs
(Cat 4 HP + WHC=901 = HW from HP + cylinder) keep ×0.9 as before
because their WHC=901 doesn't trigger the new guard. Extended
handover suite: 901 pass / 0 fail (was 900 — +1 from the new AAA
test). Pyright net-zero (43 → 43).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-04 15:40:25 +00:00
Khalim Conn-Kowlessar
c727b3cb5a Slice S0380.156: SAP 10.2 Table 3 WHC=903 electric-immersion zero-loss guard
SAP 10.2 Table 3 (PDF p.160) verbatim:

  Primary loss is set to zero for the following:
      Electric immersion heater
      Combi boiler ...
      CPSU ...
      Boiler and thermal store within a single casing
      Separate boiler and thermal store connected by no more than 1.5
      m of insulated pipework
      Direct-acting electric boiler
      Heat pump (...) with hot water vessel integral to package

The Elmhurst WHC=903 lodging signals exactly the first row: "HW from
a separate electric immersion heater" — the cylinder is heated by an
immersion element inside the tank, no primary pipework between any
heat generator and the cylinder. The rule is universal: regardless
of what main heating exists for space heating, electric immersion
means no primary circuit means no primary loss.

Pre-slice `_primary_loss_applies` only consulted `water_heating_code`
in the Table 4a wet-boiler branch (codes 151-161 / 191-196). The Cat
4 HP branch returned True unconditionally when no PCDB record was
lodged; the Cat 1/2 boiler branch returned True unconditionally; the
PCDB Table 322 + Table 4b non-PCDB branches likewise. For the
electric 2 corpus variant (sap_main_heating_code=524 Cat 5 warm-air
ASHP, main_heating_category=4 per Elmhurst mapper, no PCDB record,
WHC=903 + cylinder), the Cat-4 branch falsely returned True and the
cascade added ~510 kWh/yr primary loss to a system with no primary
circuit at all.

Per-line walk discipline applied: cascade `water_heating_from_cert`
output dump showed `primary_loss_monthly_kwh_annual = 509.98` while
worksheet (59)m = 0 every month → spec lookup found Table 3 verbatim
"Electric immersion heater" zero-loss line.

Adds `_WHC_ELECTRIC_IMMERSION: Final[int] = 903` constant + a
top-of-function `if water_heating_code == _WHC_ELECTRIC_IMMERSION:
return False` guard that fires before any of the system-type-keyed
branches.

Closures electric 2:
  HW kWh 2849.22 → 2339.24 (matches worksheet (62)/(64) = 2384.12
  within the residual ~45 kWh storage-loss gap)
  ΔSAP −0.4584 → +0.8118 (cascade swung past the worksheet by +1.27
  — the pre-slice 'near-correct' value was offsetting cascade bugs
  per [[feedback-software-no-special-handling]]; the +0.81 residual
  exposes a separate upstream gap to chase in a follow-up slice)
  Δcost +£10.56 → −£18.71
  ΔCO2 +47.89 → −7.21 kg
  ΔPE +443.13 → −161.68 kWh

No regressions on the other 24 cohort variants — only electric 2 has
the (Cat 4 HP, no PCDB, WHC=903) combination in the corpus.
Extended handover suite: 900 pass / 0 fail (was 899 — +1 from the
new AAA test). Pyright net-zero (43 → 43).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-04 15:40:25 +00:00
Khalim Conn-Kowlessar
f4a759ade3 docs: handover post S0380.153..155
Session landed three spec-clean slices closing four major residuals:

- S0380.153 (Table 3 middle row for solid-fuel boilers): SF3 EXACT all
  4 metrics (+0.30 → -0.0000). Found the rule that solid-fuel boilers
  don't ship with dual programmers per §9.2.4.

- S0380.154 (§12.4.4 back-boiler summer-immersion): SF2 SAP+cost
  EXACT (+1.86 → -0.0000 SAP; -£42.84 → -£0.00 cost). Implemented HW
  fuel kWh split + monthly blend across cost / CO2 / PE / standing.

- S0380.155 (Table 4a HP water-column dispatch): gshp closed ±0.02
  SAP (+0.94 → -0.0178). HW kWh 841 → 1138 matches worksheet exactly.

Σ |ΔSAP_c| 14.5 (session start of S0380.150) → 2.7 = 81% reduction
across 6 slices, two sessions.

Handover doc captures:
- Per-line discipline (walk worksheet before forming hypothesis)
- Elmhurst-vs-spec divergences to defer (lighting-PE +48.66 cluster
  uses Table 12 annual factor; spec Table 12d mandates monthly)
- Ranked open fronts (electric 5 R=0.20 storage MIT, electric 2
  warm-air HP HW, deferred lighting-PE cluster)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-04 15:40:25 +00:00
Daniel Roth
a68ab01d70 All downloaded PasHub files uploaded to SharePoint property folder 🟩 2026-06-04 15:40:25 +00:00
Daniel Roth
3d2a21d9d9 All downloaded PasHub files uploaded to SharePoint property folder 🟥 2026-06-04 15:40:25 +00:00
Daniel Roth
815173bc79 tweak local trigger 2026-06-04 15:40:25 +00:00
Daniel Roth
35b65d0d8d adjust comment about evidence type 2026-06-04 15:40:25 +00:00
Daniel Roth
5fa6f32644 Service persists mcs_certificate file_type when evidence_category is MCS cert 🟩 2026-06-04 15:40:25 +00:00
Daniel Roth
367f6f6cf9 Downloaded files carry evidence_category as DownloadedFile 🟩 2026-06-04 15:40:25 +00:00
Daniel Roth
1e2b997f2f Downloaded files carry evidence_category as DownloadedFile 🟥 2026-06-04 15:40:25 +00:00
Daniel Roth
e18c300deb MCS cert identified by evidence_category in get_core_file_type 🟩 2026-06-04 15:40:25 +00:00
Daniel Roth
1af1d4a82c MCS cert identified by evidence_category in get_core_file_type 🟥 2026-06-04 15:40:25 +00:00
Daniel Roth
5ed3bf73e8 evidence categories plus typehinting 2026-06-04 15:40:25 +00:00
Daniel Roth
020a24d345 run() returns core and other file paths 🟩 2026-06-04 15:40:25 +00:00
Daniel Roth
d8ec12065f run() returns core and other file paths 🟥 2026-06-04 15:40:25 +00:00
Daniel Roth
0aa6a4fc30 Other files persisted to DB with file_type OTHER 🟩 2026-06-04 15:40:25 +00:00
Daniel Roth
1650762ae2 Other files persisted to DB with file_type OTHER 🟥 2026-06-04 15:40:25 +00:00
Daniel Roth
c86dbeb4a1 Upload other files to S3 when get_other_files is True 🟩 2026-06-04 15:40:25 +00:00
Daniel Roth
098f60ecfd Upload other files to S3 when get_other_files is True 🟥 2026-06-04 15:40:25 +00:00
Daniel Roth
9c38f45c98 tidying for readability 2026-06-04 15:40:25 +00:00
Daniel Roth
c9a2ce4921 Service deletes other-file temp paths after run 🟩 2026-06-04 15:40:25 +00:00
Daniel Roth
f8d2bb8049 Service deletes other-file temp paths after run 🟥 2026-06-04 15:40:25 +00:00
Daniel Roth
49e7b7fea6 Wire service to get_evidence_files_by_job_id; retire get_core_evidence_files_by_job_id 🟪 2026-06-04 15:40:25 +00:00
Daniel Roth
662f6de0ab get_evidence_files_by_job_id downloads other files when include_other=True 🟩 2026-06-04 15:40:25 +00:00
Daniel Roth
c4ffaaa069 get_evidence_files_by_job_id downloads other files when include_other=True 🟥 2026-06-04 15:40:25 +00:00
Daniel Roth
f95b6bdd7d get_evidence_files_by_job_id returns DownloadedFiles with empty other when include_other=False 🟩 2026-06-04 15:40:25 +00:00
Daniel Roth
665dc69ad5 get_evidence_files_by_job_id returns DownloadedFiles with empty other when include_other=False 🟥 2026-06-04 15:40:25 +00:00
Daniel Roth
e7c679e0db Group evidence into core and other via _group_into_core_and_other_files 🟪 2026-06-04 15:40:25 +00:00
Daniel Roth
99229844b5 _select_other_files returns non-core evidence files 🟩 2026-06-04 15:40:25 +00:00
Daniel Roth
db796747d9 _select_other_files returns non-core evidence files 🟥 2026-06-04 15:40:25 +00:00
Daniel Roth
6cb6c8c756 allow for missing deal stage column when triggering sqs from file 2026-06-04 15:40:25 +00:00
Daniel Roth
790e430aff rename local handler trigger script 2026-06-04 15:40:25 +00:00
Khalim Conn-Kowlessar
69fdbf9f1d S0380.223: complete _part_geometry early-return key contract (RR KeyError)
5 certs in a 2026 API sample raised `KeyError: 'rr_common_wall_area_m2'`
and were blocked from computing. Root cause: `_part_geometry`'s early
return (taken when a building part lodges no sap_floor_dimensions —
e.g. a party-wall-only or RR-only extension as bp[0]) returned only 6 of
the 9 keys the full return exposes, omitting rr_common_wall_area_m2,
rr_gable_area_m2 and cantilever_floor_area_m2. The §3.9 RR contribution
block reads geom["rr_common_wall_area_m2"] / ["rr_gable_area_m2"] for
EVERY part, so the floorless part's truncated dict raised KeyError at
heat_transmission.py:974.

Fix: the early return now exposes all 9 keys, the three RR/cantilever
geometry values defaulting to 0.0 — correct, since a part with no floor
dimensions has no derivable RR shell or cantilever (no floor area).
Pure contract-completion bug; no spec/U-value change.

Regression test pins the invariant directly: a floorless part's
_part_geometry keys must equal a with-floors part's keys. Validated: all
5 certs now compute (4 within ~2 SAP of lodged; the 5th, 8536, has a
separate residual). §4 suite 2393 passed; heat_transmission.py pyright
unchanged at 12, test file at 71.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 15:33:52 +00:00
Jun-te Kim
a4670f8bc0 deploy lambda 2026-06-04 15:32:51 +00:00
Khalim Conn-Kowlessar
28634e8ae5 S0380.222: map API roof_construction codes 6 (thatched) + 7 (dwelling
above) → None

The 2026 sample lodges roof_construction=6 (1 cert, "Thatched, with
additional insulation") and =7 (6 certs, "(same dwelling above)" /
"(another dwelling above)"), both raising UnmappedApiCode and blocking
the cert. roof_construction_type is read ONLY for the §3 "sloping
ceiling" cos(30°) inclined-surface factor (Slice 89); the base roof
U-value comes from the global roofs[].description. Neither code is a
sloping ceiling:
  - 6 = thatched — U set by the description, not this field;
  - 7 = same/another dwelling above — an internal ceiling with no roof
    heat loss (the roof-side analogue of floor_construction code 0,
    governed by the roof_heat_loss / description path).

Map both to None: carries no information the cascade consumes here and
correctly avoids the cos(30°) false-trigger. Empirically inert and
validated — roof W/K is byte-identical whether 6/7 map to None or to an
explicit pitched string across all code-6/7 certs in the sample. 5 of
the 7 now compute (e.g. thatched cert 2276 SAP 62.8 vs lodged 63); the
other 2 also carry a gable_wall_type 2/3 raise (separate, worksheet-
backed slice).

Dict value type widened to Optional[str]. §4 suite 2392 passed; mapper.py
pyright unchanged at 32; new tests suppress reportPrivateUsage (net-zero).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 15:16:49 +00:00
Jun-te Kim
384e3da92a
Merge pull request #1172 from Hestia-Homes/feature/landlord_data
Feature/landlord data
2026-06-04 16:14:39 +01:00
Daniel Roth
6e986b76cc
Merge pull request #1173 from Hestia-Homes/bug/pashub-fetcher-correctly-writes-to-sharepoint
Increase pashub fetcher timeout to 5 mins
2026-06-04 16:14:35 +01:00
Daniel Roth
ab465bf010 increase pashub fetcher timeout to 5 mins 2026-06-04 15:12:29 +00:00
Khalim Conn-Kowlessar
aac3f0690a S0380.221: default a missing API post_town so the cert stays mappable
A 2026-register cert (4519-9056-4002-0222-4802) omits the top-level
post_town entirely (its town sits only in address_line_3 "BARNSTAPLE").
RdSapSchema21_0_x declares post_town as a required no-default field, so
from_dict raised "missing required field 'post_town'" and blocked the
whole cert from computing.

post_town is address metadata the SAP cascade never reads (no consumer
in domain/sap10_calculator/), so default an absent post_town to "" in a
from_api_response pre-processor (mirroring _normalize_shower_outlets) —
inert for the calculation, keeps the cert mappable. The schema dataclass
can't simply give post_town a default: it is a plain (non-kw_only)
dataclass with 57 required fields after post_town, so a mid-list default
would break field ordering.

Validated: cert 4519 now maps (post_town="") and computes SAP cont 74.68
vs lodged 75. §4 suite 2392 passed; mapper.py pyright unchanged at 32;
new tests suppress reportPrivateUsage (net-zero).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 14:59:37 +00:00
Jun-te Kim
261fae2e79 reformatted to be DDD structure 2026-06-04 14:50:04 +00:00
Khalim Conn-Kowlessar
d164850dd3 S0380.220: map API floor_construction code 0 → None (unknown/N-A)
The 2026 sample's second-largest mapper raise: 37 certs lodge
sap_floor_dimensions.floor_construction=0, which raised UnmappedApiCode
and blocked the cert. Code 0 is the "not recorded / not applicable"
sentinel — 33/37 pair it with floor_heat_loss=6 ("another dwelling
below", an upper-floor flat with no ground floor to describe); the rest
carry mixed Solid / unheated-space descriptions. There is no single
construction to assert.

Map code 0 → None, which defers to RdSAP 10 Table 19 ("where floor
construction is unknown" → age-band default) — identical to how an
unlodged floor_construction (the 993 None certs) is already handled, and
honest about the absence (cf. the no-misleading-insulation_type rule).

Empirically inert and validated: across all 37 code-0 certs the cascade
floor W/K is byte-identical whether code 0 maps to None or to an explicit
"Solid" string — the another-dwelling-below floors compute to 0.0 W/K
(handled via floor_heat_loss + property_type=Flat + floors[].description,
per the _API_FLOOR_HEAT_LOSS_TO_FLOOR_TYPE code-6 note), and the few
genuine ground/unheated floors hit the same age-band default either way.
All 37 now compute (were raising).

Dict value type widened to Optional[str] for the None entry; helper
already returns Optional[str]. §4 suite + schema-mapper tests green
(pre-existing test_total_floor_area failure unrelated); mapper.py pyright
unchanged at 32; new test suppresses reportPrivateUsage (net-zero).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 14:37:50 +00:00
Khalim Conn-Kowlessar
c89bec42cb S0380.219: map API floor_construction code 3 → "Suspended, not timber"
A random 1000-cert Jan–May 2026 EPB-register sample surfaced 53 certs
lodging sap_floor_dimensions.floor_construction=3, which raised
UnmappedApiCode and blocked the whole cert from computing (~44 of the
sample's mapper raises). RdSAP 10 field 3-1 "Floor construction"
enumerates the lowest-floor construction as solid / suspended timber /
suspended, not timber, and the spec's "Suspended not timber (structural
infiltration 0)" makes the split load-bearing.

Map code 3 to the canonical "Suspended, not timber" string (the same
value the site-notes mapper already emits — cross-mapper parity):
  - u_floor takes the suspended BS EN ISO 13370 branch via the
    "Suspended" prefix (_floor_is_suspended_from_description), and
  - _has_suspended_timber_floor_per_spec's exact-match
    `!= "Suspended timber"` gate correctly does NOT fire, so the §5 (12)
    0.1/0.2 floor-infiltration adjustment is skipped (structural
    infiltration 0) — exactly the spec rule for not-timber suspended.

Validated: all 5 sampled code-3 certs now compute (e.g.
0340-2877-5570-2606-5965 floor_construction_type="Suspended, not
timber", SAP cont 60.12 vs lodged 60). Confirmed against the cert's own
global floor descriptions ("Suspended, …", floor_heat_loss=7).

Code semantics established from the RdSAP 10 spec + the lodged certs'
human-readable floor descriptions (the EPB /api/codes endpoint carries
no floor_construction enum). §4 suite + schema-mapper tests green
(the pre-existing test_total_floor_area failure is unrelated). mapper.py
pyright unchanged at 32; new test suppresses reportPrivateUsage to keep
net-zero new errors.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 14:02:43 +00:00
Daniel Roth
d9124ee3b4
Merge pull request #1170 from Hestia-Homes/bug/pashub-fetcher-correctly-writes-to-sharepoint
Pashub fetcher bugs: Playwright /tmp cleanup, and sharepoint folder pagination
2026-06-04 13:59:55 +01:00
Daniel Roth
bdae83b715 SharePoint folder listing requests 500 items per page to cover alphabetically-late properties 🟩 2026-06-04 12:52:07 +00:00
Daniel Roth
428a5db513 Playwright tmp dirs are cleaned up after browser close 🟩 2026-06-04 12:51:14 +00:00
Daniel Roth
bd1fa09dc2 Playwright tmp dirs are cleaned up after browser close 🟥 2026-06-04 12:50:16 +00:00
Khalim Conn-Kowlessar
d3def1e254 docs: handover — S0380.218 closed the "with api 3" pair (both clean)
Record S0380.218 shipped, bump HEAD/next-slice, note both certs are
0-residual cross-validated golden fixtures and flag the optional
Summary-path regression guard as the cheap follow-up.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 12:36:23 +00:00
Khalim Conn-Kowlessar
6d9ef1143c S0380.218: golden fixtures for 2 fresh API+Summary+worksheet triples
Two certs fetched fresh from the GOV.UK EPB register, each with an
Elmhurst Summary PDF (input) and a dr87 worksheet PDF (the (1)..(286)
ground truth):

  0340-2467-9260-2006-6521  (Summary_000922 / dr87-0001-000922)
  5500-5070-0822-0201-3663  (Summary_000920 / dr87-0001-000920)

Both run through BOTH front-ends — from_api_response and
from_elmhurst_site_notes — and through the rating + demand cascades.
Cross-mapper parity holds: the two paths agree to <1e-4 on continuous
SAP, fuel cost, CO2 and PE. Both paths reproduce the worksheet exactly:

  0340: (255) cost 776.4295, (272) CO2 2875.0498, (286) PE 16474.5616;
        fabric (33) 171.6188, (37) 205.9358; SAP int 70 = lodged.
  5500: (255) cost 751.8295, (272) CO2 2423.4547, (286) PE 14397.0118;
        fabric (33) 141.1226, (37) 167.3696; SAP int 66 = lodged.

Pinned in two tables of test_golden_fixtures.py:
  - _EXPECTATIONS / test_golden_cert_residual_matches_pin — SAP/PE/CO2
    residual vs the integer-rounded lodged register (SAP resid +0 both).
  - _WORKSHEET_PE_CO2 / test_golden_cert_pe_co2_matches_worksheet —
    PE (286)/(4) and CO2 (272) vs the worksheet at +0.0000 (the
    load-bearing 1e-4 check; lodged register is integer-rounded).

Dropped-field audit (raw JSON keys vs the schema-21.0.1 dataclass
fields consumed by from_dict) re-run on both fresh JSONs: no new
silently-dropped fields — only created_at metadata and the
shower_outlet_type/shower_wwhrs keys already handled by
_normalize_shower_outlets (mapper.py:2047). No calculator or mapper
change required; this is pure validation + regression-pinning.

Full §4 suite: 2392 passed, 1 skipped.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 12:35:50 +00:00
Khalim Conn-Kowlessar
1085842395 docs: handover — flagged numbers were stale (different branch), Part 1 is the task
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 12:11:12 +00:00
Khalim Conn-Kowlessar
8bd8ff8e5c docs: handover — fresh-API cross-comparison + flagged-cert debugging
Next-agent brief: fetch certs fresh from the EPC API (two new API+Summary+
worksheet triples for cross-mapper parity, plus six dashboard-flagged certs).
Flags the critical reconciliation: the user's flagged numbers don't match the
golden-fixtures cascade (0390-2954-3640 pinned +0 but flagged -6.85; 7536/2130
flags are pre-this-session), so fresh-raw-JSON-vs-curated-fixture or a
different engine must be reconciled before debugging. Documents the EPC API
fetch mechanism, the dropped-field audit method, this session's 4 fixes, and
the conventions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 12:08:28 +00:00