Commit graph

117 commits

Author SHA1 Message Date
Daniel Roth
c8fe4585bf Send the deal's third-party surveyor as the OpenHousing resource
Carry third_party_surveyor_identifier on the Abri trigger message and
through dispatch into the log and amend flows, booking the job against
that resource. LogJobRequest gains the optional resource field
AmendJobRequest already had; the client falls back to the configured
default surveyor when the deal names no third-party resource, so
existing deals behave unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 13:00:47 +00:00
KhalimCK
04b7c26f67
PasHub SAP accuracy: infiltration inputs + immersion DHW + lights/lobby/heat-network (#1615)
* Electric-immersion DHW: bill at 100%, not the space boiler's efficiency 🟩

A separate electric immersion heater (SAP water-heating code 903) is 100%
efficient (SAP 10.2 Table 4a) and the space-heating boiler provides no water
heating, so Appendix D2.1 Eq D1 (the boiler seasonal-efficiency cascade) must
not apply to it. But `_water_heating_main` resolves DHW to the SPACE main — a
gas/oil boiler keeps its PCDB record — so three water-efficiency branches in
cert_to_inputs billed the immersion-heated cylinder at the boiler's ~87% summer
efficiency (a mongrel: electric fuel price x gas-boiler efficiency):

  1. the scalar `water_eff = water_pcdb_main.summer_efficiency_pct / 100`,
  2. the SAP §9.4.11 / Table 4c(2) -5pp no-interlock adjustment, and
  3. the Eq D1 (winter, summer) seasonal pair from `pcdb_main`.

Gate all three on `not dhw_is_electric_immersion`; the correct immersion path
(`_water_efficiency_with_category_inherit` -> 1.0, Eq D1 off) then applies,
mirroring the Table 3 zero-primary-loss gate already present for WHC 903.

General bug — fires for any WHC-903 dwelling whose space main is a PCDB gas/oil
boiler. It surfaced via the PasHub campaign's #1600 no-water-heating default
(WHC 999 -> 903) on a gas-combi dwelling: 58 Hackle St M11 4WU, SAP 55.30 ->
57.67 (pre_sap 58, verified 59).

Guardrails: the gov-API RdSAP corpus IMPROVES 78.8% -> 78.9% within-0.5, MAE
0.625 -> 0.622 (a handful of corpus certs with a boiler space main + electric
immersion move closer to accredited) — MAE ceiling ratcheted 0.626 -> 0.625.
Regression pinned in test_cert_to_inputs (RED before / GREEN after). pyright
0-new (cert_to_inputs baseline 30).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* PasHub SAP accuracy: thread low-energy lights + draught lobby (from_site_notes) 🟩

Two systematic silent-drops in the PAS Hub `from_site_notes` path, each a field
the gov-API/Elmhurst mappers thread but site-notes dropped, letting a favourable
default reach the SAP-10.2 calculator. Validated against the correct pashub
oracle (property_baseline_performance.effective_sap_score, portfolio 838):
cohort within-0.5 55.1% → 62.6%, MAE 0.599 → 0.521.

- Low-energy lights: when "exact LED/CFL known = No", PAS Hub lodges an aggregate
  "Number of fixed low energy lights?" count. Previously dropped (no dataclass
  field / extractor key / mapper thread) → calculator saw 0 low-energy bulbs and
  applied the pessimistic L5b/L8c no-data default, under-rating SAP. Now threaded
  into low_energy_fixed_lighting_bulbs_count, mirroring from_elmhurst_site_notes.
  ~6% of the cohort; resolves 58 Hackle (−0.43 vs oracle 59 with #1615).
- Draught lobby: _map_sap_ventilation set only the legacy `draught_lobby` field,
  never the canonical `has_draught_lobby` §2 (13) gate the cascade reads, so the
  surveyed lobby was ignored and infiltration over-stated. Now mirrors Elmhurst.

Two other audited levers were REJECTED against the pashub oracle (they matched
the gov-cert/RdSAP convention but pashub does not apply them): percent_draughtproofed
(cohort 30.8%) and the §A.2.2 assumed secondary heater (19.2%). A heat-network
control-code fix (2306→2303 for 16 Bingley) is HELD pending spec/Elmhurst
adjudication — it contradicts ADR-0053; see the NOTE on _PASHUB_HEAT_NETWORK_CONTROL_TO_SAP10.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* PasHub heat-network control: room-thermostat-only maps to 2308, not 2306 🟩

The Table 4e Group 3 label "charging linked to use of community heating, room
thermostat only" was mapped to 2306 — the linked-to-use *with-TRVs* code
(control type 3, space charging 1.00). The label explicitly excludes TRVs, so
it is control type 2 / space 1.05 = code 2308/2309 (linked-to-use → DHW 1.00).
2306 asserted TRVs the survey doesn't have and over-rated the cohort's sole
heat-network dwelling (16 Bingley Close 56.89 → 54.01 vs pashub oracle 52).

Verified against SAP 10.2 Table 4e as encoded in cert_to_inputs.py
(_CONTROL_TYPE_BY_CODE 2308→2, space-charging 2308→1.05, DHW 2308→1.00) and the
RdSAP control-label vocabulary in MainheatControlAttributes.py. 2303 (the earlier
audit's guess) is rejected: it is a flat-rate code (DHW 1.05) whose oracle match
was a coincidence of two offsetting spec violations. The residual +2.0 to oracle
52 is the documented SAP-10.2-engine-vs-lodged offset, not a fuel/flags gap
(those were threaded by the #1590 follow-up) — supersedes the ADR-0053 /
HANDOVER_838 "community fuel/flags" attribution.

16 Bingley is the only heat-network fixture in portfolio 838, so this moves one
fixture strictly toward its oracle. Harness ratcheted: within-0.5 0.51→0.58,
MAE 0.625→0.521 (observed 58.5% / 0.520 with all three 2026-07-15 levers).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* PasHub SAP accuracy: restore 3 dropped §2 infiltration inputs (from_site_notes)

`from_site_notes` uniquely dropped three ventilation inputs that the gov-API
and Elmhurst sibling mappers all set, causing a systematic cohort-wide SAP
under-rate vs the company's own accredited SAP-10.2 certs
(property_baseline_performance.effective_sap_score, portfolio 838):

1. percent_draughtproofed — never set, so §2(15) window infiltration was
   pinned at its 0.25-ACH worst case on every dwelling. Reconstructed as the
   area-weighted % of draught-proofed windows (mirrors Elmhurst
   `draught_proofing_percent`). Dominant term.
2. Upper-storey +0.25 m joist void — omitted; now added, matching
   `_UPPER_FLOOR_HEIGHT_ADD_M` on the gov-API/Elmhurst paths. This
   RE-ADJUDICATES #1601 ("keep raw"): that call was confounded by the
   then-present draught-proofing drop suppressing every verified dwelling.
3. sheltered_sides — left None → calculator's flat default of 2, which
   over-shelters end/semi/detached (RdSAP §S5 = 1/1/0). Now derived from
   built form.

The three are NON-ADDITIVE — each overshoots alone (which is why all three
were previously rejected individually) — but together they land all 7
verified ground-truth dwellings toward truth (none regress) and every built
form near zero. Cohort 62.1% -> 83.3% within-0.5, MAE 0.507 -> 0.377.

Guardrails: gov-API RdSAP corpus unchanged (78.9%, these are from_site_notes-
only helpers). pashub harness 58.5% -> 79.5% / MAE 0.520 -> 0.389; ratchets
tightened 0.58->0.78 and 0.521->0.40. pyright zero-new (mapper baseline 39).
New focused tests in TestFromSiteNotesInfiltrationFixes cover all three
fields incl. area-weighting and the built-form shelter map; goldens updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 00:30:55 +01:00
Daniel Roth
a412c8942b Ratchet the API-corpus SAP MAE ceiling to 0.626 for the water-only C6 standing charge 🟪
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 13:50:17 +00:00
Daniel Roth
a400005353 Ratchet the API-corpus within-0.5 floor to 78.8% for the water-only DHW fix 🟪
Enumerates the 3 corpus certs that can move (all WHC 950) individually per
#1592 sub-task D: two improve (one crossing inside 0.5), one worsens and is
handed to sub-task C for an accredited oracle. No loosening required.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 10:34:19 +00:00
Jun-te Kim
121b3d991d
Merge pull request #1533 from Hestia-Homes/fix/epc-schema-allowlist
Unsupported EPC schema versions skip with a warning instead of crashing
2026-07-10 12:43:40 +01:00
Jun-te Kim
b540cf1ca2 Unsupported EPC schema versions are skipped with a warning, not a crash
An unrecognised schema_type used to raise ValueError and abort the whole
call; now from_api_response logs a warning and returns None so one
unmapped cert doesn't break a batch. Schema 15.0 already has a mapper
(PR #1531) so it's unaffected; only genuinely unmapped versions skip.
2026-07-10 11:17:07 +00:00
Daniel Roth
f8b9d82200 Always send a bookable resource on Abri amend_job
OpenHousing logs a job but silently drops the appointment when no bookable
resource is sent, so the survey date never lands. log_job already sends the
configured default_resource; amend_job omitted the parameter unless the
request carried an explicit surveyor. Default amend's resource to the
configured surveyor too, and fail loudly at config load on a blank
ABRI_RELAY_DEFAULT_RESOURCE so the misconfiguration can't ship an empty
resource again.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 10:37:35 +00:00
Daniel Roth
7997d88375 minor updates following Abri feedback 2026-07-09 16:27:50 +00:00
Khalim Conn-Kowlessar
91db972d48 Bulk download: readable email, deal-name folders, visible worker logs
First successful live run surfaced three issues:

1. Email looked rubbish (a giant raw presigned URL). Now sends a proper HTML
   email with a 'Download documents' button plus a plain-text fallback, and a
   summary (N documents across M properties, expiry). Email delivery is now
   best-effort: a transport failure no longer loses an already-built package
   (the link is still on sub_task.outputs), and the SMTP connect has a 30s
   timeout so an unreachable SES endpoint fails fast instead of hanging to the
   900s Lambda timeout.

2. Every folder was 'address unavailable (...)': the resolver read property.address,
   but these are HubSpot deals with no property row. It now uses the deal's
   dealname from hubspot_deal_data.

3. No logs / no idea why a run took ~9 minutes: the worker's INFO logs were
   dropped (Lambda root logger defaults to WARNING). The handler now raises the
   level, and the orchestrator logs per-phase timing and volume (gather+plan,
   packaged N files / X MB, upload, email, total) so the slow phase is visible.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 15:02:38 +00:00
KhalimCK
bd24a789b9
Merge pull request #1498 from Hestia-Homes/feature/bulk-document-download
Bulk Document Download: emailed ZIP of a property set's documents (ADR-0059/0060)
2026-07-08 12:49:46 +01:00
Khalim Conn-Kowlessar
9ec7987e97 PR review: best-effort read path, true per-member streaming, race-safe 409, local email 🟩
Addresses PR #1498 review:
- A per-file read failure (missing/deleted object, or a bucket the role can't
  reach) is now a SkippedDocument(reason=unreadable), not a whole-run abort;
  an all-unreadable selection still fails (no empty package). Best-effort on
  the read path (ADR-0060).
- Each ZIP member is streamed to a temp file (S3DocumentDownloader.download,
  boto download_file) and added from disk, so a single multi-GB member never
  hits the heap — the 'never held whole in memory' claim is now true.
- The 409 double-submit guard is DB-arbitrated: the sub_task insert is
  conditional on the task having none, so a race creates only one.
- Local env gets a placeholder recipient email so the route is exercisable.
- PackageEntry carries landlord_property_id so a read failure can be reported.
- TODO noting the UploadedFile.s3_upload_timestamp typing fix.
Two new tests: per-file read failure skips-and-reports; address fallback flows
to the folder name.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 11:08:46 +00:00
Daniel Roth
0d82f9e660 Declare the XML content type the Abri relay requires 🟥
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 10:07:04 +00:00
Khalim Conn-Kowlessar
79c8890d07 Stream a local file to S3 with managed multipart upload 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:50:10 +00:00
Khalim Conn-Kowlessar
0a09b56425 Read document bytes from an arbitrary source bucket 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:42:05 +00:00
Khalim Conn-Kowlessar
5151fea9b2 STARTTLS and authenticate before sending the email 🟩
Pins the auth handshake delivered with the tracer green (no red phase);
discriminating: an unauthenticated/cleartext send would be rejected by SES.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:27:31 +00:00
Khalim Conn-Kowlessar
7466b78e54 Send a document-download email to the requesting user over SES SMTP 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:25:56 +00:00
Khalim Conn-Kowlessar
ba13300ee1 Generate a presigned GET URL for an S3 object 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:18:10 +00:00
Daniel Roth
007e561af5 Keep ambiguous or failed-transport canceljob responses retriable 🟩
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 15:36:15 +00:00
Daniel Roth
a384b87d8f Surface both canceljob failure shapes as verbatim rejections 🟥
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 15:33:16 +00:00
Daniel Roth
c206a76b22 Abandon a job through the relay canceljob route 🟥
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 15:30:24 +00:00
Jun-te Kim
bc1cca77db
Merge pull request #1356 from Hestia-Homes/feature/historic-epc-repository
Historic EPC repository: DDD port + resolver for address→UPRN
2026-07-07 10:56:33 +01:00
Jun-te Kim
c238a0e6a0
Merge pull request #1449 from Hestia-Homes/feat/sap-corpus-campaign
fix(rdsap): RdSAP 10 Table 13 200/280mm solid-brick wall boundary
2026-07-06 12:25:17 +01:00
Khalim Conn-Kowlessar
ab245de68d Merge remote-tracking branch 'origin/main' into feature/historic-epc-repository 2026-07-04 11:19:59 +00:00
Daniel Roth
05bd22269d Abri amend_job raises retriable transport errors when the amendment outcome is unknown 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 16:23:53 +00:00
Daniel Roth
65ec38012b Abri amend_job treats an unconfirmed amendment response as retriable 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 16:22:49 +00:00
Daniel Roth
e9cb46a58b Abri amend_job surfaces OpenHousing rejections with code and message verbatim 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 16:22:11 +00:00
Daniel Roth
b70afc8d9d Abri amend_job omits the surveyor resource so amendments cannot clobber Abri-side reassignments 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 16:20:41 +00:00
Daniel Roth
fbb4345116 Abri amend_job serialises the spec's amendoptiappt envelope with the surveyor resource last 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 16:18:24 +00:00
Daniel Roth
2cfed0c33d Abri amend_job amends an OpenHousing appointment and returns the echoed booking 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 16:15:44 +00:00
Daniel Roth
c0d36d1e67
Merge pull request #1458 from Hestia-Homes/feature/abri-api-integration
Abri DomnaRelay client — foundation + LogJob vertical (strictly typed)
2026-07-03 16:49:46 +01:00
Daniel Roth
c1c2a1947e Abri log_job treats a logged-job response missing its job number as retriable 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 14:35:35 +00:00
Daniel Roth
39055bafed Abri relay credentials and defaults hydrate from environment configuration 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 14:34:27 +00:00
Daniel Roth
a325f65bf0 Abri log_job treats an unexpectedly shaped relay response as retriable 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 14:30:02 +00:00
Daniel Roth
ff34872c24 Abri log_job treats a malformed relay response as retriable, never as success or rejection 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 14:29:08 +00:00
Daniel Roth
6b9f5a0140 Abri log_job raises a retriable transport error when the relay is unreachable 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 14:28:19 +00:00
Daniel Roth
9fefdf81aa Abri log_job raises a retriable transport error on an HTTP error status 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 14:27:14 +00:00
Daniel Roth
e473ea8a4f Abri log_job surfaces an OpenHousing rejection as a typed result with verbatim code and message 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 14:25:49 +00:00
Daniel Roth
79ce9dc4b1 Abri log_job sends the spec's recorded LogJob envelope to the relay endpoint 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 14:24:57 +00:00
Daniel Roth
4803ae40e1 Abri log_job sends the spec's recorded LogJob envelope to the relay endpoint 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 14:23:42 +00:00
Daniel Roth
b22afcde9f Abri log_job returns the OpenHousing job number from the relay success response 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 14:22:09 +00:00
Jun-te Kim
5cae4a0ce5 fix(rdsap): RdSAP 10 Table 13 200/280mm solid-brick wall boundary
_u_brick_thin_wall_age_a_to_e's `<= 280` check put an uninsulated solid-
brick wall of exactly 280mm in the "200 to 280mm -> 1.7" row; RdSAP 10
§5.7 Table 13 (spec PDF p.41) shares 280 as an unlabelled edge between
that row and "280 to 420mm -> 1.4", so the table text alone doesn't say
which row owns it.

Found by building cert 100031768368 (280mm exactly, band C) in Elmhurst
and comparing every worksheet line to the calculator: volume/ACH/floor/
doors matched exactly, but the wall didn't (1.70 vs Elmhurst's 1.40)
despite an identical input crosswalk. Initially reverted a first attempt
at this fix when it appeared to regress an already-pinned cert
(217091901, band A, also nominally 280mm, previously "confirmed" at
U=1.70) — but that cert's build script never actually set a wall
thickness field, so its shared Elmhurst assessment had silently
inherited a stale 260mm from an earlier build. Fixed that build script
and rebuilt with the correct 280mm entry: Elmhurst's worksheet now also
gives U=1.40, matching 100031768368 and confirming the fix rather than
contradicting it.

Both certs move to (near-)exact lodged matches: 100031768368 59.12 ->
61.21 (lodged 61, within 0.5); 217091901 60.82 -> 61.59 (lodged 62,
exact). Corpus gauge 77.8% -> 78.6% within-0.5, MAE 0.636 -> 0.627 (14
corpus certs lodge exactly 280mm solid brick). TDD'd, 46 pins + full
suite green (2310 passed).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 12:25:07 +00:00
Jun-te Kim
dee72bbdfa fix(rdsap): §5 (12) floor infiltration falls back to floors[].description
_has_suspended_timber_floor_per_spec only read the Main bp's per-part
floor_construction_type lodgement; the gov-API mapper frequently leaves
that None even when the global epc.floors[].description carries an
explicit "Suspended, ..." observation. _main_floor_u_value already fell
back to this joined description for the U-value calc — the infiltration
rule did not, so a genuinely suspended-timber floor silently entered
(12)=0 instead of 0.2 unsealed. Extracted the shared
_effective_floor_description helper and added a tri-state resolver
(explicit lodgement -> description, excluding "not timber" -> Table 19
footnote-1 age-A/B default) so both paths agree.

Cert 100061275133 (Elmhurst-validated build, PR #1439 handoff): engine
77.24 -> 76.33, now an exact match to lodged (76). Corpus gauge 77.7% ->
77.8% within-0.5, MAE 0.637 -> 0.636; floors ratcheted.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 09:12:19 +00:00
Jun-te Kim
8e33a8de53 test(accuracy): ratchet 77.3% -> 77.7% via flat-roof insulation-thickness fix (Elmhurst-validated)
A FLAT roof lodges its insulation thickness in the DEDICATED gov-EPC API
`flat_roof_insulation_thickness` field (e.g. "75mm"), leaving
`roof_insulation_thickness` None (that field is for pitched-loft joists).
`heat_transmission_from_cert` read only the latter, so a measured
flat-roof thickness was ignored and the roof billed at the uninsulated
age-band flat default (age E = 1.5) instead of its Table-16 insulated U.

Fixed by preferring `flat_roof_insulation_thickness` when the part is a
flat roof — the exact mirror of the existing rafter-thickness branch.
An "AB"/"NI" (as-built/unknown) value parses to None and keeps the
age-band default, unchanged; only measured thicknesses move.

PER-CERT ELMHURST VALIDATION (cert 47084930, top-floor flat, flat roof
75 mm): built on the lodged inputs in accredited Elmhurst RdSAP10-Online
(evidence saved: elmhurst_summary.pdf / elmhurst_worksheet.pdf). The
worksheet bills "insulated flat roof" at U 0.5 (floor 0.70 + wall 0.25
also matching the engine). The fix takes the engine roof 96.4 -> 32.1 W/K
(= 64.26 x 0.5, Elmhurst-exact), PE +47 -> +0.2, SAP 69.51 -> 74.34 =
lodged 74.

Gauge: within 77.3% -> 77.7%, SAP MAE 0.648 -> 0.641, CO2 0.074 -> 0.072,
PE 3.1 -> 2.97. Unit-pinned in test_heat_transmission
(flat_roof_insulation_thickness -> U 0.5); RealCertExpectation 47084930
= 74 (Elmhurst-validated). Also adds the build script build_47084930.py.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 23:27:57 +00:00
Jun-te Kim
11a637c6bd test(accuracy): ratchet 77.0% -> 77.3% via unsized-cylinder storage-loss fix
A hot-water cylinder lodging gov-API `cylinder_size=0` ("size not
determined", with has_hot_water_cylinder=true) resolved to a None volume
in `_cylinder_storage_loss_override` (which reads `_cylinder_volume_l_
from_code`, returning None for code 0). The subsequent `if volume_l is
None: return None` then DROPPED the cylinder's Table-2 storage loss
entirely — under-costing the DHW and over-rating the dwelling. Meanwhile
the Table-13 high-rate-fraction path already used `_hot_water_cylinder_
volume_l` (which defaults size 0 to 110 L), so the two DHW paths
disagreed on the same cylinder.

RdSAP 10 §10.5 Table 28 ("if the actual size is not determined, the size
is taken as according to Table 28") makes an unsized present cylinder the
110 L "Normal" baseline, which STILL incurs the storage loss. Fixed by
defaulting the storage-loss volume to 110 L for the explicit size-0 case.

Gated on the EXPLICIT 0 (not None): a full-SAP cert whose RdSAP
cylinder_size is simply unlodged (None, e.g. pinned 10091568921) keeps
its own cylinder handling rather than a forced 110 L RdSAP default.

7 corpus certs lodge cylinder_size=0 + has_cylinder=true. Gauge: within
77.0% -> 77.3%, SAP MAE ~flat (0.648), CO2 0.074 -> 0.073, PE 3.2 -> 3.05
(the previously-dropped storage loss now correctly counted in the demand
cascade). Unit-pinned in test_cert_to_inputs (storage loss non-None for
size 0); RealCertExpectation 200004017091 = 71.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 23:02:27 +00:00
Jun-te Kim
0e784cbb1c test(accuracy): ratchet SAP MAE 0.658 -> 0.647 via wall added-insulation unknown-thickness default
A SOLID-BRICK or STONE wall lodging External/Internal insulation
(wall_insulation_type 1/3) but no measured insulation thickness (gov-API
"NI" -> None) was billed at its UNINSULATED §5.6/Table-6 U-value: the §5.8
added-insulation R-value adjustment gates on `insulation_thickness_mm > 0`,
so an unknown thickness fell straight through to the raw solid-wall U
(~1.4-1.7). This over-counted wall heat loss and under-rated the dwelling.

RdSAP 10 §5.4 (PDF p.32): "the U-value with additional insulation is based
on the thickness of insulation of mineral wool type (assume 100 mm if
thickness is unknown)". Apply that 100 mm default so the §5.8 Table-14
R-value fires. Scoped to solid brick / stone (the only constructions with
a §5.8 R-value path in u_wall) so cavity (composite path) and timber /
system-built walls are untouched.

Localised by deep-diving corpus cert 200004296092 (end-terrace, solid
brick 360 mm "with external insulation", NI): walls 101.7 W/K (U~1.40,
HLP 3.46 = absurd for an insulated wall) -> ~0.29; PE +59.6 -> ~0; engine
64.35 (Δ -6.65) -> 71.74 = lodged 71. ~8 solid-brick/stone corpus certs
carry this NI-thickness-with-insulation shape.

Gauge: within-0.5 held at 77.0%, SAP MAE 0.658 -> 0.647, CO2 0.074 ->
0.073, PE 3.2 -> 3.1 (floor ratcheted). Unit-pinned in test_rdsap_uvalues
(brick 360 mm external NI -> 0.29) + RealCertExpectation 200004296092 = 72.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 20:24:56 +00:00
Jun-te Kim
cb089a630c test(accuracy): ratchet SAP MAE 0.668 -> 0.658 via instantaneous-electric-DHW off-peak fix
An instantaneous point-of-use electric water heater (WHC 907/909, no
cylinder) on an off-peak tariff was billed 100% at the off-peak LOW rate
by the generic electric-off-peak else-branch. But SAP 10.2 §12 (PDF line
2680) computes the off-peak on-peak HW proportion via Table 13, "dependent
on the total floor area and the CYLINDER size" — it presumes a stored-water
cylinder charged overnight. An instantaneous heater has NO cylinder, heats
on demand, and cannot shift to the off-peak window, so 100% of its
consumption bills at the HIGH rate (Table 12a WH high-rate fraction 1.0).

Both the SAP-cost rate (`_hot_water_fuel_cost_gbp_per_kwh`) and the
ADR-0014 / CO2-PE fraction (`_hot_water_high_rate_fraction`) fixed
consistently (low-rate scalar -> 7-hour high rate; fraction 0.0 -> 1.0).

Localised by deep-diving corpus cert 74061136 (HHR-storage mid-floor flat,
WHC 909): PE matched lodged (+1.6, roof+floor zero-loss) while SAP over-rated
+7.72 — the cost-only signature. Its DHW was ours £59 vs lodged £344 (5.8x);
the tariff was the whole gap. Fix: +7.72 -> -1.25 (residual is separate small
fabric). 7 corpus certs carry electric-instantaneous DHW on off-peak; the 3
outside 0.5 all move sharply inward (MAE the win, not within-0.5 crossings).

RdSAP 10 §12 tariff routing confirmed spec-correct (Unknown meter + storage
409 -> off-peak 7-hour, Rule 2) — the bug was the DHW rate, not the tariff.
Unit-pinned in test_cert_to_inputs; RealCertExpectation 74061136 = 72.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 20:04:58 +00:00
Jun-te Kim
06a2c413f5 test(accuracy): ratchet 75.5% -> 77.0% via secondary-fuel collision fix on cost path
The gov-API `secondary_fuel_type` enums 5 (anthracite) / 9 (dual fuel) /
33 (coal) collide in VALUE with same-valued RdSAP-10 Table-32 codes for
OTHER fuels (5=bulk LPG, 9=LPG SC11F, 33=elec 10h-low). The main-fuel
boundary and the CO2/display `_secondary_fuel_cost_gbp_per_kwh` already
canonicalise these via `canonical_fuel_code`, but the SAP-DRIVING
`_fuel_cost` cascade had a SEPARATE inline secondary rate that passed the
raw enum straight to the price lookup — so the two paths diverged and the
cost path silently mis-priced.

Anthracite (enum 5) was billed at the bulk-LPG code-5 rate 12.19 p/kWh
instead of 3.64 — a 3x over-cost on the secondary that under-rated every
solid-fuel-secondary dwelling. Worst: corpus cert 100050355518 (semi,
2 extensions, anthracite room-heater secondary) lodged 36 / engine 20.8
-> 35.5. Its CO2/PE already matched lodged (the demand cascade was right)
— a pure cost-side gap, which is what localised it.

Fix: canonicalise `secondary_fuel` before the Table-32 lookup in
`_fuel_cost`, mirroring the main-fuel boundary. 55 corpus certs carry a
colliding secondary.

Gauge: within-0.5 75.5% -> 77.0%, SAP MAE 0.708 -> 0.668 (floors
ratcheted). The prior enum-9 test only guarded the display helper; new
test_fuel_cost_secondary_colliding_fuel_priced_at_canonical_rate pins the
_fuel_cost cost path directly. RealCertExpectation pinned for
100050355518 (36 = lodged).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 19:43:31 +00:00
Jun-te Kim
c380377948 test(accuracy): ratchet 74.2% -> 75.5% via 'insulated (assumed)' roof Table-18 fix
RdSAP 10 §5.11.4: a pitched roof lodging thickness "NI" with description
"...insulated (assumed)" was billed at the §5.11.4 observed-retrofit
50 mm row (U 0.68) while the SAME description lodged with "ND" fell
through to the Table 18 age-band default (0.40 for bands A-G). The
ND-vs-NI sentinel is lodging-software noise; "(assumed)" means the
insulation PRESENCE is an age-band assumption, not an observed
retrofit, so Table 18's "thickness cannot be determined" clause
governs both forms. The bare "insulated" description (observed
retrofit, no qualifier) keeps the 50 mm row.

Evidence: the 27-cert loc4+NI corpus cohort was systematically
under-rated (~-1 SAP; zero positive movers among "(assumed)" certs),
vs the 102-cert loc4+ND cohort already on Table 18 at ~0 bias.

Gauge: within-0.5 74.2% -> 75.5%, SAP MAE 0.721 -> 0.708, CO2 MAE
0.09 -> 0.074, PE MAE 3.5 -> 3.2 (floors ratcheted). Unit-pinned in
test_rdsap_uvalues (assumed -> Table 18; bare-insulated -> 0.68
regression guard); integration pin in test_heat_transmission updated;
RealCertExpectation pinned for 10094975827 (66 = lodged, was -1.87).

Ledger: C002 103001004 deferred pending Elmhurst arbitration
(single-immersion off-peak pricing + declared cylinder loss vs
Table-2); C006 quadruplet marked ⚠ (LIG-21.0 software honours lodged
0 extract fans literally vs the Elmhurst-validated 0=unknown->default
convention, worksheet case 46).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 19:06:26 +00:00
Jun-te Kim
7aea692521 historic EPC: read via infrastructure/s3, not the utils.s3 utility
HistoricEpcS3Repository reached into utils/s3.py (read_csv_gz_from_s3 +
parse_s3_uri), the legacy utility that self-constructs boto3 inside free
functions. The other S3 repositories deliberately depend on the
infrastructure/s3 layer instead (UnstandardisedAddressListCsvS3Repository
injects a CsvS3Client). Bring historic EPC into line.

- Add GzipCsvS3Client(S3Client) in infrastructure/s3: read_csv_gz(key) ->
  DataFrame (get_object + gzip decode).
- Inject it into HistoricEpcS3Repository; the bucket lives in the client and
  the repo only builds the per-postcode key + maps rows (no S3/HTTP code).
  Add with_default_s3_client(s3_root) for composition roots.
- Update main.py and the match_addresses_for_postcode seam to the factory.
- Repo tests inject a real GzipCsvS3Client over a controlled boto stub
  (exact key assertions + AccessDenied); add a moto-based client test and a
  factory test covering s3_root -> bucket+key.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MQE5TsSuQTeNSCSz9A9GQf
2026-06-30 09:19:57 +00:00
Khalim Conn-Kowlessar
8914496456 test(accuracy): ratchet after HP water-heating 100% floor 🟪
The Appendix N3.7 water-heating 100% floor drops corpus MAE 0.726 -> 0.721
and lifts within-0.5 74.1% -> 74.2% on the 1000-cert RdSAP-21.0.1 sample
(cert 100110101713 moves inside +-0.5). Tighten the MAE ceiling to 0.722 and
the within-0.5 floor to 0.742, and log the slice.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 15:40:53 +00:00