Commit graph

8588 commits

Author SHA1 Message Date
Jun-te Kim
2f8c086310 Record the DHW-energy residual investigation: not a bug 🟪
Root-caused the 2675-vs-1839 kWh communal-DHW delta: the engine correctly
applies the SAP §4.3 distribution-loss factor (1/(1.05×1.50)); the difference is
the DHW energy content (64) alone. The engine's 1698 kWh tracks the cert's own
lodged RHI water heating (1909) far better than Elmhurst's reduced-data 1168,
and its £113 is closer to the cert's lodged £147 than Elmhurst's £78 — so the
engine is more faithful to the cert here, not buggy. Comment accuracy only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DX8oAGsGkBHq3U4dsxYRzz
2026-07-29 14:11:52 +00:00
Jun-te Kim
6e6c27076e Correct the DHW-energy residual note: no cylinder loss on communal DHW 🟪
The dwelling has no hot-water cylinder (has_hot_water_cylinder false), so the
2675-vs-1839 kWh DHW-energy gap is not a cylinder storage loss as previously
stated — it is an un-root-caused SAP §4.3 community-DHW demand/distribution-loss
difference. No behaviour change; comment accuracy only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DX8oAGsGkBHq3U4dsxYRzz
2026-07-29 14:11:52 +00:00
Jun-te Kim
db274adcb3 Pin the Dovestone water-only community DHW cert recovering to SAP 80 🟩
Full-SAP SAP-Schema-18.0.0 cert 9556-3047-3304-5355-1200 (Dovestone Gardens
Apt 4): communal heat-pump DHW + direct-electric room heaters. Was 67 (D) with
DHW mis-billed as direct-electric immersion; now 80 with DHW on the community
heat-pump fuel. Accredited Elmhurst worksheet 74 (evidence saved); the +6 is the
documented full-SAP measured-U/MVHR vs RdSAP age-band residual.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DX8oAGsGkBHq3U4dsxYRzz
2026-07-29 14:11:52 +00:00
Jun-te Kim
20268a4a6d Bill water-only community DHW on the heat-pump network fuel 🟩
A water-only community scheme (water_heating_code 950/951/952) is served
by its own heat network, independent of the space main. _water_heating_fuel_code
now resolves the network's SAP Table 12 community fuel (e.g. 41 = heat from
electric heat pump @ 4.24 p/kWh, COP already priced into the row) instead of
defaulting to the space main's fuel. For a direct-electric main this had
mis-billed communal heat-pump DHW at the 13.19p standard-electric rate.

Dovestone Apt 4 (cert 9556-3047-3304-5355-1200): hot-water cost £353 → £113,
SAP 67 → 80.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DX8oAGsGkBHq3U4dsxYRzz
2026-07-29 14:11:52 +00:00
Jun-te Kim
c89843e384 Map full-SAP water-only community DHW heat source onto the domain 🟩
Water-only community hot water (water_heating_code 950/951/952,
community_heating_use 2) carries its own heat source in
sap_community_heating_systems, independent of the space main. Expose the
network's heat-fraction-weighted efficiency (COP×100) + Table-12 fuel on
SapHeating so the calculator can price DHW on the network instead of
dropping it to a direct-electric immersion cylinder.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DX8oAGsGkBHq3U4dsxYRzz
2026-07-29 14:11:52 +00:00
Daniel Roth
91e2591de5
Merge pull request #1708 from Hestia-Homes/fix/pin-aws-provider-6.56.0
Pin the AWS provider to 6.56.0 — 6.57.0 breaks deploys
2026-07-29 14:02:20 +01:00
Daniel Roth
8e9c1c257d Pin the AWS provider to 6.56.0 across every Terraform stack
v6.57.0 (released 29 July 2026) breaks ordinary reads across unrelated AWS
services — SSM GetParameter returns SerializationException, IAM GetPolicy
returns a 302, ECR returns InvalidSignatureException — with no config
change. Reported upstream as hashicorp/terraform-provider-aws#49170, open
with no root cause identified; 6.56.0 is confirmed good.

Every stack declared `>= 5.0` with no upper bound and no lock files are
committed, so each CI run silently resolved whatever HashiCorp had shipped
most recently. That is how a provider released today broke a pipeline
nobody had touched. Pinning exactly makes the deployed version a reviewed
decision rather than a discovery.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 10:46:45 +00:00
Daniel Roth
72944dccab
Merge pull request #1706 from Hestia-Homes/feature/renamer-task-handler
SharePoint Renamer: put the run on the app-owned-task lane
2026-07-29 11:22:49 +01:00
Daniel Roth
292bd7e17d Restore the Abri ops guide to main
An unrelated in-progress edit that was already in the working tree when
this branch's work started, swept in by a bare `git add -A`. It belongs to
its author, not to this PR.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 10:09:39 +00:00
Daniel Roth
3e6464fb15 Bind a property's rename callback without a default-argument trick 🟪
partial binds plan eagerly and without introducing a parameter the caller
never passes, so there is no late-binding question to reason about.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 10:05:29 +00:00
Daniel Roth
35a3ebe0e3 Complete a property whose file SharePoint rejected 🟩
Green on arrival — rename() records rejections rather than raising.
Pinned after verifying it bites: letting the rejection escape fails the
child sub_task, which turns the whole Task red.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 09:53:58 +00:00
Daniel Roth
5b1b319c2d Give each property that needs renaming its own sub_task 🟩
Properties are planned and renamed one at a time, so only those with work
get a row and a run killed by the timeout still leaves every property it
reached both renamed and recorded.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 09:52:30 +00:00
Daniel Roth
cc07cc0114 Plan each property's renames before applying any of them 🟩
The walk now yields a PropertyPlan per address-list row, and rename()
applies one. run() composes the two, so single-pass callers are unchanged.
Knowing a property's size before the first rename is what makes "when is
this property finished?" answerable.

Tests that reached through the old private _process_folder are rewritten
against the public run(), so they survive this kind of change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 09:50:28 +00:00
Daniel Roth
737d07a951 Plan each property's renames before applying any of them 🟥
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 09:47:20 +00:00
Daniel Roth
bf1f5693be Build the handler test's address list without a shared mutable default 🟪
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 09:21:11 +00:00
Daniel Roth
bbb3c9e057 Verify a local rename run the way the queue delivers it 🟪
The local invoke now sends an SQS-shaped payload — a direct invoke is the
one shape that cannot reproduce the dry_run bug class. Test modules route
protected-method calls through typed helpers so both pass pyright strict.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 09:19:51 +00:00
Daniel Roth
9870005230 Ship the Renamer image and infrastructure with database access 🟩
The task lane pulls the task domain types, both task repositories and four
Postgres modules into the image, plus SQLAlchemy, SQLModel and a driver.
Terraform gains the DB credentials block and five Postgres env vars; the
deploy job gains the three DB secrets the shared workflow already declares.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 09:11:41 +00:00
Daniel Roth
be1d6faea0 Complete a rename run that hit locked files or missing folders, and reject an unusable site 🟩
Green on arrival: both fall out of the summary the run now returns and of
the guarded site lookup. Pinned after verifying they bite — an unguarded
DomnaSites[name] completes an "ECO" run that renames files under a
different site's alias.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 09:03:43 +00:00
Daniel Roth
0836b18d5b Attribute a rename run to its request and target site 🟩
Green on arrival: @task_handler records inputs and reads source_id from
the body key matching the source's value. Pinned because it is the only
place the new sharepoint_site member is proven against a real Postgres
enum rather than a Python one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 09:01:51 +00:00
Daniel Roth
1213b80e3f Honour dry_run on a queue-delivered rename request 🟩
Green on arrival: reading the request from the SQS record body is what
taking the task lane gave us. Verified the guard bites by restoring the
raw-event read, under which the "dry" run renames a live file.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:59:45 +00:00
Daniel Roth
694881c110 Record a rename run's result summary on its sub_task 🟩
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:58:47 +00:00
Daniel Roth
e7785f6b76 Record a rename run's result summary on its sub_task 🟥
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:58:18 +00:00
Daniel Roth
5309c1f097 Merge subfolder rename results into their parent folder's summary 🟩
Passed on arrival: threading the summary back up through the traversal
already carried subfolder results. Pinned so the merge cannot regress.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:51:36 +00:00
Daniel Roth
b738840c84 Count files skipped as already canonical apart from skipped images 🟩
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:51:02 +00:00
Daniel Roth
2aa698da1e Count files skipped as already canonical apart from skipped images 🟥
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:50:45 +00:00
Daniel Roth
ae7c9cdd46 Record each file SharePoint refused to rename, with its error 🟩
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:49:51 +00:00
Daniel Roth
a69d603cde Record each file SharePoint refused to rename, with its error 🟥
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:49:17 +00:00
Daniel Roth
9248aee139 List the UPRNs with no SharePoint folder under any root 🟩
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:45:46 +00:00
Daniel Roth
9e48f39fa5 List the UPRNs with no SharePoint folder under any root 🟥
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:45:20 +00:00
Daniel Roth
371df4d54f Count dry-run rename intents separately from real renames 🟩
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:44:37 +00:00
Daniel Roth
0135e71590 Count dry-run rename intents separately from real renames 🟥
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:44:20 +00:00
Daniel Roth
391e7bee37 Report how many files a rename run renamed 🟩
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:43:31 +00:00
Daniel Roth
5dc524be8f Report how many files a rename run renamed 🟥
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:42:33 +00:00
Jun-te Kim
82e1dd555a
Merge pull request #1700 from Hestia-Homes/investigatin/bugs_for_esh
Recognise "complete" as a MAIN building part, and anchor lone parts
2026-07-28 18:25:50 +01:00
KhalimCK
3afc43f484
Merge pull request #1691 from Hestia-Homes/fix/default-plan-refresh-on-rerun
Refresh a property's default Plan when its owning scenario is re-run
2026-07-28 18:13:02 +01:00
Jun-te Kim
2aba56c28e
Merge pull request #1696 from Hestia-Homes/fix/wall-construction-enum-mapping
Offer EWI/IWI on gov-API system-built walls (wall_construction code 8)
2026-07-28 18:04:46 +01:00
Jun-te Kim
bb5746c9b5 Recognise "complete" as a MAIN building part, and anchor lone parts 🟪
Six properties failed task 4d006fab (portfolio 854 / scenario 1334) with
DegeneratePredictionError. None has a lodged EPC, so each was predicted from a
neighbouring-cert cohort — and every cohort cert lodges its single building part
as `identifier: "complete"`. from_api_string accepted only "Main Dwelling" /
"Main building", so "complete" fell to OTHER, _has_main_part returned False, and
prediction refused to proceed.

"complete" is what SAP 9.91/9.92-era software writes for an unextended dwelling:
semantically identical to "Main Dwelling". Casing varies within a single postcode
(cert 8824-7422-1180-6934-1902 lodges "complete", 199 Highfield Road "Complete"),
so matching is now case-folded.

The API's `identifier` is free text with no schema enum — api.yml declares the
whole cert body `additionalProperties: true` and documents no field of it, and
`identifier` is absent from the 17 vocabularies at /api/codes. So the recognised
set can only grow by discovery, and each miss has cost an incident ("Main
building" → task a40e71c4; "complete" → task 4d006fab). Hence three changes, not
one:

- "complete" → MAIN, case-insensitively.
- Bare "Extension" → EXTENSION_1. The regex required a digit, so 668 corpus
  occurrences silently dropped a real extension from the structure. This reverses
  a prior deliberate pin whose stated rationale (RdSAP10 §1.2's 4-extension cap)
  does not apply — there is no out-of-range number in a bare "Extension".
- _anchor_lone_building_part: a cert lodging exactly ONE part describes the whole
  dwelling whatever it is called, so an unclassified lone part becomes MAIN.
  Multi-part certs are untouched — there the identifier carries real information
  and guessing would invent structure the cert does not state.

Also removes a false claim from DegeneratePredictionError's message: it asserted
the template was "lodged with a null part identifier", which was hardcoded, never
checked against a real cert, and wrong. It misdirected this investigation.

Verified against the live gov API: all 13 cohort certs across BL1 8EB, BL3 6XJ
and BL4 0RA now resolve a MAIN part. The full modelling e2e was NOT run (no local
AWS creds for the geospatial lookup) — this clears the blocking error, it does not
prove the predictions are good.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-28 15:12:01 +00:00
KhalimCK
4c0ff31454
Merge pull request #1693 from Hestia-Homes/debug/pashub-lrha-sap-gaps
PasHub LRHA WAVE 3: three spec-correct SAP fixes (58.3% → 62.1% within-0.5)
2026-07-28 12:14:58 +01:00
Jun-te Kim
aa4446757f Amend ADR-0019: gov-API code 8 is system built, not park home 🟪
Record why the generator now keys system-built on both code 6 and the
gov-EPC API code 8, citing the GOV.UK RdSAP WallConstructionCode XSD, and
flag the remaining mapper-level enum-normalisation work as open.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DX8oAGsGkBHq3U4dsxYRzz
2026-07-28 10:33:52 +00:00
Jun-te Kim
d1f064433c Offer EWI/IWI on a gov-API system-built wall (code 8) 🟩
Key the solid-wall Recommendation Generator's system-built branch on the
gov-EPC API code 8 (`_WALL_SYSTEM_BUILT_GOV_API`) alongside the internal
code 6, so a precast/no-fines-concrete system-built wall gets its EWI+IWI
Options. This mirrors `u_wall`, which already resolves code 8 as
system-built via `rdsap_uvalues._GOV_API_WALL_CODE_TO_TYPE[8] =
WALL_SYSTEM_BUILT`. Repoint the explicit park-home exclusion to the
gov-canonical park-home code 10. Supersedes ADR-0019's "do not key
system-built on 8", which read code 8 as the Elmhurst park-home label.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DX8oAGsGkBHq3U4dsxYRzz
2026-07-28 10:33:16 +00:00
Jun-te Kim
79bf28d8cb Offer EWI/IWI on a gov-API system-built wall (code 8) 🟥
The gov-EPC API lodges a system-built wall as `wall_construction == 8`
per the authoritative GOV.UK RdSAP `WallConstructionCode` enumeration
(communitiesuk/epb-data-warehouse `.../SAP-Domains.xsd`: 8 = "system
built" in every schema version 17.0-21.0.1; park home is 10). The solid-
wall Recommendation Generator instead read code 8 as the Elmhurst
park-home label and suppressed the whole cohort's EWI/IWI Recommendation
(a 2026 DB sweep finds ~5.1k certs at code 8, ~2.1k uninsulated main
walls). Rewrite the mislabelled park-home test to pin the correct
behaviour (code 8 → EWI+IWI), and add a genuine park-home test on the
gov code 10 so the fix doesn't start over-offering on real park homes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DX8oAGsGkBHq3U4dsxYRzz
2026-07-28 10:33:06 +00:00
Daniel Roth
7f0056c700
Merge pull request #1694 from Hestia-Homes/fix/pashub-evidence-pagination
Paginate PasHub evidence list instead of capping at one page
2026-07-28 11:21:19 +01:00
Daniel Roth
fdccad9c37 Fetch every evidence page when a job has more files than one page holds 🟪
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 10:12:25 +00:00
Daniel Roth
3c52a0aad6 Raise truncation error when PasHub stops serving pages before the full evidence set arrives 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 10:12:03 +00:00
Daniel Roth
f02441b6c4 Raise truncation error when PasHub stops serving pages before the full evidence set arrives 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 10:11:44 +00:00
Daniel Roth
4efb7eccbc Fetch every evidence page when a job has more files than one page holds 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 10:07:28 +00:00
Daniel Roth
5a85403c3d Fetch every evidence page when a job has more files than one page holds 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 10:06:00 +00:00
Khalim Conn-Kowlessar
7b50e5b7f5 docs(pashub): Elmhurst input sheets for the Varsity Rise cluster
The 12 Varsity Rise (Louth) homes are the only community-heated dwellings in
the LRHA WAVE 3 cohort and the only residual over-rate cluster (~+1 SAP). Two
candidate causes cannot be adjudicated from PasHub's `pre_sap` alone and need
accredited Elmhurst worksheets: (A) an electric-immersion DHW fuel-code
collision that bills immersion as community biomass (code 42), and (B) a
suspected community-DHW solar over-credit (the parser drops solar-collector
geometry, so the engine takes Table-29 defaults).

Add page-by-page Elmhurst RdSAP input sheets for one cert of each type
(461386632388 group A, 497671579889 group B) with a "what to read back" list
of the worksheet lines needed to diff term-by-term. Also correct the immersion
single → dual note on the 497655371974 build sheet (the cert lodges Dual).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-28 09:23:11 +00:00
Khalim Conn-Kowlessar
4d9dd9aaf3 fix(sap): (203)-weight the dual-main Table 4e MIT adjustment
For a dwelling with two main systems heating different parts, the (93)m
Table 4e mean-internal-temperature adjustment was taken from system 1 alone
(`details[0]`). SAP 10.2 p.186 requires the (203)-weighted mean of each
system's control adjustment — (1 - (203)) for system 1, (203) for system 2 —
mirroring the Table 9b responsiveness weighting already applied two lines
above. Keyed to system 1, a dwelling led by a small-fraction system (e.g. a
20% HHRSH, adj 0.0 °C, ahead of an 80% manual-charge storage heater, adj
+0.7 °C) was modelled ~0.7 °C too cool → space heat under-counted → over-rated.

Compute the weighted adjustment at the call site. Shared `cert_to_inputs`
path, so it also lifts the accredited gov-API RdSAP corpus (81.5% → 81.7%,
guardrail). 5 Edmund Close +2.74 → +0.59; 4 Edmund Close -2.51 → -0.55.
LRHA cohort within-0.5 60.2% → 62.1%, MAE 0.496 → 0.445; ratchets re-based to
0.62 / 0.45. Also corrects an oil-combi docstring (462051619031 is a Grant
Vortex 10599, not the Worcester 18415).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-28 09:22:59 +00:00
Khalim Conn-Kowlessar
138f5688e0 fix(pashub): wire the surveyed measured cylinder heat-loss factor
A PasHub survey can lodge a manufacturer's measured cylinder loss ("What is
the cylinder measured heat loss: 0.94 kWh/24 hours") — the SAP 10.2 §4
(48)-(50) declared-loss factor that must override the Table 2 V×L×VF
insulation computation. The site-notes path dropped it two ways: the
extractor looked for "Cylinder Measured Heat Loss:" (matches 0 fixtures) while
the PDF label is "What is the cylinder measured heat loss:" (92 fixtures), and
the mapper never wired `cylinder_heat_loss` even when captured. So the
calculator fell back to the age-band insulation default — a lossier cylinder
than surveyed — over-costing hot water and under-rating SAP.

Fix the extractor label and add `_pashub_cylinder_measured_heat_loss` (parses
the leading float, returns None for "Not known"), mirroring the gov-API and
Elmhurst paths that already pass `cylinder_heat_loss` through. 3 Woodmans
Court -1.33 → -0.09; LRHA cohort within-0.5 58.3% → 60.2%, MAE 0.515 → 0.496.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-28 09:22:05 +00:00