Commit graph

1315 commits

Author SHA1 Message Date
Khalim Conn-Kowlessar
063eb17a89 feat(overlay): enable gas A/B bands — Elmhurst-validated (winter,summer)
Gas A = (90.6, 81.6) [PCDB 18790 Worcester Greenstar 8000 Life], B = (88.0, 79.4)
[PCDB 15029 Baxi Duo-tec Combi 24 HE], read off accredited Elmhurst RdSAP10
worksheets (206)/(217). Applied to gas regular (102) + combi (104); gas/LPG/biogas
share the codes. A-rated real cert 15017550: +1.78 SAP (the under-credit fix).
Oil A/B still pending (no slot). Corpus unmoved; 797 tests green (ADR-0068).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-29 18:37:10 +00:00
Khalim Conn-Kowlessar
c690cd3183 feat(backfill): boiler_efficiency_band rows from existing heating descriptions
Idempotent, dry-run-by-default script (mirrors reclassify_main_heating). Parses
the SEDBUK band off each main_heating_system row's original_spreadsheet_description
with the SAME guard as the live classifier (no drift), upserting a
boiler_efficiency_band row per boiler that carries one. Pure core band_backfill_rows
unit-tested. FE-enum-gated writes (Class-A/B deferred) (ADR-0068).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-29 16:34:53 +00:00
Khalim Conn-Kowlessar
291717ca8f test(finaliser): RED — boiler_efficiency_band alongside heating; UNKNOWN band skips
The band rides the Heating source column and yields its own override row; unlike
the mandatory components an UNKNOWN band is a legitimate no-value and must be
skipped, not fail the finalise (ADR-0068).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-29 16:26:47 +00:00
Khalim Conn-Kowlessar
e91e714221 test(overrides): RED — thread the Boiler Efficiency Band to the overlay call site
Parse-on-read from the main_heating_system row's original_spreadsheet_description
(ships before FE); an explicit boiler_efficiency_band override row wins when
present; band applied after fuel resolution (oil re-point); no band -> no slot.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-29 16:18:53 +00:00
Khalim Conn-Kowlessar
be0fb08e65 test(calculator): RED — overlay (winter,summer) slot wins over Table 4b code
The override-supplied seasonal efficiency must take precedence in both the §206
winter path (main_heating_efficiency) and the Appendix D Eq D1 summer water path,
the same precedence slot the PCDB winter_efficiency_pct occupies (ADR-0068).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-29 16:13:39 +00:00
Khalim Conn-Kowlessar
7a35d177b2 test(overlay): RED — band -> (winter,summer) slot on gas/oil boiler overlay
C-and-below reuse accredited Table 4b pairs; A/B pending Elmhurst (no slot yet,
no invented numbers); non-banded boilers (solid/electric/CPSU) ignore the band.
The base code is unchanged; only the efficiency slot is set (ADR-0068).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-29 16:06:43 +00:00
Khalim Conn-Kowlessar
4844d030d7 test(overrides): RED — BoilerEfficiencyBand enum + SEDBUK band parser
Boiler Efficiency Band is the 5th Heating Companion (ADR-0068). The shared
parser extracts the SEDBUK A-G band from the Landlord 'Heating' description
(`Boiler: <A-G> rated ...`), primary-system-1 on multi-system cells, None
when absent. Single source of truth for read/backfill/classifier.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-29 16:03:14 +00:00
Jun-te Kim
269fcf6e47 Persist the water-only community DHW network across the DB round-trip 🟩
The two new `SapHeating` fields were mapped from the API but dropped by
`EpcPostgresRepository._compose`, which the ADR-0036 structural guard caught.

Allow-listing them would have been wrong. `PropertyBaselineOrchestrator.run`
(applications/modelling_e2e/handler.py:885) always re-derives the persisted
baseline from the composed object, so an unpersisted field means the deployed
baseline keeps pricing heat-pump DHW as direct electric — the Dovestone cert
would still land at SAP 67 even with the mapper fix in place. Several plan
paths read the stored EPC too (refetch_epc=False, no lodged cert, stored cert
newer, stored survey wins).

Round-trip fidelity also matters for churn: `_reconcile_lodged` compares
`fetched != stored`, so a field that survives the API mapper but not `_compose`
makes that inequality permanently true and re-saves every public cert forever.

Needs the matching Drizzle columns in assessment-model before deploy.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 14:32:57 +00:00
Jun-te Kim
3fe4c3856a Record exact engine↔Elmhurst convergence at SAP 80 for the Dovestone cert 🟪
A faithfully-built Elmhurst RdSAP10 (AP50 3.2, MVHR unit 500167, phantom
secondary + open-flue cleared) reproduces the engine EXACTLY: worksheet 80 =
engine 80, fabric heat loss 41.2 vs 41.6 W/K. The earlier 74 was Elmhurst
data-entry fidelity (four artefacts), not calculator error. Refresh the saved
worksheet/summary PDFs to the SAP-80 build; pin unchanged at 80.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DX8oAGsGkBHq3U4dsxYRzz
2026-07-29 14:11:52 +00:00
Jun-te Kim
2f8c086310 Record the DHW-energy residual investigation: not a bug 🟪
Root-caused the 2675-vs-1839 kWh communal-DHW delta: the engine correctly
applies the SAP §4.3 distribution-loss factor (1/(1.05×1.50)); the difference is
the DHW energy content (64) alone. The engine's 1698 kWh tracks the cert's own
lodged RHI water heating (1909) far better than Elmhurst's reduced-data 1168,
and its £113 is closer to the cert's lodged £147 than Elmhurst's £78 — so the
engine is more faithful to the cert here, not buggy. Comment accuracy only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DX8oAGsGkBHq3U4dsxYRzz
2026-07-29 14:11:52 +00:00
Jun-te Kim
6e6c27076e Correct the DHW-energy residual note: no cylinder loss on communal DHW 🟪
The dwelling has no hot-water cylinder (has_hot_water_cylinder false), so the
2675-vs-1839 kWh DHW-energy gap is not a cylinder storage loss as previously
stated — it is an un-root-caused SAP §4.3 community-DHW demand/distribution-loss
difference. No behaviour change; comment accuracy only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DX8oAGsGkBHq3U4dsxYRzz
2026-07-29 14:11:52 +00:00
Jun-te Kim
db274adcb3 Pin the Dovestone water-only community DHW cert recovering to SAP 80 🟩
Full-SAP SAP-Schema-18.0.0 cert 9556-3047-3304-5355-1200 (Dovestone Gardens
Apt 4): communal heat-pump DHW + direct-electric room heaters. Was 67 (D) with
DHW mis-billed as direct-electric immersion; now 80 with DHW on the community
heat-pump fuel. Accredited Elmhurst worksheet 74 (evidence saved); the +6 is the
documented full-SAP measured-U/MVHR vs RdSAP age-band residual.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DX8oAGsGkBHq3U4dsxYRzz
2026-07-29 14:11:52 +00:00
Daniel Roth
35a3ebe0e3 Complete a property whose file SharePoint rejected 🟩
Green on arrival — rename() records rejections rather than raising.
Pinned after verifying it bites: letting the rejection escape fails the
child sub_task, which turns the whole Task red.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 09:53:58 +00:00
Daniel Roth
5b1b319c2d Give each property that needs renaming its own sub_task 🟩
Properties are planned and renamed one at a time, so only those with work
get a row and a run killed by the timeout still leaves every property it
reached both renamed and recorded.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 09:52:30 +00:00
Daniel Roth
cc07cc0114 Plan each property's renames before applying any of them 🟩
The walk now yields a PropertyPlan per address-list row, and rename()
applies one. run() composes the two, so single-pass callers are unchanged.
Knowing a property's size before the first rename is what makes "when is
this property finished?" answerable.

Tests that reached through the old private _process_folder are rewritten
against the public run(), so they survive this kind of change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 09:50:28 +00:00
Daniel Roth
737d07a951 Plan each property's renames before applying any of them 🟥
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 09:47:20 +00:00
Daniel Roth
bf1f5693be Build the handler test's address list without a shared mutable default 🟪
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 09:21:11 +00:00
Daniel Roth
bbb3c9e057 Verify a local rename run the way the queue delivers it 🟪
The local invoke now sends an SQS-shaped payload — a direct invoke is the
one shape that cannot reproduce the dry_run bug class. Test modules route
protected-method calls through typed helpers so both pass pyright strict.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 09:19:51 +00:00
Daniel Roth
be1d6faea0 Complete a rename run that hit locked files or missing folders, and reject an unusable site 🟩
Green on arrival: both fall out of the summary the run now returns and of
the guarded site lookup. Pinned after verifying they bite — an unguarded
DomnaSites[name] completes an "ECO" run that renames files under a
different site's alias.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 09:03:43 +00:00
Daniel Roth
0836b18d5b Attribute a rename run to its request and target site 🟩
Green on arrival: @task_handler records inputs and reads source_id from
the body key matching the source's value. Pinned because it is the only
place the new sharepoint_site member is proven against a real Postgres
enum rather than a Python one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 09:01:51 +00:00
Daniel Roth
1213b80e3f Honour dry_run on a queue-delivered rename request 🟩
Green on arrival: reading the request from the SQS record body is what
taking the task lane gave us. Verified the guard bites by restoring the
raw-event read, under which the "dry" run renames a live file.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:59:45 +00:00
Daniel Roth
e7785f6b76 Record a rename run's result summary on its sub_task 🟥
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:58:18 +00:00
Daniel Roth
5309c1f097 Merge subfolder rename results into their parent folder's summary 🟩
Passed on arrival: threading the summary back up through the traversal
already carried subfolder results. Pinned so the merge cannot regress.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:51:36 +00:00
Daniel Roth
2aa698da1e Count files skipped as already canonical apart from skipped images 🟥
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:50:45 +00:00
Daniel Roth
a69d603cde Record each file SharePoint refused to rename, with its error 🟥
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:49:17 +00:00
Daniel Roth
9e48f39fa5 List the UPRNs with no SharePoint folder under any root 🟥
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:45:20 +00:00
Daniel Roth
0135e71590 Count dry-run rename intents separately from real renames 🟥
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:44:20 +00:00
Daniel Roth
391e7bee37 Report how many files a rename run renamed 🟩
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:43:31 +00:00
Daniel Roth
5dc524be8f Report how many files a rename run renamed 🟥
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:42:33 +00:00
Jun-te Kim
82e1dd555a
Merge pull request #1700 from Hestia-Homes/investigatin/bugs_for_esh
Recognise "complete" as a MAIN building part, and anchor lone parts
2026-07-28 18:25:50 +01:00
KhalimCK
3afc43f484
Merge pull request #1691 from Hestia-Homes/fix/default-plan-refresh-on-rerun
Refresh a property's default Plan when its owning scenario is re-run
2026-07-28 18:13:02 +01:00
Jun-te Kim
bb5746c9b5 Recognise "complete" as a MAIN building part, and anchor lone parts 🟪
Six properties failed task 4d006fab (portfolio 854 / scenario 1334) with
DegeneratePredictionError. None has a lodged EPC, so each was predicted from a
neighbouring-cert cohort — and every cohort cert lodges its single building part
as `identifier: "complete"`. from_api_string accepted only "Main Dwelling" /
"Main building", so "complete" fell to OTHER, _has_main_part returned False, and
prediction refused to proceed.

"complete" is what SAP 9.91/9.92-era software writes for an unextended dwelling:
semantically identical to "Main Dwelling". Casing varies within a single postcode
(cert 8824-7422-1180-6934-1902 lodges "complete", 199 Highfield Road "Complete"),
so matching is now case-folded.

The API's `identifier` is free text with no schema enum — api.yml declares the
whole cert body `additionalProperties: true` and documents no field of it, and
`identifier` is absent from the 17 vocabularies at /api/codes. So the recognised
set can only grow by discovery, and each miss has cost an incident ("Main
building" → task a40e71c4; "complete" → task 4d006fab). Hence three changes, not
one:

- "complete" → MAIN, case-insensitively.
- Bare "Extension" → EXTENSION_1. The regex required a digit, so 668 corpus
  occurrences silently dropped a real extension from the structure. This reverses
  a prior deliberate pin whose stated rationale (RdSAP10 §1.2's 4-extension cap)
  does not apply — there is no out-of-range number in a bare "Extension".
- _anchor_lone_building_part: a cert lodging exactly ONE part describes the whole
  dwelling whatever it is called, so an unclassified lone part becomes MAIN.
  Multi-part certs are untouched — there the identifier carries real information
  and guessing would invent structure the cert does not state.

Also removes a false claim from DegeneratePredictionError's message: it asserted
the template was "lodged with a null part identifier", which was hardcoded, never
checked against a real cert, and wrong. It misdirected this investigation.

Verified against the live gov API: all 13 cohort certs across BL1 8EB, BL3 6XJ
and BL4 0RA now resolve a MAIN part. The full modelling e2e was NOT run (no local
AWS creds for the geospatial lookup) — this clears the blocking error, it does not
prove the predictions are good.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-28 15:12:01 +00:00
Jun-te Kim
79bf28d8cb Offer EWI/IWI on a gov-API system-built wall (code 8) 🟥
The gov-EPC API lodges a system-built wall as `wall_construction == 8`
per the authoritative GOV.UK RdSAP `WallConstructionCode` enumeration
(communitiesuk/epb-data-warehouse `.../SAP-Domains.xsd`: 8 = "system
built" in every schema version 17.0-21.0.1; park home is 10). The solid-
wall Recommendation Generator instead read code 8 as the Elmhurst
park-home label and suppressed the whole cohort's EWI/IWI Recommendation
(a 2026 DB sweep finds ~5.1k certs at code 8, ~2.1k uninsulated main
walls). Rewrite the mislabelled park-home test to pin the correct
behaviour (code 8 → EWI+IWI), and add a genuine park-home test on the
gov code 10 so the fix doesn't start over-offering on real park homes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DX8oAGsGkBHq3U4dsxYRzz
2026-07-28 10:33:06 +00:00
Daniel Roth
fdccad9c37 Fetch every evidence page when a job has more files than one page holds 🟪
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 10:12:25 +00:00
Daniel Roth
f02441b6c4 Raise truncation error when PasHub stops serving pages before the full evidence set arrives 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 10:11:44 +00:00
Daniel Roth
5a85403c3d Fetch every evidence page when a job has more files than one page holds 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 10:06:00 +00:00
Khalim Conn-Kowlessar
0ca8ca31a0 Refresh a property's default Plan when its owning scenario is re-run
A Plan became default only when its Scenario was flagged default or the
property had no Plan yet. Scenarios are created is_default=False, so in
practice only the first-ever Plan was ever promoted — re-modelling a
property appended a new Plan but left the default frozen on the first run,
so a re-model never surfaced on the FE (property 792168 kept its 2026-07-22
scenario-1328 default through three same-scenario re-runs).

Also promote when the re-run models the SAME scenario that currently owns
the property's default, so a re-run refreshes the shown Plan in place (the
save demotes the stale one). A re-run of a different scenario still does not
hijack the default. New PlanRepository.default_plan_scenario_by_property
surfaces each property's default scenario for the decision.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-27 16:53:24 +00:00
KhalimCK
035d2f5136
Merge pull request #1687 from Hestia-Homes/fix/epc-property-per-property-persistence
EPC persistence: write a per-property lodged copy, and refresh a stale stored cert on refetch
2026-07-27 16:44:55 +01:00
Jun-te Kim
dc2913b7db
Merge pull request #1688 from Hestia-Homes/fix/sap17-1-community-heating
Map full-SAP community heating (heat networks) instead of dropping it
2026-07-27 15:53:11 +01:00
Khalim Conn-Kowlessar
de08d93369 Refresh a stale stored gov cert on a refetch when its content differs
refetch_epc=True fetched a fresh cert but the recency tie-break (strict >)
dropped it whenever the stored row shared the cert's inspection date — so a
row written before a schema field/table existed (e.g. room-in-roof geometry)
could never be healed by re-fetching. The dwelling was modelled on stale
fabric indefinitely.

Reconcile instead of just comparing dates: on an equal inspection date,
refresh a public gov cert (uploaded_file_id NULL) when the fetched cert
differs from the stored one, so a plain re-fetch re-ingests it. One of our
own surveys (PasHub / ECMK / Elmhurst — uploaded_file_id set) is preserved on
the tie, keeping the survey-wins rule (ADR-0001 / #1589); an unchanged public
cert is still not re-persisted.

`_reconcile_lodged` layers the content refresh over `_newer_lodged` (recency
untouched). New `EpcRepository.survey_lodged_uprns` surfaces the row-level
uploaded_file_id provenance the hydrated EpcPropertyData drops. Together with
the property_id-anchored persist, a refetch_epc=True re-run now heals both a
missing epc_property row and a stale one.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-27 14:34:26 +00:00
Daniel Roth
7680d9346e
Merge pull request #1686 from Hestia-Homes/feature/abri-api-void-job
Abri API: Log empty properties in hubspot as Void
2026-07-27 15:21:07 +01:00
Daniel Roth
5c596627ef Space the void marker off the existing note exactly once 🟩
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 14:06:34 +00:00
Daniel Roth
ed43ed48ca "Void. " not "Void ||" 2026-07-27 13:40:47 +00:00
Khalim Conn-Kowlessar
2b22b0f2ef Persist a property_id-anchored lodged EPC when the property owns no row
The lodged EPC read keys on UPRN (ADR-0001 recency tie-break), but writes
and the front end both key on property_id. When a dwelling is onboarded
under two portfolios, the shared UPRN surfaces the sibling's lodged row,
so `lodged_epc_is_new` stays False and the modelling run skips persisting
a copy for this property — leaving it with no epc_property row the FE can
read. The EPC is modelled but invisible (portfolio 850 / property 792235,
uprn 100021979925, also under property 753896 in portfolio 830).

Drive the lodged persist off "does THIS property own a lodged row?"
(property_id-scoped) rather than "is the cert new?" (UPRN-scoped): add
`EpcRepository.property_ids_with_lodged_epc` and persist a per-property
copy whenever the cert is new OR the property has no lodged row of its
own. Idempotent (save_batch is replace-by-property_id), and the recency
skip still holds for a property that already owns its row.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-27 13:39:31 +00:00
Daniel Roth
b77769fdca Leave an already-flagged deal alone and fail loudly on a failed flag 🟩
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 13:34:50 +00:00
Daniel Roth
f742795434 Flag a void property at the front of the deal's booking information 🟥
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 13:28:50 +00:00
Jun-te Kim
efd171fd55 Pin full-SAP community cert recovering out of band F end-to-end 🟩
Apt 2 (Tracey's Place) through from_api_response -> cert_to_inputs ->
calculate_sap_from_inputs: priced as mains gas (fuel 51), SAP 67 D — recovered
from the pre-fix 27 F. Accredited Elmhurst RdSAP is 62 D; lodged 71 C.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 13:19:41 +00:00
Jun-te Kim
58084a316f Map full-SAP community CHP + heat-pump networks 🟥
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 13:17:54 +00:00
Jun-te Kim
cca2c18ef9 Map full-SAP community boilers to SAP code 301 🟥
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 13:14:44 +00:00
Jun-te Kim
86043e267c Map full-SAP community heat source to its mains-gas fuel 🟥
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 13:11:02 +00:00