Commit graph

1297 commits

Author SHA1 Message Date
Daniel Roth
be1d6faea0 Complete a rename run that hit locked files or missing folders, and reject an unusable site 🟩
Green on arrival: both fall out of the summary the run now returns and of
the guarded site lookup. Pinned after verifying they bite — an unguarded
DomnaSites[name] completes an "ECO" run that renames files under a
different site's alias.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 09:03:43 +00:00
Daniel Roth
0836b18d5b Attribute a rename run to its request and target site 🟩
Green on arrival: @task_handler records inputs and reads source_id from
the body key matching the source's value. Pinned because it is the only
place the new sharepoint_site member is proven against a real Postgres
enum rather than a Python one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 09:01:51 +00:00
Daniel Roth
1213b80e3f Honour dry_run on a queue-delivered rename request 🟩
Green on arrival: reading the request from the SQS record body is what
taking the task lane gave us. Verified the guard bites by restoring the
raw-event read, under which the "dry" run renames a live file.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:59:45 +00:00
Daniel Roth
e7785f6b76 Record a rename run's result summary on its sub_task 🟥
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:58:18 +00:00
Daniel Roth
5309c1f097 Merge subfolder rename results into their parent folder's summary 🟩
Passed on arrival: threading the summary back up through the traversal
already carried subfolder results. Pinned so the merge cannot regress.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:51:36 +00:00
Daniel Roth
2aa698da1e Count files skipped as already canonical apart from skipped images 🟥
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:50:45 +00:00
Daniel Roth
a69d603cde Record each file SharePoint refused to rename, with its error 🟥
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:49:17 +00:00
Daniel Roth
9e48f39fa5 List the UPRNs with no SharePoint folder under any root 🟥
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:45:20 +00:00
Daniel Roth
0135e71590 Count dry-run rename intents separately from real renames 🟥
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:44:20 +00:00
Daniel Roth
391e7bee37 Report how many files a rename run renamed 🟩
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:43:31 +00:00
Daniel Roth
5dc524be8f Report how many files a rename run renamed 🟥
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:42:33 +00:00
Jun-te Kim
82e1dd555a
Merge pull request #1700 from Hestia-Homes/investigatin/bugs_for_esh
Recognise "complete" as a MAIN building part, and anchor lone parts
2026-07-28 18:25:50 +01:00
KhalimCK
3afc43f484
Merge pull request #1691 from Hestia-Homes/fix/default-plan-refresh-on-rerun
Refresh a property's default Plan when its owning scenario is re-run
2026-07-28 18:13:02 +01:00
Jun-te Kim
bb5746c9b5 Recognise "complete" as a MAIN building part, and anchor lone parts 🟪
Six properties failed task 4d006fab (portfolio 854 / scenario 1334) with
DegeneratePredictionError. None has a lodged EPC, so each was predicted from a
neighbouring-cert cohort — and every cohort cert lodges its single building part
as `identifier: "complete"`. from_api_string accepted only "Main Dwelling" /
"Main building", so "complete" fell to OTHER, _has_main_part returned False, and
prediction refused to proceed.

"complete" is what SAP 9.91/9.92-era software writes for an unextended dwelling:
semantically identical to "Main Dwelling". Casing varies within a single postcode
(cert 8824-7422-1180-6934-1902 lodges "complete", 199 Highfield Road "Complete"),
so matching is now case-folded.

The API's `identifier` is free text with no schema enum — api.yml declares the
whole cert body `additionalProperties: true` and documents no field of it, and
`identifier` is absent from the 17 vocabularies at /api/codes. So the recognised
set can only grow by discovery, and each miss has cost an incident ("Main
building" → task a40e71c4; "complete" → task 4d006fab). Hence three changes, not
one:

- "complete" → MAIN, case-insensitively.
- Bare "Extension" → EXTENSION_1. The regex required a digit, so 668 corpus
  occurrences silently dropped a real extension from the structure. This reverses
  a prior deliberate pin whose stated rationale (RdSAP10 §1.2's 4-extension cap)
  does not apply — there is no out-of-range number in a bare "Extension".
- _anchor_lone_building_part: a cert lodging exactly ONE part describes the whole
  dwelling whatever it is called, so an unclassified lone part becomes MAIN.
  Multi-part certs are untouched — there the identifier carries real information
  and guessing would invent structure the cert does not state.

Also removes a false claim from DegeneratePredictionError's message: it asserted
the template was "lodged with a null part identifier", which was hardcoded, never
checked against a real cert, and wrong. It misdirected this investigation.

Verified against the live gov API: all 13 cohort certs across BL1 8EB, BL3 6XJ
and BL4 0RA now resolve a MAIN part. The full modelling e2e was NOT run (no local
AWS creds for the geospatial lookup) — this clears the blocking error, it does not
prove the predictions are good.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-28 15:12:01 +00:00
Jun-te Kim
79bf28d8cb Offer EWI/IWI on a gov-API system-built wall (code 8) 🟥
The gov-EPC API lodges a system-built wall as `wall_construction == 8`
per the authoritative GOV.UK RdSAP `WallConstructionCode` enumeration
(communitiesuk/epb-data-warehouse `.../SAP-Domains.xsd`: 8 = "system
built" in every schema version 17.0-21.0.1; park home is 10). The solid-
wall Recommendation Generator instead read code 8 as the Elmhurst
park-home label and suppressed the whole cohort's EWI/IWI Recommendation
(a 2026 DB sweep finds ~5.1k certs at code 8, ~2.1k uninsulated main
walls). Rewrite the mislabelled park-home test to pin the correct
behaviour (code 8 → EWI+IWI), and add a genuine park-home test on the
gov code 10 so the fix doesn't start over-offering on real park homes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DX8oAGsGkBHq3U4dsxYRzz
2026-07-28 10:33:06 +00:00
Daniel Roth
fdccad9c37 Fetch every evidence page when a job has more files than one page holds 🟪
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 10:12:25 +00:00
Daniel Roth
f02441b6c4 Raise truncation error when PasHub stops serving pages before the full evidence set arrives 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 10:11:44 +00:00
Daniel Roth
5a85403c3d Fetch every evidence page when a job has more files than one page holds 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 10:06:00 +00:00
Khalim Conn-Kowlessar
0ca8ca31a0 Refresh a property's default Plan when its owning scenario is re-run
A Plan became default only when its Scenario was flagged default or the
property had no Plan yet. Scenarios are created is_default=False, so in
practice only the first-ever Plan was ever promoted — re-modelling a
property appended a new Plan but left the default frozen on the first run,
so a re-model never surfaced on the FE (property 792168 kept its 2026-07-22
scenario-1328 default through three same-scenario re-runs).

Also promote when the re-run models the SAME scenario that currently owns
the property's default, so a re-run refreshes the shown Plan in place (the
save demotes the stale one). A re-run of a different scenario still does not
hijack the default. New PlanRepository.default_plan_scenario_by_property
surfaces each property's default scenario for the decision.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-27 16:53:24 +00:00
KhalimCK
035d2f5136
Merge pull request #1687 from Hestia-Homes/fix/epc-property-per-property-persistence
EPC persistence: write a per-property lodged copy, and refresh a stale stored cert on refetch
2026-07-27 16:44:55 +01:00
Jun-te Kim
dc2913b7db
Merge pull request #1688 from Hestia-Homes/fix/sap17-1-community-heating
Map full-SAP community heating (heat networks) instead of dropping it
2026-07-27 15:53:11 +01:00
Khalim Conn-Kowlessar
de08d93369 Refresh a stale stored gov cert on a refetch when its content differs
refetch_epc=True fetched a fresh cert but the recency tie-break (strict >)
dropped it whenever the stored row shared the cert's inspection date — so a
row written before a schema field/table existed (e.g. room-in-roof geometry)
could never be healed by re-fetching. The dwelling was modelled on stale
fabric indefinitely.

Reconcile instead of just comparing dates: on an equal inspection date,
refresh a public gov cert (uploaded_file_id NULL) when the fetched cert
differs from the stored one, so a plain re-fetch re-ingests it. One of our
own surveys (PasHub / ECMK / Elmhurst — uploaded_file_id set) is preserved on
the tie, keeping the survey-wins rule (ADR-0001 / #1589); an unchanged public
cert is still not re-persisted.

`_reconcile_lodged` layers the content refresh over `_newer_lodged` (recency
untouched). New `EpcRepository.survey_lodged_uprns` surfaces the row-level
uploaded_file_id provenance the hydrated EpcPropertyData drops. Together with
the property_id-anchored persist, a refetch_epc=True re-run now heals both a
missing epc_property row and a stale one.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-27 14:34:26 +00:00
Daniel Roth
7680d9346e
Merge pull request #1686 from Hestia-Homes/feature/abri-api-void-job
Abri API: Log empty properties in hubspot as Void
2026-07-27 15:21:07 +01:00
Daniel Roth
5c596627ef Space the void marker off the existing note exactly once 🟩
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 14:06:34 +00:00
Daniel Roth
ed43ed48ca "Void. " not "Void ||" 2026-07-27 13:40:47 +00:00
Khalim Conn-Kowlessar
2b22b0f2ef Persist a property_id-anchored lodged EPC when the property owns no row
The lodged EPC read keys on UPRN (ADR-0001 recency tie-break), but writes
and the front end both key on property_id. When a dwelling is onboarded
under two portfolios, the shared UPRN surfaces the sibling's lodged row,
so `lodged_epc_is_new` stays False and the modelling run skips persisting
a copy for this property — leaving it with no epc_property row the FE can
read. The EPC is modelled but invisible (portfolio 850 / property 792235,
uprn 100021979925, also under property 753896 in portfolio 830).

Drive the lodged persist off "does THIS property own a lodged row?"
(property_id-scoped) rather than "is the cert new?" (UPRN-scoped): add
`EpcRepository.property_ids_with_lodged_epc` and persist a per-property
copy whenever the cert is new OR the property has no lodged row of its
own. Idempotent (save_batch is replace-by-property_id), and the recency
skip still holds for a property that already owns its row.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-27 13:39:31 +00:00
Daniel Roth
b77769fdca Leave an already-flagged deal alone and fail loudly on a failed flag 🟩
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 13:34:50 +00:00
Daniel Roth
f742795434 Flag a void property at the front of the deal's booking information 🟥
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 13:28:50 +00:00
Jun-te Kim
efd171fd55 Pin full-SAP community cert recovering out of band F end-to-end 🟩
Apt 2 (Tracey's Place) through from_api_response -> cert_to_inputs ->
calculate_sap_from_inputs: priced as mains gas (fuel 51), SAP 67 D — recovered
from the pre-fix 27 F. Accredited Elmhurst RdSAP is 62 D; lodged 71 C.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 13:19:41 +00:00
Jun-te Kim
58084a316f Map full-SAP community CHP + heat-pump networks 🟥
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 13:17:54 +00:00
Jun-te Kim
cca2c18ef9 Map full-SAP community boilers to SAP code 301 🟥
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 13:14:44 +00:00
Jun-te Kim
86043e267c Map full-SAP community heat source to its mains-gas fuel 🟥
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 13:11:02 +00:00
KhalimCK
f4932569ba
Merge pull request #1685 from Hestia-Homes/feat/1639-pashub-lrha-wave3-residual
PasHub LRHA Wave 3: close residual mapper/calculator gaps (#1639)
2026-07-27 14:01:29 +01:00
Khalim Conn-Kowlessar
ca27f0b9d8 Charge an oil storage-combi its PCDB internal-store water-heating loss 🟩
An oil STORAGE combi (Worcester Greenstar Heatslave II, PCDB 18415,
store_type 1) zeroed its internal-store loss: pcdb_combi_loss_override
defers on store_type in {1,2,3} and a combi has no cylinder, so line
(56) dropped to zero — under-costing hot water and over-rating SAP by
+2.0..+4.9 across the five LRHA WAVE 3 oil combis.

SAP 10.2 Table 2 note b) now includes the store loss when the combi's
efficiency is from the PCDB: the PCDB store volume (field 42) and
insulation thickness (field 44) feed Table 2/2a with the Table 2b
storage-combi temperature factor, reproducing the accredited Elmhurst
P960 worksheet exactly (5.4175 kWh/day, Jan 167.94, ~1977 kWh/yr).

Cohort: within-0.5 55.3% -> 58.3%, MAE 0.675 -> 0.531; ratchet
tightened accordingly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-27 12:43:52 +00:00
Daniel Roth
00adbcd519 Complete the task for a void property instead of dead-lettering it 🟥
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 11:26:12 +00:00
Daniel Roth
a3e1174d0d Complete the tenant sync without contacts when the property is void 🟥
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 11:24:49 +00:00
Daniel Roth
6ee560af29 Keep the query-dump form of "no tenancy found" a rejection 🟩
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 11:23:49 +00:00
Daniel Roth
a7ae408ab2 Report a void property when Abri finds no tenancy for the place 🟥
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 11:22:28 +00:00
Daniel Roth
ab7b5e388f
Merge pull request #1683 from Hestia-Homes/feature/abri-api-tweaks
Key Abri log/amend/abandon off client_booking_reference, not the survey-date edge
2026-07-27 12:10:22 +01:00
Khalim Conn-Kowlessar
a21d2cc579 Bill an electric second main at its own §12/Table-12a rate 🟩
Closes #1684. `_main_2_space_heating_fuel_cost_gbp_per_kwh` billed an
electric Main 2 at Main 1's space-heating rate. That was a no-op on a
homogeneous off-peak cohort (storage+storage, direct+direct → equal
rates) but mis-billed the HETEROGENEOUS case: an off-peak STORAGE Main 1
(5.5 p/kWh low) + an on-peak DIRECT-ACTING panel Main 2 (15.29 p/kWh
high) rode the storage low rate, under-costing the panel half and
over-rating SAP.

Flip it to bill Main 2 at its OWN Table 12a Grid 1 SH row, the same
per-system resolution Main 1 and a non-electric Main 2 already use — and
the same one `_main_2_high_rate_fraction` already applies, so the scalar
cost and the high-rate split are now consistent.

The docstring's "regresses certs 13/34 (Parkers Hill / Dunley Road)"
warning was stale: those names appear nowhere but that docstring, and
the full guard suite shows zero new regressions (only the two
pre-existing `test_heating_systems_corpus` fails on main). LRHA WAVE 3
fixture 497655371974 (storage+panel) moves +15.3 → -2.5 vs PasHub; cohort
MAE 0.799 → 0.675 (ratchet re-baselined 0.80 → 0.68). within-0.5 holds at
55.3% — the -2.5 residual is PasHub-vs-Elmhurst divergence, not a bug, so
the fixture stays out of the within-0.5 pin. A ground-truth Elmhurst pin
is queued (see docs/HANDOVER_1684_ELMHURST_INPUTS_497655371974.md).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-27 09:06:20 +00:00
Daniel Roth
fb64f9687e Fail non-retriably when amending or abandoning a deal that never logged to Abri 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 15:53:50 +00:00
Daniel Roth
92fc9052e5 Carry the client_booking_reference on the Abri trigger message 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 15:37:22 +00:00
Khalim Conn-Kowlessar
398692703f Bill secondary electric heating at the dwelling's §12 tariff, not the meter default 🟩
An electric room/panel-heater dwelling (main SAP code 691) on a Dual meter is a
10-hour tariff by RdSAP 10 §12 Rule 3, so its main heating bills the secondary's
identical electricity at a 0.50-blended 11.09 p/kWh. Its SECONDARY heating,
however, resolved its tariff from the bare meter string — where a Dual meter
defaults to 7-hour (`tariff_from_meter_type`) — and billed at the 7-hour
all-high-rate 15.29 p/kWh. The ~10% secondary fraction was over-priced by ~4 p/kWh,
under-rating SAP by ~1 point across the LRHA WAVE 3 code-691 cohort.

Route the dwelling's resolved `_rdsap_tariff(epc)` into `_secondary_fuel_cost_gbp_
per_kwh` / `_secondary_off_peak_rate_gbp_per_kwh`, exactly as the main-heating cost
path (`_space_heating_fuel_cost_gbp_per_kwh`) and the secondary CO2 path
(`_secondary_heating_co2_factor_kg_per_kwh`) already do — removing the internal
meter-only resolution that diverged from the dwelling's single §12 tariff.

Storage-heater dwellings (7-hour by §12) are unchanged — their secondary correctly
stays at 15.29 p (regression-pinned in test_cert_to_inputs). LRHA WAVE 3 within-0.5
49.5% → 52.4%, MAE 0.977; gov-API, Elmhurst RealCert, Guinness and Wythenshawe
corpora unaffected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 14:04:08 +00:00
KhalimCK
68a147edac
Merge pull request #1678 from Hestia-Homes/feat/boiler-fuel-archetypes
Oil & solid-fuel boiler archetypes, fuel-resolved (issue #1676)
2026-07-24 14:32:55 +01:00
KhalimCK
d5b94996df
Merge pull request #1677 from Hestia-Homes/feat/roof-sloping-ceiling
Pitched sloping-ceiling roof-type family (closes #1676)
2026-07-24 14:32:42 +01:00
KhalimCK
44072ed159
Merge pull request #1674 from Hestia-Homes/fix/epc-persistence-dropped-fields
EPC persistence blind spot: persist dropped calculator-read fields + round-trip gate (#1665)
2026-07-24 14:32:17 +01:00
KhalimCK
206ac197ee
Merge pull request #1544 from Hestia-Homes/feat/calico-stock-condition-ingestion
Calico roof-covering stock-condition ingestion (ADR-0064)
2026-07-24 14:31:54 +01:00
Khalim Conn-Kowlessar
abbad2b44d Resolve the boiler archetype by fuel when building a property's overlays 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 09:19:31 +00:00
Khalim Conn-Kowlessar
abec38a397 Re-point a fuel-agnostic gas boiler to oil or solid fuel by main_fuel 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 09:18:06 +00:00
Khalim Conn-Kowlessar
04774a8888 Map oil and solid-fuel boilers to their SAP codes and companion set 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 09:14:33 +00:00