test_override_component_consistency caught two real gaps: the band was in the
override_component pgEnum but (a) missing from _ROW_TYPES — so the finaliser
couldn't load its classified vocab (a latent bug the injected-vocab finaliser
tests missed), and (b) not in _COMPONENT_OVERLAYS (correct — it's an attribute
consumed by the main_heating_system overlay, not a standalone one).
Add the band to _ROW_TYPES (reader), and introduce _ATTRIBUTE_COMPONENTS to name
the classified+stored-but-no-standalone-overlay case. The guard now asserts:
pgEnum == _ROW_TYPES; and _ROW_TYPES minus attribute components == _COMPONENT_OVERLAYS.
Full repositories suite (247) + finaliser green; pyright clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Aligns the historical backfill with the go-forward finaliser: it now writes a
boiler_efficiency_band row for EVERY main_heating_system row — the parseable band
(A-G) where present, else an explicit Unknown — so historical and go-forward data
match. Unknown is fine on non-boilers (inert to modelling). Confirmed with Khalim.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
kimjunte + Khalim alignment: when the band column is mapped and a description
can't be resolved to A-G, write the row with an explicit Unknown value rather
than silently skipping. The landlord declared the column, so every row gets a
value; 'assessed, no band' is now distinct from 'never assessed'. Still never
fails the finalise (unlike the mandatory components). Modelling reads Unknown as
no band (no efficiency anchor), so it stays inert to SAP.
Requires the FE boiler_efficiency_band pgEnum to include 'Unknown' (assessment-
model#486 updated). 7 finaliser tests green; pyright clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses reviewer feedback:
- #3 (Khalim + kimjunte): the boiler_efficiency_band classifier column is now
GUARD-ONLY — LLM fallback removed. The band format is deterministic, so an LLM
guess could only fabricate a band the landlord never stated (moves SAP +
eligibility). New LoggingUnknownColumnClassifier is the non-fabricating fallback:
maps guard-misses to UNKNOWN (never stored) and logs them for review — kimjunte's
'warn on None', placed in the classifier path not the shared guard (which sees
legitimate None constantly on the modelling/backfill paths).
- #2 (Khalim): backfill --apply now reports rows ACTUALLY written (upsert
rowcount), not the candidate count, so an idempotent re-run reports 0.
- #1 (Khalim): overlay module docstring scrubbed of stale 'slot/pending' wording
to match the cert-native anchor mechanism.
kimjunte's finaliser optional-skip question (declared-vs-shared-column flow) left
for reviewer alignment, not changed. 529 tests green; pyright clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reviewer feedback: seasonal_efficiency_override_pct had no lodged-cert analogue
and polluted the EpcPropertyData/MainHeatingDetail cert datatype. Replace it with
the cert-native efficiency anchors a real cert already uses:
- C–G -> sap_main_heating_code = the band's Table 4b code (combi/regular
preserved by the code itself); resolved by the existing Table 4b path.
- A/B -> main_heating_index_number = a type-matched representative PCDB product;
resolved by the existing PCDB Appendix D2.1 path (winter+summer+combi-loss).
_fold_heating clears the base code, so the effective cert reads as a real PCDB
cert. Ready for future user-supplied product -> PCDB id.
Deletes seasonal_efficiency_override_pct from MainHeatingDetail, HeatingOverlay
and _MAIN_HEATING_FIELDS, and REVERTS both cert_to_inputs branches — those four
files are now byte-identical to main (no synthesis on the cert type, no new
calculator branch). Each A/B PCDB id is CI-guarded to its accredited (winter,
summer) so a PCDB refresh fails loudly. Oil A-combi (no >90% product, ≈0 homes)
keeps its code-130 default. Real certs: A-combi +2.06 SAP (idx 18964), D-regular
-0.98 (code 106); corpus unmoved; 810 tests green (ADR-0068).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Oil A = (90.6, 82.8) [PCDB 17292 Firebird Enviromax Blue Supreme], B = (88.0, 80.2)
[PCDB 10498 Worcester Greenstar Danesmoor], applied to oil regular (127) + combi
(130). No separate oil Elmhurst build: the gas builds proved Elmhurst's worksheet
(206)/(217) for a database boiler == its PCDB winter/summer read verbatim (SAP 10.2
App D2.1), so the PCDB record is the accredited value. Corpus unmoved; 68 overlay
tests green (ADR-0068). Full A/B matrix (gas/LPG/biogas + oil) now live.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Gas A = (90.6, 81.6) [PCDB 18790 Worcester Greenstar 8000 Life], B = (88.0, 79.4)
[PCDB 15029 Baxi Duo-tec Combi 24 HE], read off accredited Elmhurst RdSAP10
worksheets (206)/(217). Applied to gas regular (102) + combi (104); gas/LPG/biogas
share the codes. A-rated real cert 15017550: +1.78 SAP (the under-credit fix).
Oil A/B still pending (no slot). Corpus unmoved; 797 tests green (ADR-0068).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Idempotent, dry-run-by-default script (mirrors reclassify_main_heating). Parses
the SEDBUK band off each main_heating_system row's original_spreadsheet_description
with the SAME guard as the live classifier (no drift), upserting a
boiler_efficiency_band row per boiler that carries one. Pure core band_backfill_rows
unit-tested. FE-enum-gated writes (Class-A/B deferred) (ADR-0068).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The band rides the Heating source column and yields its own override row; unlike
the mandatory components an UNKNOWN band is a legitimate no-value and must be
skipped, not fail the finalise (ADR-0068).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Parse-on-read from the main_heating_system row's original_spreadsheet_description
(ships before FE); an explicit boiler_efficiency_band override row wins when
present; band applied after fuel resolution (oil re-point); no band -> no slot.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The override-supplied seasonal efficiency must take precedence in both the §206
winter path (main_heating_efficiency) and the Appendix D Eq D1 summer water path,
the same precedence slot the PCDB winter_efficiency_pct occupies (ADR-0068).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
C-and-below reuse accredited Table 4b pairs; A/B pending Elmhurst (no slot yet,
no invented numbers); non-banded boilers (solid/electric/CPSU) ignore the band.
The base code is unchanged; only the efficiency slot is set (ADR-0068).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Boiler Efficiency Band is the 5th Heating Companion (ADR-0068). The shared
parser extracts the SEDBUK A-G band from the Landlord 'Heating' description
(`Boiler: <A-G> rated ...`), primary-system-1 on multi-system cells, None
when absent. Single source of truth for read/backfill/classifier.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The two new `SapHeating` fields were mapped from the API but dropped by
`EpcPostgresRepository._compose`, which the ADR-0036 structural guard caught.
Allow-listing them would have been wrong. `PropertyBaselineOrchestrator.run`
(applications/modelling_e2e/handler.py:885) always re-derives the persisted
baseline from the composed object, so an unpersisted field means the deployed
baseline keeps pricing heat-pump DHW as direct electric — the Dovestone cert
would still land at SAP 67 even with the mapper fix in place. Several plan
paths read the stored EPC too (refetch_epc=False, no lodged cert, stored cert
newer, stored survey wins).
Round-trip fidelity also matters for churn: `_reconcile_lodged` compares
`fetched != stored`, so a field that survives the API mapper but not `_compose`
makes that inequality permanently true and re-saves every public cert forever.
Needs the matching Drizzle columns in assessment-model before deploy.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A faithfully-built Elmhurst RdSAP10 (AP50 3.2, MVHR unit 500167, phantom
secondary + open-flue cleared) reproduces the engine EXACTLY: worksheet 80 =
engine 80, fabric heat loss 41.2 vs 41.6 W/K. The earlier 74 was Elmhurst
data-entry fidelity (four artefacts), not calculator error. Refresh the saved
worksheet/summary PDFs to the SAP-80 build; pin unchanged at 80.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DX8oAGsGkBHq3U4dsxYRzz
Root-caused the 2675-vs-1839 kWh communal-DHW delta: the engine correctly
applies the SAP §4.3 distribution-loss factor (1/(1.05×1.50)); the difference is
the DHW energy content (64) alone. The engine's 1698 kWh tracks the cert's own
lodged RHI water heating (1909) far better than Elmhurst's reduced-data 1168,
and its £113 is closer to the cert's lodged £147 than Elmhurst's £78 — so the
engine is more faithful to the cert here, not buggy. Comment accuracy only.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DX8oAGsGkBHq3U4dsxYRzz
The dwelling has no hot-water cylinder (has_hot_water_cylinder false), so the
2675-vs-1839 kWh DHW-energy gap is not a cylinder storage loss as previously
stated — it is an un-root-caused SAP §4.3 community-DHW demand/distribution-loss
difference. No behaviour change; comment accuracy only.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DX8oAGsGkBHq3U4dsxYRzz
Full-SAP SAP-Schema-18.0.0 cert 9556-3047-3304-5355-1200 (Dovestone Gardens
Apt 4): communal heat-pump DHW + direct-electric room heaters. Was 67 (D) with
DHW mis-billed as direct-electric immersion; now 80 with DHW on the community
heat-pump fuel. Accredited Elmhurst worksheet 74 (evidence saved); the +6 is the
documented full-SAP measured-U/MVHR vs RdSAP age-band residual.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DX8oAGsGkBHq3U4dsxYRzz
Green on arrival — rename() records rejections rather than raising.
Pinned after verifying it bites: letting the rejection escape fails the
child sub_task, which turns the whole Task red.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Properties are planned and renamed one at a time, so only those with work
get a row and a run killed by the timeout still leaves every property it
reached both renamed and recorded.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The walk now yields a PropertyPlan per address-list row, and rename()
applies one. run() composes the two, so single-pass callers are unchanged.
Knowing a property's size before the first rename is what makes "when is
this property finished?" answerable.
Tests that reached through the old private _process_folder are rewritten
against the public run(), so they survive this kind of change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The local invoke now sends an SQS-shaped payload — a direct invoke is the
one shape that cannot reproduce the dry_run bug class. Test modules route
protected-method calls through typed helpers so both pass pyright strict.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Green on arrival: both fall out of the summary the run now returns and of
the guarded site lookup. Pinned after verifying they bite — an unguarded
DomnaSites[name] completes an "ECO" run that renames files under a
different site's alias.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Green on arrival: @task_handler records inputs and reads source_id from
the body key matching the source's value. Pinned because it is the only
place the new sharepoint_site member is proven against a real Postgres
enum rather than a Python one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Green on arrival: reading the request from the SQS record body is what
taking the task lane gave us. Verified the guard bites by restoring the
raw-event read, under which the "dry" run renames a live file.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Passed on arrival: threading the summary back up through the traversal
already carried subfolder results. Pinned so the merge cannot regress.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Six properties failed task 4d006fab (portfolio 854 / scenario 1334) with
DegeneratePredictionError. None has a lodged EPC, so each was predicted from a
neighbouring-cert cohort — and every cohort cert lodges its single building part
as `identifier: "complete"`. from_api_string accepted only "Main Dwelling" /
"Main building", so "complete" fell to OTHER, _has_main_part returned False, and
prediction refused to proceed.
"complete" is what SAP 9.91/9.92-era software writes for an unextended dwelling:
semantically identical to "Main Dwelling". Casing varies within a single postcode
(cert 8824-7422-1180-6934-1902 lodges "complete", 199 Highfield Road "Complete"),
so matching is now case-folded.
The API's `identifier` is free text with no schema enum — api.yml declares the
whole cert body `additionalProperties: true` and documents no field of it, and
`identifier` is absent from the 17 vocabularies at /api/codes. So the recognised
set can only grow by discovery, and each miss has cost an incident ("Main
building" → task a40e71c4; "complete" → task 4d006fab). Hence three changes, not
one:
- "complete" → MAIN, case-insensitively.
- Bare "Extension" → EXTENSION_1. The regex required a digit, so 668 corpus
occurrences silently dropped a real extension from the structure. This reverses
a prior deliberate pin whose stated rationale (RdSAP10 §1.2's 4-extension cap)
does not apply — there is no out-of-range number in a bare "Extension".
- _anchor_lone_building_part: a cert lodging exactly ONE part describes the whole
dwelling whatever it is called, so an unclassified lone part becomes MAIN.
Multi-part certs are untouched — there the identifier carries real information
and guessing would invent structure the cert does not state.
Also removes a false claim from DegeneratePredictionError's message: it asserted
the template was "lodged with a null part identifier", which was hardcoded, never
checked against a real cert, and wrong. It misdirected this investigation.
Verified against the live gov API: all 13 cohort certs across BL1 8EB, BL3 6XJ
and BL4 0RA now resolve a MAIN part. The full modelling e2e was NOT run (no local
AWS creds for the geospatial lookup) — this clears the blocking error, it does not
prove the predictions are good.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The gov-EPC API lodges a system-built wall as `wall_construction == 8`
per the authoritative GOV.UK RdSAP `WallConstructionCode` enumeration
(communitiesuk/epb-data-warehouse `.../SAP-Domains.xsd`: 8 = "system
built" in every schema version 17.0-21.0.1; park home is 10). The solid-
wall Recommendation Generator instead read code 8 as the Elmhurst
park-home label and suppressed the whole cohort's EWI/IWI Recommendation
(a 2026 DB sweep finds ~5.1k certs at code 8, ~2.1k uninsulated main
walls). Rewrite the mislabelled park-home test to pin the correct
behaviour (code 8 → EWI+IWI), and add a genuine park-home test on the
gov code 10 so the fix doesn't start over-offering on real park homes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DX8oAGsGkBHq3U4dsxYRzz
A Plan became default only when its Scenario was flagged default or the
property had no Plan yet. Scenarios are created is_default=False, so in
practice only the first-ever Plan was ever promoted — re-modelling a
property appended a new Plan but left the default frozen on the first run,
so a re-model never surfaced on the FE (property 792168 kept its 2026-07-22
scenario-1328 default through three same-scenario re-runs).
Also promote when the re-run models the SAME scenario that currently owns
the property's default, so a re-run refreshes the shown Plan in place (the
save demotes the stale one). A re-run of a different scenario still does not
hijack the default. New PlanRepository.default_plan_scenario_by_property
surfaces each property's default scenario for the decision.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
refetch_epc=True fetched a fresh cert but the recency tie-break (strict >)
dropped it whenever the stored row shared the cert's inspection date — so a
row written before a schema field/table existed (e.g. room-in-roof geometry)
could never be healed by re-fetching. The dwelling was modelled on stale
fabric indefinitely.
Reconcile instead of just comparing dates: on an equal inspection date,
refresh a public gov cert (uploaded_file_id NULL) when the fetched cert
differs from the stored one, so a plain re-fetch re-ingests it. One of our
own surveys (PasHub / ECMK / Elmhurst — uploaded_file_id set) is preserved on
the tie, keeping the survey-wins rule (ADR-0001 / #1589); an unchanged public
cert is still not re-persisted.
`_reconcile_lodged` layers the content refresh over `_newer_lodged` (recency
untouched). New `EpcRepository.survey_lodged_uprns` surfaces the row-level
uploaded_file_id provenance the hydrated EpcPropertyData drops. Together with
the property_id-anchored persist, a refetch_epc=True re-run now heals both a
missing epc_property row and a stale one.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The lodged EPC read keys on UPRN (ADR-0001 recency tie-break), but writes
and the front end both key on property_id. When a dwelling is onboarded
under two portfolios, the shared UPRN surfaces the sibling's lodged row,
so `lodged_epc_is_new` stays False and the modelling run skips persisting
a copy for this property — leaving it with no epc_property row the FE can
read. The EPC is modelled but invisible (portfolio 850 / property 792235,
uprn 100021979925, also under property 753896 in portfolio 830).
Drive the lodged persist off "does THIS property own a lodged row?"
(property_id-scoped) rather than "is the cert new?" (UPRN-scoped): add
`EpcRepository.property_ids_with_lodged_epc` and persist a per-property
copy whenever the cert is new OR the property has no lodged row of its
own. Idempotent (save_batch is replace-by-property_id), and the recency
skip still holds for a property that already owns its row.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
