diff --git a/CONTEXT.md b/CONTEXT.md index 047087e97..3a7489b29 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -121,6 +121,18 @@ _Avoid_: energy assessment, site survey, field survey, Domna survey, Hestia surv Property data supplied by a landlord that may correct or supplement the public EPC for a single Property; triggers Rebaselining when applied; not applicable when Site Notes are present. _Avoid_: patches (deprecated), corrections, manual EPC, edits +**Stock Condition Data**: +Landlord-supplied per-**component** lifecycle records — one row per element (e.g. a dwelling's **roof covering**) carrying its material/type, fitted-or-renewed date, quantity, and a **Remaining Life**. It answers *"when is this component due for renewal?"*, not *"what is the dwelling's SAP picture?"* — so unlike **Site Notes** and **Landlord Overrides** it is **not read by the modelling engine** and never enters `EpcPropertyData`, Rebaselining, or a Recommendation (ADR-0064). Sourced from a landlord's stock-condition survey (asset-management data); ingested by this repo purely as a **producer** and persisted keyed by UPRN as `PropertyConditionSurvey → Element → AspectCondition`. The consumer is the Next.js/Drizzle front-end, which reads the tables directly (the DB schema is the contract — the `element_type` / `aspect_type` PG enums are an FE-owned surface, cf. [[main-heating-system-pgenum-is-fe-owned]]) to display component data against a Property and to raise the **Roof Replacement Flag**. **Remaining Life** derives from the component's **Decent Homes Standard** expected lifetime (`fitted date + standard lifespan − report date`), so it is trusted as given, not recomputed here. +_Avoid_: condition survey (ambiguous with **Site Notes**), asset data, patches; treating it as a **Landlord Override** or fabric signal (the engine never reads it — this iteration) + +**Component Renewal Year**: +The absolute calendar year a **Stock Condition Data** component is due for replacement — stored as `renewal_year` on the `AspectCondition`. Computed at load as **`export_year + Remaining Life`** (the export/report date the landlord's file was generated, not `now()`), so it is stable across re-runs — an absolute year does not drift, whereas a stored "years remaining" would. The fitted/renewed date is retained separately as `install_date`. +_Avoid_: remaining life (that is the input; the stored value is an absolute year), replacement date, end-of-life date + +**Roof Replacement Flag**: +A **front-end-owned** presentation signal raised on a **Solar PV Recommendation** when a Property's roof-covering **Component Renewal Year** falls within (or near) the measure horizon — warning a human that installing solar onto a soon-to-be-replaced roof is unwise. It is **not** computed in this repo and does **not** change what the engine recommends (the optimiser's solar Option is unchanged): the Next.js/Drizzle app joins **Stock Condition Data** to the recommendation by UPRN and applies the "due soon" threshold itself (ADR-0064). A future engine upgrade that *acts* on this (bundling a re-roof, deferring solar) is explicitly out of scope for this iteration. +_Avoid_: roof warning (unspecific), re-roof recommendation (there is no engine measure yet), solar suppression + **Landlord-Description Classification**: Resolving a **Landlord Description** (unbounded free-text a landlord supplies for one component — "CWI" / "Cav filled" / "cavity insulated" all name one thing) onto a **Recognised Internal Description** via an LLM classifier, persisted in the `landlord_*_overrides` table (`source=classifier`) as a reviewed cache. Four vocabularies are kept **distinct** and must not be conflated: a **Landlord Description** (unbounded input); a **Recognised Internal Description** (the closed target taxonomy — e.g. a `MainHeatingSystemType` archetype — each binding to a Simulation Overlay); a **Lodged Description** (the gov-EPC `main_heating[].description` rendering, e.g. "Room heaters, electric" — only an example of which system *types* occur, never a map key); and the **SAP main heating code** (Table 4a/4b, what the calculator consumes). The classifier maps Landlord → Recognised Internal → SAP code. When it cannot confidently place the text it emits **`None`** (no overlay → the lodged EPC stands, surfaced to the user as "no suitable match"), **never the nearest wrong archetype** — the target taxonomy must be complete enough that a real system always has a correct home, so the classifier never overflows into a garbage-drawer archetype (ADR-0041). diff --git a/docs/adr/0064-stock-condition-data-is-engine-isolated-and-fe-consumed.md b/docs/adr/0064-stock-condition-data-is-engine-isolated-and-fe-consumed.md new file mode 100644 index 000000000..43fdaf102 --- /dev/null +++ b/docs/adr/0064-stock-condition-data-is-engine-isolated-and-fe-consumed.md @@ -0,0 +1,108 @@ +--- +status: accepted +--- + +# Stock condition data is a producer-only, engine-isolated dataset; the roof-replacement warning on solar is a front-end overlay + +We are ingesting a landlord's (Calico) **stock-condition survey** — for now, one +component: **roof covering**, ~5,346 rows of `Asset Reference, Type (material), +Fitted/Renewed Date, Quantity, Remaining Life`. The temptation is to treat any +landlord-supplied component fact like a **Landlord Override** and fold it into the +`EpcPropertyData` picture. That would be wrong here, and the whole design turns on +saying why. + +**Stock Condition Data answers a different question from the modelling picture.** +A **Landlord Override** ("this wall is solid brick") and **Site Notes** (a full +Domna survey) both correct *what the dwelling is* so the SAP score is right — the +engine reads them. Stock Condition Data answers *when a component is due for +renewal*: the roof-covering material plus a **Remaining Life** derived from the +component's **Decent Homes Standard** expected lifetime. It is asset-management +data, not a fabric signal. Conflating the two would drag component-lifecycle rows +into `EpcPropertyData`, Rebaselining, and the Recommendation model for no scoring +benefit and real coupling cost. + +**The consumer is the product, not the engine.** The value is: (1) show component +data against a Property in the front-end, and (2) warn a human, on a **Solar PV +Recommendation**, when the roof is near end-of-life — because installing solar +onto a roof due for replacement is a poor sequence. That warning is a *human +information* signal, not an optimiser input: the engine still recommends solar +unchanged; a badge says "mind the roof". The cross-reference (roof renewal year ↔ +solar recommendation) and the "due soon" threshold live in the **Next.js/Drizzle +front-end**, which reads the condition tables directly. This repo is purely a +**producer** into a schema the front-end consumes — so the DB schema (the +`property_condition_survey` / `element` / `aspect_condition` tables and their +`element_type` / `aspect_type` PG enums) is an **FE-owned contract**, the same +pattern as `main_heating_system`. + +Decided in a grill-with-docs session with Khalim, 2026-07-13. + +## Decision + +**Stock Condition Data is ingested producer-only and is engine-isolated. The +modelling engine never reads it (this iteration); the roof-replacement warning on +solar is a front-end presentation overlay, not an engine output.** + +- **Engine isolation.** Condition rows are written only to + `PropertyConditionSurvey → Element → AspectCondition`, keyed by UPRN. They never + enter `EpcPropertyData`, Rebaselining, Bill Derivation, or a Recommendation. The + optimiser's Solar PV Option is unchanged whether or not the roof is near + end-of-life. A future engine upgrade that *acts* on condition data (bundling a + re-roof, deferring solar) is a separate, later decision — explicitly out of + scope now. + +- **The flag lives above modelling.** The **Roof Replacement Flag** is computed by + the Next.js/Drizzle app, which joins condition data to the solar recommendation + by UPRN and owns the "due soon" threshold. This repo neither computes the flag + nor exposes a read API for it — it only writes the tables the FE reads. The DB + schema is therefore a shared FE contract; schema changes need a companion + Drizzle view. + +- **UPRN resolution from the property table, not a CSV.** Calico's `Asset + Reference` maps to `property.landlord_property_id` scoped to `portfolio_id = + 824`, which yields the `uprn`. The lookup is a Postgres-backed `UprnLookup` + reading the property table — fulfilling the existing "replace CSV with postgres" + TODO — not an exported file. A key-alignment test (Asset Reference == stored + `landlord_property_id`) is pinned during TDD implementation. + +- **Drop-and-report reconciliation; nothing vanishes silently.** Rows that do not + resolve to a portfolio-824 UPRN are dropped and counted; rows whose resolved + `property.uprn` is null are dropped **and reported** (none expected — we want to + see any that are); the ~1,239 blank-`Type` placeholder rows (Quantity 0, + sentinel date, nonsense Remaining Life) are dropped on parse. Every load emits a + reconciliation report: `loaded / unmatched-to-portfolio / null-uprn / + blank-placeholder`. + +- **Renewal year is anchored to the export date, not `now()`.** `Remaining Life` + is years-from-report; the stored **Component Renewal Year** = `export_year + + Remaining Life` (export date passed explicitly as a load parameter — Calico's + file is dated ≈ 2026-06-02), so the absolute renewal year is reproducible across + re-runs. The fitted/renewed date is retained as `install_date`. This replaces + the module's `survey_year = datetime.now().year` TODO for this loader. + +- **Delivery: one-off runner load, from S3.** Calico is a bulk one-off import, run + through the local/orchestration runner (not the SQS-Lambda path Peabody and LBWF + use). The raw `.xlsx` is stored in S3 under a documented Calico key and kept out + of git (it carries real addresses). + +## Consequences + +- The condition-ingestion module moves out of legacy `backend/condition/` into the + DDD layout (domain / repositories / infrastructure / applications / + orchestration) as a full lift-and-shift — shared core, the Peabody and LBWF + adapters, and the delivery path — with Calico added as a new adapter alongside + them. Peabody/LBWF keep their Lambda after the move; the relocation is + behaviour-preserving for them. +- A new `Calico` `ConditionFileType`, parser + row DTO, roof-covering element map + (`"Roof Covering"` → roof `ElementType`, `MATERIAL` aspect = `Type`), and mapper + are added and registered in the factory. Roof covering is already a modelled + component type in `ElementType`, so no enum change is needed for this load. +- CONTEXT.md gains **Stock Condition Data**, **Component Renewal Year**, and + **Roof Replacement Flag**, kept distinct from **Site Notes** and **Landlord + Overrides** so the ubiquitous language records that this source is *not* read by + the engine. +- Because the FE reads these tables through Drizzle, any later schema change here + (a new component, a new aspect) is a cross-repo contract change requiring a + companion Drizzle migration — the `main_heating_system` FE-ownership lesson + applies. +- If the "mind the roof" signal later needs to *drive* modelling, that is a new + ADR that deliberately re-opens engine isolation — not an incremental tweak.