From 6871aff62ad633c57bec0c0d6e9034d8bf2c5bda Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Fri, 10 Jul 2026 23:21:21 +0000 Subject: [PATCH 01/11] =?UTF-8?q?docs(portfolio):=20ADR-0012=20=E2=80=94?= =?UTF-8?q?=20list=20resolves=20descriptors=20via=20override=20precedence?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Records the decision to make the portfolio property-list override-aware for Property Type / Built Form / Construction Age / Main Fuel, extending ADR-0008's override → EPC-derived → Unknown precedence to a second surface via shared epcSources fragments. Adds a CONTEXT.md Flagged-ambiguities note. Co-Authored-By: Claude Opus 4.8 (1M context) --- CONTEXT.md | 1 + ...ves-descriptors-via-override-precedence.md | 89 +++++++++++++++++++ 2 files changed, 90 insertions(+) create mode 100644 docs/adr/0012-portfolio-list-resolves-descriptors-via-override-precedence.md diff --git a/CONTEXT.md b/CONTEXT.md index de164f7b..da92a07b 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -192,6 +192,7 @@ _Avoid_: estimation notification, banner (those are component names) ## Flagged ambiguities +- The portfolio **Property table**'s **Property Type**, **Built Form**, **Construction Age** and **Main Fuel** columns and filters show the **landlord-override-resolved** value — the same precedence the modelling **Run filter** applies (override fact → EPC-derived → **Unknown**) — not the raw `epc_property` value or the legacy property-row column. See [ADR-0012](./docs/adr/0012-portfolio-list-resolves-descriptors-via-override-precedence.md). Two label subtleties follow from this: **Construction Age** is an RdSAP band (e.g. "1900–1929", "before 1900"), never a single year (it was formerly the "Year Built" column); and **Main Fuel** is override-only for new-approach properties, so it reads **Unknown** unless a landlord override supplies it. - The UI surfaces labelled **"Current EPC"** and **"Current Efficiency State"** (property table, building-passport card) show **Effective performance**, not Lodged — despite the glossary advising against "current performance" for Effective. The label is a product choice ("current" = the property's true present-day modelled state); the underlying datum is still **Effective performance**. The **"Lodged EPC"** column/badge is the only surface showing **Lodged performance**, and only when a real certificate exists (`source = lodged`). - "Upload" is used in the codebase to mean both the file-on-S3 and the BulkUpload row. We standardise on **BulkUpload** for the row; the file is just "the source file." - "Onboarding" appears in some route paths (`bulk_onboarding_inputs/...`) but isn't part of this glossary — we use **BulkUpload** end-to-end. diff --git a/docs/adr/0012-portfolio-list-resolves-descriptors-via-override-precedence.md b/docs/adr/0012-portfolio-list-resolves-descriptors-via-override-precedence.md new file mode 100644 index 00000000..08f4c461 --- /dev/null +++ b/docs/adr/0012-portfolio-list-resolves-descriptors-via-override-precedence.md @@ -0,0 +1,89 @@ +# 12. The portfolio property-list resolves descriptors through the landlord-override precedence + +Date: 2026-07-10 + +## Status + +Accepted + +## Context + +The portfolio property-list's **Property Type**, **Built Form**, **Year Built** +and **Main Fuel** columns and filters read raw `property` row columns (and, for +main fuel, the legacy `property_details_epc` table). Those columns are NULL for +**new-approach** properties (`property.updated_at >= 2026-06-01`, the EPC-graph +cutoff — see `epcSources.ts`), so for any new portfolio the columns render blank +and the filters match nothing. The already-migrated columns (current/lodged EPC, +floor area, CO₂, provenance, expiry) route through `epcSources` fragments and +work; these four were simply never migrated. + +Separately, [ADR-0008](./0008-modelling-runs-filters-to-distributor.md) +established the resolution precedence for the modelling **Run filter**: +**landlord-override fact → EPC-derived value (new-approach graph, lodged over +predicted, RdSAP codes mapped; legacy property-row fallback) → Unknown**. It is +implemented in-app inline in `previewModellingRun` and mirrored by the +distributor; divergence between the two is defined as a bug. + +## Decision + +The portfolio list resolves Property Type, Built Form, Construction Age and Main +Fuel through the **same override precedence as the Run filter**, so the browse +surface agrees with what a modelling run would actually select. Concretely: + +- **Shared fragments, one home for the rule.** `epcSources.ts` gains + `overrideJoins` (LEFT JOINs of `property_overrides` aliased per component at + `building_part = 0`) and `resolvedPropertyTypeSql` / `resolvedBuiltFormSql` / + `resolvedConstructionBandSql` / `resolvedMainFuelSql`. Both the list query and + `previewModellingRun` consume them; the inline COALESCE in + `previewModellingRun` is refactored onto the shared fragments. +- **Never-null.** Each fragment is `COALESCE(override, epcSql, 'Unknown')`. Every + property resolves to a concrete, filterable string; the **Unknown** filter chip + matches "no resolvable value at all" — character-for-character the semantics + CONTEXT.md already gives the Run filter's Unknown. The list columns render an + explicit **Unknown** pill instead of a blank `—`. +- **Construction Age is band-native, not a year.** Override and EPC both store + the RdSAP band **letter** (`A..M`, `Unknown`); the field resolves in letter + space and displays the band **label** ("before 1900", "1900–1929", … "2023 + onwards"). A numeric year cannot represent an `Unknown` override and misreads a + band as a precise year. The column is **renamed "Year Built" → "Construction + Age"** and its filter options become the bands, not years; legacy raw years are + bucketed into bands. +- **Main Fuel is override-only for new-approach.** The EPC graph exposes no clean + main-fuel string (a documented `epcSources` gap → NULL), so for new-approach + properties the override is the *only* source; unoverridden properties read + **Unknown**. The override vocabulary (`"mains gas"`, `"LPG (bulk)"`, …) is + folded into the existing collapsed filter labels (Mains Gas, LPG, …). +- **`tenure` is removed from the list**, not fixed. It has no override component, + and its `epc_property.tenure` values don't cleanly reconcile with the filter's + `dbValues`. It is dropped from the list column and filter only; + `resolvePropertyMeta`/`resolvePropertyDescriptors` still resolve it for the + per-property building passport, and `property.tenure` is retained. + +## Alternatives considered + +- **EPC-derived only, no overrides.** Smaller, and consistent with how the other + already-migrated columns behave. Rejected: the list would then disagree with + the modelling Run filter — e.g. show "Flat" for a property a landlord override + makes (and modelling treats as) "Maisonette" — reintroducing exactly the + browse-vs-model mismatch this work set out to remove. +- **Construction Age as a numeric year.** Familiar and a smaller diff. Rejected: + cannot represent an `Unknown` override (collapses to `—`, breaking the + never-null contract) and misrepresents a band as a point year. +- **Keep the precedence rule inline in `previewModellingRun` and duplicate it in + the list.** Avoids touching a working feature. Rejected: it re-creates the + two-homes divergence ADR-0008 explicitly warns against — now within the app. + +## Consequences + +- The count query (`getPropertiesCount`) must join `property_overrides` when any + of these fields is filtered — `EPC_JOIN_FILTER_FIELDS` (or a sibling set) is + extended. Joins are perf-sensitive (the count path already brushes Vercel's + 15s limit), so the joins stay conditional on an active filter. +- Every new-approach property without a main-fuel override shows **Unknown** — a + lot of Unknown pills, but an honest signal of the EPC gap rather than a blank + implying "not applicable". +- Filter option lists grow: Property Type gains **Park home** + **Unknown**; + Built Form gains **Unknown** (keeping **Not Recorded** for the explicit + override); Main Fuel folds the override vocab into existing labels. +- `previewModellingRun` and the portfolio list are now provably in sync — the + shared fragment is the single reference implementation ADR-0008 asked for. From 3bf816130ca240c6fcbc67de61e1a2f89a65732f Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Fri, 10 Jul 2026 23:21:22 +0000 Subject: [PATCH 02/11] feat(portfolio): resolve Property Type column + filter via override precedence MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Slice 1 of the portfolio-list descriptor fix. Property Type was read from the raw property row (p.property_type), which is NULL for new-approach properties (updated_at >= 2026-06-01), so the column and filter broke for new portfolios. - New pure resolver resolvePropertyType (descriptorResolution.ts, mirrors the provenance.ts split) owning override → EPC-derived (lodged over predicted, RdSAP codes mapped, legacy fallback) → 'Unknown', with 9 unit tests. - SQL twin resolvedPropertyTypeSql + propertyTypeOverrideJoin in epcSources.ts. - Wired into getProperties SELECT, the filter colMap, and the count query's conditional join set (propertyType added to EPC_JOIN_FILTER_FIELDS). - PROPERTY_TYPE_OPTIONS gains Park home + Unknown. Co-Authored-By: Claude Opus 4.8 (1M context) --- src/app/portfolio/[slug]/utils.ts | 16 ++- src/app/utils/propertyFilters.ts | 6 + src/lib/services/descriptorResolution.test.ts | 132 ++++++++++++++++++ src/lib/services/descriptorResolution.ts | 49 +++++++ src/lib/services/epcSources.ts | 25 ++++ 5 files changed, 225 insertions(+), 3 deletions(-) create mode 100644 src/lib/services/descriptorResolution.test.ts create mode 100644 src/lib/services/descriptorResolution.ts diff --git a/src/app/portfolio/[slug]/utils.ts b/src/app/portfolio/[slug]/utils.ts index 0eb1b88e..a3228f17 100644 --- a/src/app/portfolio/[slug]/utils.ts +++ b/src/app/portfolio/[slug]/utils.ts @@ -31,6 +31,8 @@ import { lodgedEpcBandSql, lodgedSapSql, provenanceSignalSql, + propertyTypeOverrideJoin, + resolvedPropertyTypeSql, } from "@/lib/services/epcSources"; import { FilterGroups, @@ -595,7 +597,8 @@ function buildConditionSql(filter: PropertyFilter): ReturnType | nul const options = ENUM_FIELD_DB_OPTIONS[filter.field]; const colMap: Record> = { - propertyType: sql`p.property_type`, + // Override-resolved to match the column + the Run filter (ADR-0012). + propertyType: resolvedPropertyTypeSql, builtForm: sql`p.built_form`, tenure: sql`p.tenure`, yearBuilt: sql`p.year_built`, @@ -695,6 +698,8 @@ const EPC_JOIN_FILTER_FIELDS = new Set([ "floorArea", "epcExpiryDate", "mainfuel", + // Override-resolved (ADR-0012): needs the EPC graph joins + the override join. + "propertyType", ]); const PLAN_JOIN_FILTER_FIELDS = new Set(["expectedEpc"]); @@ -720,7 +725,8 @@ export async function getPropertiesCount( const epcJoins = needsEpcJoins ? sql`LEFT JOIN property_details_epc epc ON epc.property_id = p.id - ${newApproachJoins}` + ${newApproachJoins} + ${propertyTypeOverrideJoin}` : sql``; const planJoin = needsPlanJoin ? sql`LEFT JOIN LATERAL ( @@ -789,7 +795,10 @@ export async function getProperties( COALESCE(rec.cost, 0) AS "totalRecommendationCost", p.landlord_property_id AS "landlordPropertyId", p.original_sap_points AS "originalSapPoints", - p.property_type AS "propertyType", + -- Property Type is resolved through the landlord-override precedence + -- (override → EPC-derived → 'Unknown'), matching the modelling Run filter. + -- See resolvePropertyType / ADR-0012. + ${resolvedPropertyTypeSql} AS "propertyType", p.built_form AS "builtForm", p.tenure AS tenure, p.year_built AS "yearBuilt", @@ -833,6 +842,7 @@ export async function getProperties( LEFT JOIN property_details_epc epc ON epc.property_id = p.id ${newApproachJoins} + ${propertyTypeOverrideJoin} WHERE p.portfolio_id = ${portfolioId} -- Unmatched (no-UPRN) properties live in the "Needs attention" tab until -- resolved, so they're excluded from the main table. diff --git a/src/app/utils/propertyFilters.ts b/src/app/utils/propertyFilters.ts index 1f85675e..f62181e5 100644 --- a/src/app/utils/propertyFilters.ts +++ b/src/app/utils/propertyFilters.ts @@ -64,11 +64,17 @@ export interface EnumOption { dbValues: string[]; } +// dbValues are the label vocabulary emitted by resolvedPropertyTypeSql (override +// snapshot / EPC-derived label). "Park home" is reachable from an RdSAP code (4) +// or an override; "Unknown" is the never-null terminal + an explicit override — +// it selects properties with no resolvable value at all (ADR-0012). export const PROPERTY_TYPE_OPTIONS: EnumOption[] = [ { label: "House", dbValues: ["House"] }, { label: "Flat", dbValues: ["Flat"] }, { label: "Bungalow", dbValues: ["Bungalow"] }, { label: "Maisonette", dbValues: ["Maisonette"] }, + { label: "Park home", dbValues: ["Park home"] }, + { label: "Unknown", dbValues: ["Unknown"] }, ]; export const BUILT_FORM_OPTIONS: EnumOption[] = [ diff --git a/src/lib/services/descriptorResolution.test.ts b/src/lib/services/descriptorResolution.test.ts new file mode 100644 index 00000000..69c68a51 --- /dev/null +++ b/src/lib/services/descriptorResolution.test.ts @@ -0,0 +1,132 @@ +import { describe, expect, it } from "vitest"; +import { resolvePropertyType } from "./descriptorResolution"; + +describe("resolvePropertyType", () => { + it("prefers a landlord override over the EPC-derived value", () => { + expect( + resolvePropertyType({ + override: "Maisonette", + isNewApproach: true, + lodgedPropertyType: "2", // EPC code for Flat + lodgedDwellingType: null, + predictedPropertyType: null, + predictedDwellingType: null, + legacyPropertyType: null, + }), + ).toBe("Maisonette"); + }); + + it("maps an RdSAP property-type code to its label when there is no override", () => { + expect( + resolvePropertyType({ + override: null, + isNewApproach: true, + lodgedPropertyType: "0", // RdSAP code for House + lodgedDwellingType: null, + predictedPropertyType: null, + predictedDwellingType: null, + legacyPropertyType: null, + }), + ).toBe("House"); + }); + + it("falls back to the predicted EPC when there is no lodged value", () => { + expect( + resolvePropertyType({ + override: null, + isNewApproach: true, + lodgedPropertyType: null, + lodgedDwellingType: null, + predictedPropertyType: "2", // RdSAP code for Flat + predictedDwellingType: null, + legacyPropertyType: null, + }), + ).toBe("Flat"); + }); + + it("prefers the lodged EPC over the predicted EPC when both exist", () => { + expect( + resolvePropertyType({ + override: null, + isNewApproach: true, + lodgedPropertyType: "0", // House + lodgedDwellingType: null, + predictedPropertyType: "2", // Flat + predictedDwellingType: null, + legacyPropertyType: null, + }), + ).toBe("House"); + }); + + it("falls back to dwelling_type when the lodged property_type is absent", () => { + expect( + resolvePropertyType({ + override: null, + isNewApproach: true, + lodgedPropertyType: null, + lodgedDwellingType: "Flat", + predictedPropertyType: null, + predictedDwellingType: null, + legacyPropertyType: null, + }), + ).toBe("Flat"); + }); + + it("reads the legacy property-row value for a legacy property", () => { + expect( + resolvePropertyType({ + override: null, + isNewApproach: false, + lodgedPropertyType: "0", // must be ignored for legacy + lodgedDwellingType: null, + predictedPropertyType: null, + predictedDwellingType: null, + legacyPropertyType: "Bungalow", + }), + ).toBe("Bungalow"); + }); + + it("never reads the EPC graph nor the legacy row across the cutoff (new-approach ignores the legacy row)", () => { + expect( + resolvePropertyType({ + override: null, + isNewApproach: true, + lodgedPropertyType: null, + lodgedDwellingType: null, + predictedPropertyType: null, + predictedDwellingType: null, + legacyPropertyType: "Bungalow", // legacy row must NOT surface for a new-approach property + }), + ).toBe("Unknown"); + }); + + // Spec locks for decisions taken in ADR-0012 — an override always wins (even an + // explicit "Unknown"), and an unrecognised RdSAP code is passed through as-is. + it("lets an explicit 'Unknown' override win over a known EPC value", () => { + expect( + resolvePropertyType({ + override: "Unknown", + isNewApproach: true, + lodgedPropertyType: "0", // EPC knows "House" — the override still wins + lodgedDwellingType: null, + predictedPropertyType: null, + predictedDwellingType: null, + legacyPropertyType: null, + }), + ).toBe("Unknown"); + }); + + it("passes an unrecognised RdSAP code through unchanged", () => { + expect( + resolvePropertyType({ + override: null, + isNewApproach: true, + lodgedPropertyType: "9", // not in PROPERTY_TYPE_LABELS + lodgedDwellingType: null, + predictedPropertyType: null, + predictedDwellingType: null, + legacyPropertyType: null, + }), + ).toBe("9"); + }); +}); diff --git a/src/lib/services/descriptorResolution.ts b/src/lib/services/descriptorResolution.ts new file mode 100644 index 00000000..18dc015b --- /dev/null +++ b/src/lib/services/descriptorResolution.ts @@ -0,0 +1,49 @@ +/** + * Pure precedence resolvers for the portfolio-list property descriptors. + * + * Mirrors the provenance.ts split: the DB query gathers the raw inputs (landlord + * override snapshot, lodged/predicted EPC values, legacy property-row value) and + * these functions own the precedence + RdSAP code→label mapping. The SQL twins + * in epcSources.ts (resolvedPropertyTypeSql, …) mirror this logic — keep in sync. + * + * Precedence (ADR-0008, extended to the list by ADR-0012): + * landlord override → EPC-derived (lodged over predicted, codes mapped; + * legacy property-row fallback) → 'Unknown' (never null) + */ +import { PROPERTY_TYPE_LABELS } from "./epcSources"; + +export interface PropertyTypeInputs { + /** property_overrides.override_value for component 'property_type', building_part 0. */ + override: string | null; + /** Whether the property reads the new EPC graph (updated_at >= cutoff). */ + isNewApproach: boolean; + /** epc_property(source='lodged').property_type — RdSAP code or text. */ + lodgedPropertyType: string | null; + /** epc_property(source='lodged').dwelling_type — fallback when property_type is absent. */ + lodgedDwellingType: string | null; + predictedPropertyType: string | null; + predictedDwellingType: string | null; + /** property.property_type — the legacy row value. */ + legacyPropertyType: string | null; +} + +export function resolvePropertyType(i: PropertyTypeInputs): string { + if (i.override) return i.override; + const derived = i.isNewApproach + ? epcPropertyType(i.lodgedPropertyType, i.lodgedDwellingType) ?? + epcPropertyType(i.predictedPropertyType, i.predictedDwellingType) + : i.legacyPropertyType; + return derived ?? "Unknown"; +} + +/** + * One EPC source's property-type label: the RdSAP code mapped to a label (text + * passes through), falling back to dwelling_type. Mirrors epcPropertyTypeSql. + */ +function epcPropertyType( + propertyType: string | null, + dwellingType: string | null, +): string | null { + if (propertyType != null) return PROPERTY_TYPE_LABELS[propertyType] ?? propertyType; + return dwellingType; +} diff --git a/src/lib/services/epcSources.ts b/src/lib/services/epcSources.ts index 3a75bfd9..3392d667 100644 --- a/src/lib/services/epcSources.ts +++ b/src/lib/services/epcSources.ts @@ -283,6 +283,31 @@ export const builtFormTypeSql = sql`CASE ELSE p.built_form END`; +// ───────────────────────────────────────────────────────────────────────────── +// Landlord-override-resolved descriptors (ADR-0012). Each fragment wraps the +// EPC-derived expression above with the property_overrides snapshot and a +// never-null 'Unknown' terminal — the same precedence previewModellingRun +// applies (override → EPC-derived → Unknown). The TS twins in +// descriptorResolution.ts (resolvePropertyType, …) own and unit-test this +// precedence; keep the two in sync. +// +// A resolved fragment references an override alias that its join provides — add +// the matching *OverrideJoin to any query that selects/filters on the fragment. +// Overrides are whole-dwelling for these components, so the join pins +// building_part = 0 (the main building) and never multiplies rows. +// ───────────────────────────────────────────────────────────────────────────── + +/** LEFT JOIN exposing the property_type override snapshot (alias `pot`). */ +export const propertyTypeOverrideJoin = sql` + LEFT JOIN property_overrides pot + ON pot.property_id = p.id + AND pot.override_component = 'property_type' + AND pot.building_part = 0 +`; + +/** property_type: override → EPC-derived → 'Unknown'. Mirrors resolvePropertyType. */ +export const resolvedPropertyTypeSql = sql`COALESCE(pot.override_value, ${propertyTypeSql}, 'Unknown')`; + /** * The main building part's construction age band for one epc_property alias, as * a correlated scalar subquery — properties can also have extension parts, so a From e2ab52e4f91086210c952894684b845711073ecb Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Fri, 10 Jul 2026 23:26:07 +0000 Subject: [PATCH 03/11] feat(portfolio): resolve Built Form column + filter via override precedence Slice 2. Same shape as slice 1: resolveBuiltForm + builtFormOverrideJoin / resolvedBuiltFormSql, wired into getProperties SELECT, the filter colMap, and the count query's join set (builtForm added to EPC_JOIN_FILTER_FIELDS). BUILT_FORM_OPTIONS gains Unknown (Not Recorded now matches only an explicit override of that literal). Co-Authored-By: Claude Opus 4.8 (1M context) --- src/app/portfolio/[slug]/utils.ts | 11 +++- src/app/utils/propertyFilters.ts | 5 ++ src/lib/services/descriptorResolution.test.ts | 64 ++++++++++++++++++- src/lib/services/descriptorResolution.ts | 27 +++++++- src/lib/services/epcSources.ts | 11 ++++ 5 files changed, 113 insertions(+), 5 deletions(-) diff --git a/src/app/portfolio/[slug]/utils.ts b/src/app/portfolio/[slug]/utils.ts index a3228f17..e8d6a91f 100644 --- a/src/app/portfolio/[slug]/utils.ts +++ b/src/app/portfolio/[slug]/utils.ts @@ -33,6 +33,8 @@ import { provenanceSignalSql, propertyTypeOverrideJoin, resolvedPropertyTypeSql, + builtFormOverrideJoin, + resolvedBuiltFormSql, } from "@/lib/services/epcSources"; import { FilterGroups, @@ -599,7 +601,7 @@ function buildConditionSql(filter: PropertyFilter): ReturnType | nul const colMap: Record> = { // Override-resolved to match the column + the Run filter (ADR-0012). propertyType: resolvedPropertyTypeSql, - builtForm: sql`p.built_form`, + builtForm: resolvedBuiltFormSql, tenure: sql`p.tenure`, yearBuilt: sql`p.year_built`, provenance: provenanceSignalSql, @@ -700,6 +702,7 @@ const EPC_JOIN_FILTER_FIELDS = new Set([ "mainfuel", // Override-resolved (ADR-0012): needs the EPC graph joins + the override join. "propertyType", + "builtForm", ]); const PLAN_JOIN_FILTER_FIELDS = new Set(["expectedEpc"]); @@ -726,7 +729,8 @@ export async function getPropertiesCount( const epcJoins = needsEpcJoins ? sql`LEFT JOIN property_details_epc epc ON epc.property_id = p.id ${newApproachJoins} - ${propertyTypeOverrideJoin}` + ${propertyTypeOverrideJoin} + ${builtFormOverrideJoin}` : sql``; const planJoin = needsPlanJoin ? sql`LEFT JOIN LATERAL ( @@ -799,7 +803,7 @@ export async function getProperties( -- (override → EPC-derived → 'Unknown'), matching the modelling Run filter. -- See resolvePropertyType / ADR-0012. ${resolvedPropertyTypeSql} AS "propertyType", - p.built_form AS "builtForm", + ${resolvedBuiltFormSql} AS "builtForm", p.tenure AS tenure, p.year_built AS "yearBuilt", ${lodgementDateSql(sql`epc`)}::text AS "epcLodgementDate", @@ -843,6 +847,7 @@ export async function getProperties( ON epc.property_id = p.id ${newApproachJoins} ${propertyTypeOverrideJoin} + ${builtFormOverrideJoin} WHERE p.portfolio_id = ${portfolioId} -- Unmatched (no-UPRN) properties live in the "Needs attention" tab until -- resolved, so they're excluded from the main table. diff --git a/src/app/utils/propertyFilters.ts b/src/app/utils/propertyFilters.ts index f62181e5..88add78a 100644 --- a/src/app/utils/propertyFilters.ts +++ b/src/app/utils/propertyFilters.ts @@ -77,6 +77,10 @@ export const PROPERTY_TYPE_OPTIONS: EnumOption[] = [ { label: "Unknown", dbValues: ["Unknown"] }, ]; +// dbValues are the label vocabulary emitted by resolvedBuiltFormSql (override +// snapshot / EPC-derived label). "Not Recorded" matches only an explicit +// override of that literal; "Unknown" is the never-null terminal + an explicit +// override — it selects properties with no resolvable value at all (ADR-0012). export const BUILT_FORM_OPTIONS: EnumOption[] = [ { label: "Detached", dbValues: ["Detached"] }, { label: "Semi-Detached", dbValues: ["Semi-Detached"] }, @@ -85,6 +89,7 @@ export const BUILT_FORM_OPTIONS: EnumOption[] = [ { label: "Enclosed End-Terrace", dbValues: ["Enclosed End-Terrace"] }, { label: "Enclosed Mid-Terrace", dbValues: ["Enclosed Mid-Terrace"] }, { label: "Not Recorded", dbValues: ["Not Recorded"] }, + { label: "Unknown", dbValues: ["Unknown"] }, ]; export const TENURE_OPTIONS: EnumOption[] = [ diff --git a/src/lib/services/descriptorResolution.test.ts b/src/lib/services/descriptorResolution.test.ts index 69c68a51..2defc884 100644 --- a/src/lib/services/descriptorResolution.test.ts +++ b/src/lib/services/descriptorResolution.test.ts @@ -1,5 +1,5 @@ import { describe, expect, it } from "vitest"; -import { resolvePropertyType } from "./descriptorResolution"; +import { resolveBuiltForm, resolvePropertyType } from "./descriptorResolution"; describe("resolvePropertyType", () => { it("prefers a landlord override over the EPC-derived value", () => { @@ -130,3 +130,65 @@ describe("resolvePropertyType", () => { ).toBe("9"); }); }); + +describe("resolveBuiltForm", () => { + it("prefers a landlord override over the EPC-derived value", () => { + expect( + resolveBuiltForm({ + override: "Detached", + isNewApproach: true, + lodgedBuiltForm: "4", // EPC code for Mid-Terrace + predictedBuiltForm: null, + legacyBuiltForm: null, + }), + ).toBe("Detached"); + }); + + it("maps an RdSAP built-form code to its label when there is no override", () => { + expect( + resolveBuiltForm({ + override: null, + isNewApproach: true, + lodgedBuiltForm: "2", // RdSAP code for Semi-Detached + predictedBuiltForm: null, + legacyBuiltForm: null, + }), + ).toBe("Semi-Detached"); + }); + + it("falls back to the predicted EPC when there is no lodged value", () => { + expect( + resolveBuiltForm({ + override: null, + isNewApproach: true, + lodgedBuiltForm: null, + predictedBuiltForm: "1", // Detached + legacyBuiltForm: null, + }), + ).toBe("Detached"); + }); + + it("reads the legacy property-row value for a legacy property", () => { + expect( + resolveBuiltForm({ + override: null, + isNewApproach: false, + lodgedBuiltForm: "1", // must be ignored for legacy + predictedBuiltForm: null, + legacyBuiltForm: "End-Terrace", + }), + ).toBe("End-Terrace"); + }); + + it("resolves to 'Unknown' when nothing is available (never null)", () => { + expect( + resolveBuiltForm({ + override: null, + isNewApproach: true, + lodgedBuiltForm: null, + predictedBuiltForm: null, + legacyBuiltForm: "Detached", // legacy row must NOT surface for new-approach + }), + ).toBe("Unknown"); + }); +}); diff --git a/src/lib/services/descriptorResolution.ts b/src/lib/services/descriptorResolution.ts index 18dc015b..8abc0047 100644 --- a/src/lib/services/descriptorResolution.ts +++ b/src/lib/services/descriptorResolution.ts @@ -10,7 +10,7 @@ * landlord override → EPC-derived (lodged over predicted, codes mapped; * legacy property-row fallback) → 'Unknown' (never null) */ -import { PROPERTY_TYPE_LABELS } from "./epcSources"; +import { BUILT_FORM_LABELS, PROPERTY_TYPE_LABELS } from "./epcSources"; export interface PropertyTypeInputs { /** property_overrides.override_value for component 'property_type', building_part 0. */ @@ -47,3 +47,28 @@ function epcPropertyType( if (propertyType != null) return PROPERTY_TYPE_LABELS[propertyType] ?? propertyType; return dwellingType; } + +export interface BuiltFormInputs { + /** property_overrides.override_value for component 'built_form_type', building_part 0. */ + override: string | null; + isNewApproach: boolean; + /** epc_property(source='lodged').built_form — RdSAP code or text. */ + lodgedBuiltForm: string | null; + predictedBuiltForm: string | null; + /** property.built_form — the legacy row value. */ + legacyBuiltForm: string | null; +} + +export function resolveBuiltForm(i: BuiltFormInputs): string { + if (i.override) return i.override; + const derived = i.isNewApproach + ? epcBuiltForm(i.lodgedBuiltForm) ?? epcBuiltForm(i.predictedBuiltForm) + : i.legacyBuiltForm; + return derived ?? "Unknown"; +} + +/** One EPC source's built-form label: RdSAP code → label, text passes through. Mirrors builtFormTypeSql. */ +function epcBuiltForm(builtForm: string | null): string | null { + if (builtForm == null) return null; + return BUILT_FORM_LABELS[builtForm] ?? builtForm; +} diff --git a/src/lib/services/epcSources.ts b/src/lib/services/epcSources.ts index 3392d667..88473813 100644 --- a/src/lib/services/epcSources.ts +++ b/src/lib/services/epcSources.ts @@ -308,6 +308,17 @@ export const propertyTypeOverrideJoin = sql` /** property_type: override → EPC-derived → 'Unknown'. Mirrors resolvePropertyType. */ export const resolvedPropertyTypeSql = sql`COALESCE(pot.override_value, ${propertyTypeSql}, 'Unknown')`; +/** LEFT JOIN exposing the built_form_type override snapshot (alias `pobf`). */ +export const builtFormOverrideJoin = sql` + LEFT JOIN property_overrides pobf + ON pobf.property_id = p.id + AND pobf.override_component = 'built_form_type' + AND pobf.building_part = 0 +`; + +/** built_form: override → EPC-derived → 'Unknown'. Mirrors resolveBuiltForm. */ +export const resolvedBuiltFormSql = sql`COALESCE(pobf.override_value, ${builtFormTypeSql}, 'Unknown')`; + /** * The main building part's construction age band for one epc_property alias, as * a correlated scalar subquery — properties can also have extension parts, so a From 558e20f2feb0620bb47afb93dbfbb447701d4094 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Fri, 10 Jul 2026 23:35:14 +0000 Subject: [PATCH 04/11] feat(portfolio): band-native Construction Age column + filter (was Year Built) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Slice 3. Construction age is modelled as an RdSAP band, not a year (ADR-0012) — the only representation that faithfully carries an 'Unknown' override. resolve- ConstructionAge owns override(letter)→label / EPC band(letter)→label / legacy year→band bucketing / 'Unknown', with 10 unit tests incl. band boundaries. resolvedConstructionAgeSql mirrors it (emits the band label). YEAR_BUILT_OPTIONS becomes the band labels + Unknown, guarded against label/en-dash drift by a new epcSources.test. Column + filter + CSV header renamed to 'Construction Age' (internal 'yearBuilt' key retained to avoid a wide rename). Co-Authored-By: Claude Opus 4.8 (1M context) --- .../[slug]/components/PropertyFilters.tsx | 2 +- .../[slug]/components/PropertyTable.tsx | 2 +- .../components/propertyTableColumns.tsx | 4 +- src/app/portfolio/[slug]/utils.ts | 13 ++- src/app/utils/propertyFilters.ts | 13 ++- src/lib/services/descriptorResolution.test.ts | 99 ++++++++++++++++++- src/lib/services/descriptorResolution.ts | 56 ++++++++++- src/lib/services/epcSources.test.ts | 25 ++++- src/lib/services/epcSources.ts | 46 +++++++++ 9 files changed, 242 insertions(+), 18 deletions(-) diff --git a/src/app/portfolio/[slug]/components/PropertyFilters.tsx b/src/app/portfolio/[slug]/components/PropertyFilters.tsx index 7add0920..a89bc9ad 100644 --- a/src/app/portfolio/[slug]/components/PropertyFilters.tsx +++ b/src/app/portfolio/[slug]/components/PropertyFilters.tsx @@ -33,7 +33,7 @@ const FIELD_OPTIONS: { value: FilterField; label: string }[] = [ { value: "propertyType", label: "Property Type" }, { value: "builtForm", label: "Built Form" }, { value: "tenure", label: "Tenure" }, - { value: "yearBuilt", label: "Year Built" }, + { value: "yearBuilt", label: "Construction Age" }, { value: "provenance", label: "Provenance" }, { value: "floorArea", label: "Floor Area (m²)" }, { value: "co2Emissions", label: "CO₂ Emissions (kg/m²/yr)" }, diff --git a/src/app/portfolio/[slug]/components/PropertyTable.tsx b/src/app/portfolio/[slug]/components/PropertyTable.tsx index b5a87197..eaa8867b 100644 --- a/src/app/portfolio/[slug]/components/PropertyTable.tsx +++ b/src/app/portfolio/[slug]/components/PropertyTable.tsx @@ -85,7 +85,7 @@ function exportToCsv(data: PropertyWithRelations[]) { "Property Type", "Built Form", "Tenure", - "Year Built", + "Construction Age", "Floor Area (m²)", "CO₂ Emissions (kg/m²/yr)", "Main Fuel", diff --git a/src/app/portfolio/[slug]/components/propertyTableColumns.tsx b/src/app/portfolio/[slug]/components/propertyTableColumns.tsx index 9d2955d5..f2aee81a 100644 --- a/src/app/portfolio/[slug]/components/propertyTableColumns.tsx +++ b/src/app/portfolio/[slug]/components/propertyTableColumns.tsx @@ -181,7 +181,7 @@ const OPTIONAL_COLUMN_LABELS: Record = { propertyType: "Property Type", builtForm: "Built Form", tenure: "Tenure", - yearBuilt: "Year Built", + yearBuilt: "Construction Age", totalFloorArea: "Floor Area (m²)", co2Emissions: "CO₂ Emissions", mainfuel: "Main Fuel", @@ -434,7 +434,7 @@ const optionalColumns: ColumnDef[] = [ { id: "yearBuilt", accessorKey: "yearBuilt", - header: () =>
Year Built
, + header: () =>
Construction Age
, cell: ({ row }) => (
{row.original.yearBuilt ?? "—"}
), diff --git a/src/app/portfolio/[slug]/utils.ts b/src/app/portfolio/[slug]/utils.ts index e8d6a91f..b6b2859e 100644 --- a/src/app/portfolio/[slug]/utils.ts +++ b/src/app/portfolio/[slug]/utils.ts @@ -35,6 +35,8 @@ import { resolvedPropertyTypeSql, builtFormOverrideJoin, resolvedBuiltFormSql, + constructionAgeOverrideJoin, + resolvedConstructionAgeSql, } from "@/lib/services/epcSources"; import { FilterGroups, @@ -603,7 +605,7 @@ function buildConditionSql(filter: PropertyFilter): ReturnType | nul propertyType: resolvedPropertyTypeSql, builtForm: resolvedBuiltFormSql, tenure: sql`p.tenure`, - yearBuilt: sql`p.year_built`, + yearBuilt: resolvedConstructionAgeSql, provenance: provenanceSignalSql, // GAP: new-approach properties have no main-fuel string → NULL, so they // only match the "no value" filter branch. See epcSources.ts. @@ -703,6 +705,7 @@ const EPC_JOIN_FILTER_FIELDS = new Set([ // Override-resolved (ADR-0012): needs the EPC graph joins + the override join. "propertyType", "builtForm", + "yearBuilt", ]); const PLAN_JOIN_FILTER_FIELDS = new Set(["expectedEpc"]); @@ -730,7 +733,8 @@ export async function getPropertiesCount( ? sql`LEFT JOIN property_details_epc epc ON epc.property_id = p.id ${newApproachJoins} ${propertyTypeOverrideJoin} - ${builtFormOverrideJoin}` + ${builtFormOverrideJoin} + ${constructionAgeOverrideJoin}` : sql``; const planJoin = needsPlanJoin ? sql`LEFT JOIN LATERAL ( @@ -805,7 +809,9 @@ export async function getProperties( ${resolvedPropertyTypeSql} AS "propertyType", ${resolvedBuiltFormSql} AS "builtForm", p.tenure AS tenure, - p.year_built AS "yearBuilt", + -- Construction Age is band-native and override-resolved (ADR-0012); the + -- "yearBuilt" key is retained to avoid a wide rename. See resolveConstructionAge. + ${resolvedConstructionAgeSql} AS "yearBuilt", ${lodgementDateSql(sql`epc`)}::text AS "epcLodgementDate", ${isExpiredSql(sql`epc`)} AS "epcIsExpired", ${totalFloorAreaSql(sql`epc`)} AS "totalFloorArea", @@ -848,6 +854,7 @@ export async function getProperties( ${newApproachJoins} ${propertyTypeOverrideJoin} ${builtFormOverrideJoin} + ${constructionAgeOverrideJoin} WHERE p.portfolio_id = ${portfolioId} -- Unmatched (no-UPRN) properties live in the "Needs attention" tab until -- resolved, so they're excluded from the main table. diff --git a/src/app/utils/propertyFilters.ts b/src/app/utils/propertyFilters.ts index 88add78a..b1db3479 100644 --- a/src/app/utils/propertyFilters.ts +++ b/src/app/utils/propertyFilters.ts @@ -109,12 +109,15 @@ export const PROVENANCE_OPTIONS: EnumOption[] = [ { label: "Standard", dbValues: ["none"] }, ]; +// Construction Age is band-native (ADR-0012): labels + dbValues are the RdSAP +// band labels emitted by resolvedConstructionAgeSql. These MUST match +// CONSTRUCTION_AGE_BANDS[*].label in epcSources.ts exactly (note the en-dash); +// epcSources.test.ts guards against drift. "Unknown" is the never-null terminal. export const YEAR_BUILT_OPTIONS: EnumOption[] = [ - "1900","1930","1950","1967","1976","1983","1991","1996", - "2003","2007","2008","2009","2010","2011","2012","2013", - "2014","2015","2016","2017","2018","2019","2020","2021", - "2022","2023","2024","2025", -].map((y) => ({ label: y, dbValues: [y] })); + "before 1900", "1900–1929", "1930–1949", "1950–1966", "1967–1975", + "1976–1982", "1983–1990", "1991–1995", "1996–2002", "2003–2006", + "2007–2011", "2012–2022", "2023 onwards", "Unknown", +].map((label) => ({ label, dbValues: [label] })); export const MAINFUEL_OPTIONS: EnumOption[] = [ { label: "Mains Gas", dbValues: ["Gas mains gas", "Mains gas community", "Mains gas not community", "Mains gas this is for backwards compatibility only and should not be used"] }, diff --git a/src/lib/services/descriptorResolution.test.ts b/src/lib/services/descriptorResolution.test.ts index 2defc884..763ee857 100644 --- a/src/lib/services/descriptorResolution.test.ts +++ b/src/lib/services/descriptorResolution.test.ts @@ -1,5 +1,9 @@ import { describe, expect, it } from "vitest"; -import { resolveBuiltForm, resolvePropertyType } from "./descriptorResolution"; +import { + resolveBuiltForm, + resolveConstructionAge, + resolvePropertyType, +} from "./descriptorResolution"; describe("resolvePropertyType", () => { it("prefers a landlord override over the EPC-derived value", () => { @@ -192,3 +196,96 @@ describe("resolveBuiltForm", () => { ).toBe("Unknown"); }); }); + +describe("resolveConstructionAge", () => { + it("maps a landlord override band letter to its band label", () => { + expect( + resolveConstructionAge({ + override: "B", + isNewApproach: true, + lodgedAgeBand: "F", // EPC says a different band — override still wins + predictedAgeBand: null, + legacyYearBuilt: null, + }), + ).toBe("1900–1929"); + }); + + it("maps the lodged EPC band letter to its label when there is no override", () => { + expect( + resolveConstructionAge({ + override: null, + isNewApproach: true, + lodgedAgeBand: "C", + predictedAgeBand: null, + legacyYearBuilt: null, + }), + ).toBe("1930–1949"); + }); + + it("falls back to the predicted EPC band when there is no lodged band", () => { + expect( + resolveConstructionAge({ + override: null, + isNewApproach: true, + lodgedAgeBand: null, + predictedAgeBand: "M", + legacyYearBuilt: null, + }), + ).toBe("2023 onwards"); + }); + + it("buckets a legacy year_built into its band (mid-range)", () => { + expect(legacyAge("1985")).toBe("1983–1990"); + }); + + it("buckets a legacy year before 1900 into band A", () => { + expect(legacyAge("1899")).toBe("before 1900"); + }); + + it("buckets the first year of a band correctly (boundary)", () => { + expect(legacyAge("1900")).toBe("1900–1929"); + }); + + it("buckets a recent legacy year into the open-ended final band", () => { + expect(legacyAge("2025")).toBe("2023 onwards"); + }); + + it("resolves a non-numeric legacy year_built to 'Unknown'", () => { + expect(legacyAge("N/A")).toBe("Unknown"); + }); + + it("resolves to 'Unknown' when nothing is available; ignores the legacy row for new-approach", () => { + expect( + resolveConstructionAge({ + override: null, + isNewApproach: true, + lodgedAgeBand: null, + predictedAgeBand: null, + legacyYearBuilt: "1985", // must NOT surface for a new-approach property + }), + ).toBe("Unknown"); + }); + + it("lets an explicit 'Unknown' override win", () => { + expect( + resolveConstructionAge({ + override: "Unknown", + isNewApproach: true, + lodgedAgeBand: "C", + predictedAgeBand: null, + legacyYearBuilt: null, + }), + ).toBe("Unknown"); + }); +}); + +/** Legacy-property helper: only year_built is set, cutoff is false. */ +function legacyAge(year: string): string { + return resolveConstructionAge({ + override: null, + isNewApproach: false, + lodgedAgeBand: null, + predictedAgeBand: null, + legacyYearBuilt: year, + }); +} diff --git a/src/lib/services/descriptorResolution.ts b/src/lib/services/descriptorResolution.ts index 8abc0047..5fd3a45d 100644 --- a/src/lib/services/descriptorResolution.ts +++ b/src/lib/services/descriptorResolution.ts @@ -10,7 +10,11 @@ * landlord override → EPC-derived (lodged over predicted, codes mapped; * legacy property-row fallback) → 'Unknown' (never null) */ -import { BUILT_FORM_LABELS, PROPERTY_TYPE_LABELS } from "./epcSources"; +import { + BUILT_FORM_LABELS, + CONSTRUCTION_AGE_BANDS, + PROPERTY_TYPE_LABELS, +} from "./epcSources"; export interface PropertyTypeInputs { /** property_overrides.override_value for component 'property_type', building_part 0. */ @@ -72,3 +76,53 @@ function epcBuiltForm(builtForm: string | null): string | null { if (builtForm == null) return null; return BUILT_FORM_LABELS[builtForm] ?? builtForm; } + +export interface ConstructionAgeInputs { + /** property_overrides.override_value for 'construction_age_band', building_part 0 — an RdSAP letter (A..M) or "Unknown". */ + override: string | null; + isNewApproach: boolean; + /** The main building part's epc_building_part.construction_age_band — an RdSAP letter. */ + lodgedAgeBand: string | null; + predictedAgeBand: string | null; + /** property.year_built — a raw year string (legacy). Bucketed into a band. */ + legacyYearBuilt: string | null; +} + +/** + * Construction age as an RdSAP band LABEL ("before 1900" … "2023 onwards", + * "Unknown") — the band is the underlying datum, never a single year (ADR-0012). + * Override and EPC both store the band letter; legacy year_built is bucketed into + * a band. Mirrors resolvedConstructionAgeSql. + */ +export function resolveConstructionAge(i: ConstructionAgeInputs): string { + if (i.override) return bandLetterToLabel(i.override); + const band = i.isNewApproach + ? i.lodgedAgeBand ?? i.predictedAgeBand + : bucketLegacyYearToBand(i.legacyYearBuilt); + return band != null ? bandLetterToLabel(band) : "Unknown"; +} + +/** RdSAP band letter → band label; unrecognised values (e.g. "Unknown") pass through. */ +function bandLetterToLabel(letter: string): string { + return CONSTRUCTION_AGE_BANDS[letter]?.label ?? letter; +} + +/** + * Bucket a legacy raw year_built into its RdSAP band letter: the band with the + * greatest start year not after the given year. Non-numeric text → null (Unknown); + * a year below the earliest start is pre-1900 → band A. Mirrors the legacy branch + * of resolvedConstructionAgeSql. + */ +function bucketLegacyYearToBand(yearBuilt: string | null): string | null { + if (yearBuilt == null || !/^[0-9]+$/.test(yearBuilt)) return null; + const year = parseInt(yearBuilt, 10); + let best: string | null = null; + let bestStart = -Infinity; + for (const [letter, { year: start }] of Object.entries(CONSTRUCTION_AGE_BANDS)) { + if (start <= year && start > bestStart) { + best = letter; + bestStart = start; + } + } + return best ?? "A"; +} diff --git a/src/lib/services/epcSources.test.ts b/src/lib/services/epcSources.test.ts index 1d85a6ab..5b859fd9 100644 --- a/src/lib/services/epcSources.test.ts +++ b/src/lib/services/epcSources.test.ts @@ -3,7 +3,10 @@ import { CONSTRUCTION_AGE_BANDS, PROPERTY_TYPE_LABELS, } from "./epcSources"; -import { PROPERTY_TYPE_OPTIONS } from "@/app/utils/propertyFilters"; +import { + PROPERTY_TYPE_OPTIONS, + YEAR_BUILT_OPTIONS, +} from "@/app/utils/propertyFilters"; // The reporting age-band bucketing in getCountByAgeBand (databaseFunctions.ts), // mirrored so the CONSTRUCTION_AGE_BANDS representative years stay anchored to @@ -57,12 +60,26 @@ describe("CONSTRUCTION_AGE_BANDS", () => { }); describe("PROPERTY_TYPE_LABELS", () => { - it("maps codes to the labels the property-type filter options use", () => { + it("maps every code to a label the property-type filter offers as an option", () => { const optionLabels = new Set(PROPERTY_TYPE_OPTIONS.map((o) => o.label)); for (const label of Object.values(PROPERTY_TYPE_LABELS)) { - // Park home has no filter option — it's rare and charts render it as-is. - if (label === "Park home") continue; expect(optionLabels).toContain(label); } }); }); + +describe("YEAR_BUILT_OPTIONS (Construction Age)", () => { + // Guards the band-native filter (ADR-0012) against label drift — a mismatched + // en-dash or reworded band would silently make the filter match nothing. + it("is exactly the RdSAP band labels in order, plus Unknown", () => { + const bandLabels = Object.values(CONSTRUCTION_AGE_BANDS).map((b) => b.label); + expect(YEAR_BUILT_OPTIONS.map((o) => o.label)).toEqual([ + ...bandLabels, + "Unknown", + ]); + }); + + it("uses each band label as its own dbValue", () => { + for (const o of YEAR_BUILT_OPTIONS) expect(o.dbValues).toEqual([o.label]); + }); +}); diff --git a/src/lib/services/epcSources.ts b/src/lib/services/epcSources.ts index 88473813..5f8466c3 100644 --- a/src/lib/services/epcSources.ts +++ b/src/lib/services/epcSources.ts @@ -352,6 +352,52 @@ export const constructionYearSql = sql`CASE ELSE NULL END`; +// ─── Construction age as a band LABEL (ADR-0012) ───────────────────────────── +// Unlike constructionYearSql (which flattens the band to a representative year +// for reporting buckets), the portfolio list keeps the band itself. Mirrors +// resolveConstructionAge in descriptorResolution.ts — keep in sync. + +/** RdSAP band letter → band label (e.g. B → "1900–1929"). */ +const AGE_BAND_LABELS: Record = Object.fromEntries( + Object.entries(CONSTRUCTION_AGE_BANDS).map(([letter, { label }]) => [letter, label]), +); + +/** Bucket a legacy year_built expression into its band label; non-numeric → NULL. */ +const legacyYearBandLabelSql = (yearExpr: Alias) => { + const y = sql`CAST(${yearExpr} AS int)`; + const thresholds = Object.entries(CONSTRUCTION_AGE_BANDS) + .filter(([letter]) => letter !== "A") + .sort((a, b) => b[1].year - a[1].year) + .map(([, { year, label }]) => sql`WHEN ${y} >= ${year} THEN ${label}`); + // The numeric guard fires the CAST only on digit strings; below the earliest + // band start falls through to band A. + return sql`CASE WHEN ${yearExpr} ~ '^[0-9]+$' THEN (CASE ${sql.join( + thresholds, + sql` `, + )} ELSE ${CONSTRUCTION_AGE_BANDS.A.label} END) ELSE NULL END`; +}; + +/** EPC-or-legacy band label: new-approach main-part band (lodged over predicted); legacy year bucketed. */ +const constructionAgeBandLabelSql = sql`CASE + WHEN ${isNewApproachSql} THEN (CASE WHEN epl.id IS NOT NULL THEN ${codeLabelCaseSql(mainPartAgeBandSql(sql`epl`), AGE_BAND_LABELS)} ELSE ${codeLabelCaseSql(mainPartAgeBandSql(sql`epp`), AGE_BAND_LABELS)} END) + ELSE ${legacyYearBandLabelSql(sql`p.year_built`)} +END`; + +/** LEFT JOIN exposing the construction_age_band override snapshot (alias `pab`). */ +export const constructionAgeOverrideJoin = sql` + LEFT JOIN property_overrides pab + ON pab.property_id = p.id + AND pab.override_component = 'construction_age_band' + AND pab.building_part = 0 +`; + +/** construction age: override → EPC band → legacy year bucket → 'Unknown'. Mirrors resolveConstructionAge. */ +export const resolvedConstructionAgeSql = sql`COALESCE( + ${codeLabelCaseSql(sql`pab.override_value`, AGE_BAND_LABELS)}, + ${constructionAgeBandLabelSql}, + 'Unknown' +)`; + // ───────────────────────────────────────────────────────────────────────────── // Per-property resolvers (TypeScript object building). // ───────────────────────────────────────────────────────────────────────────── From fc2f7d5b30ffb9fc7bc2b6beb3fa398375368262 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Fri, 10 Jul 2026 23:51:40 +0000 Subject: [PATCH 05/11] feat(portfolio): resolve Main Fuel column + filter via override precedence MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Slice 4. The new EPC graph exposes no main-fuel string (documented gap), so for new-approach properties the landlord override is the only source. resolveMainFuel returns the raw value (override → legacy EPC → 'Unknown'); the cell's resolveEnumLabel folds it to a chip. MAINFUEL_OPTIONS.dbValues extended with the override vocabulary so overridden fuels are both displayable and filterable — guarded by a coverage test over MainFuelValues. resolvedMainFuelSql mirrors it. Co-Authored-By: Claude Opus 4.8 (1M context) --- src/app/portfolio/[slug]/utils.ts | 18 ++++--- src/app/utils/propertyFilters.ts | 21 +++++--- src/lib/services/descriptorResolution.test.ts | 52 +++++++++++++++++++ src/lib/services/descriptorResolution.ts | 20 +++++++ src/lib/services/epcSources.ts | 17 ++++++ 5 files changed, 114 insertions(+), 14 deletions(-) diff --git a/src/app/portfolio/[slug]/utils.ts b/src/app/portfolio/[slug]/utils.ts index b6b2859e..7d32dd53 100644 --- a/src/app/portfolio/[slug]/utils.ts +++ b/src/app/portfolio/[slug]/utils.ts @@ -25,7 +25,6 @@ import { totalFloorAreaSql, lodgementDateSql, isExpiredSql, - mainfuelSql, effectiveSapSql, effectiveEpcBandSql, lodgedEpcBandSql, @@ -37,6 +36,8 @@ import { resolvedBuiltFormSql, constructionAgeOverrideJoin, resolvedConstructionAgeSql, + mainFuelOverrideJoin, + resolvedMainFuelSql, } from "@/lib/services/epcSources"; import { FilterGroups, @@ -607,9 +608,10 @@ function buildConditionSql(filter: PropertyFilter): ReturnType | nul tenure: sql`p.tenure`, yearBuilt: resolvedConstructionAgeSql, provenance: provenanceSignalSql, - // GAP: new-approach properties have no main-fuel string → NULL, so they - // only match the "no value" filter branch. See epcSources.ts. - mainfuel: mainfuelSql(sql`epc`), + // Override-resolved (ADR-0012): the new EPC graph has no main fuel, so a + // new-approach property matches a fuel filter only via a landlord override; + // otherwise it resolves to 'Unknown'. + mainfuel: resolvedMainFuelSql(sql`epc`), }; const col = colMap[filter.field]; @@ -734,7 +736,8 @@ export async function getPropertiesCount( ${newApproachJoins} ${propertyTypeOverrideJoin} ${builtFormOverrideJoin} - ${constructionAgeOverrideJoin}` + ${constructionAgeOverrideJoin} + ${mainFuelOverrideJoin}` : sql``; const planJoin = needsPlanJoin ? sql`LEFT JOIN LATERAL ( @@ -816,7 +819,9 @@ export async function getProperties( ${isExpiredSql(sql`epc`)} AS "epcIsExpired", ${totalFloorAreaSql(sql`epc`)} AS "totalFloorArea", ${carbonSql(sql`epc`)} AS "co2Emissions", - ${mainfuelSql(sql`epc`)} AS mainfuel, + -- Main Fuel: override → EPC-derived → 'Unknown'. The new EPC graph has no + -- main fuel, so overrides are the only new-approach source (ADR-0012). + ${resolvedMainFuelSql(sql`epc`)} AS mainfuel, p.lexiscore AS lexiscore FROM property p -- LATERAL one-row lookups keep this query at "N properties × index probe" @@ -855,6 +860,7 @@ export async function getProperties( ${propertyTypeOverrideJoin} ${builtFormOverrideJoin} ${constructionAgeOverrideJoin} + ${mainFuelOverrideJoin} WHERE p.portfolio_id = ${portfolioId} -- Unmatched (no-UPRN) properties live in the "Needs attention" tab until -- resolved, so they're excluded from the main table. diff --git a/src/app/utils/propertyFilters.ts b/src/app/utils/propertyFilters.ts index b1db3479..792d4d0b 100644 --- a/src/app/utils/propertyFilters.ts +++ b/src/app/utils/propertyFilters.ts @@ -119,19 +119,24 @@ export const YEAR_BUILT_OPTIONS: EnumOption[] = [ "2007–2011", "2012–2022", "2023 onwards", "Unknown", ].map((label) => ({ label, dbValues: [label] })); +// dbValues fold two vocabularies onto one user-facing label: the raw EPC +// mainfuel strings (legacy) AND the landlord main_fuel override values +// ("mains gas", "LPG (bulk)", … — see MainFuelValues). resolvedMainFuelSql emits +// whichever applies; both must resolve to the same chip (ADR-0012). A +// descriptorResolution.test asserts every override value is covered here. export const MAINFUEL_OPTIONS: EnumOption[] = [ - { label: "Mains Gas", dbValues: ["Gas mains gas", "Mains gas community", "Mains gas not community", "Mains gas this is for backwards compatibility only and should not be used"] }, - { label: "LPG", dbValues: ["Bottled lpg", "Lpg community", "Lpg not community", "Lpg this is for backwards compatibility only and should not be used"] }, - { label: "Oil", dbValues: ["Oil heating oil", "Oil community", "Oil not community", "Oil this is for backwards compatibility only and should not be used"] }, - { label: "Electricity", dbValues: ["Electricity electricity unspecified tariff", "Electricity community", "Electricity not community", "Electricity this is for backwards compatibility only and should not be used"] }, - { label: "Biomass", dbValues: ["Bulk wood pellets", "Wood chips", "Wood logs", "Biomass community", "Biomass this is for backwards compatibility only and should not be used"] }, - { label: "Coal", dbValues: ["Anthracite", "House coal not community", "House coal this is for backwards compatibility only and should not be used", "Smokeless coal", "Coal community"] }, - { label: "Dual Fuel (Mineral + Wood)", dbValues: ["Dual fuel mineral wood"] }, + { label: "Mains Gas", dbValues: ["Gas mains gas", "Mains gas community", "Mains gas not community", "Mains gas this is for backwards compatibility only and should not be used", "mains gas", "mains gas (community)"] }, + { label: "LPG", dbValues: ["Bottled lpg", "Lpg community", "Lpg not community", "Lpg this is for backwards compatibility only and should not be used", "LPG (bulk)", "bottled LPG", "LPG special condition"] }, + { label: "Oil", dbValues: ["Oil heating oil", "Oil community", "Oil not community", "Oil this is for backwards compatibility only and should not be used", "oil"] }, + { label: "Electricity", dbValues: ["Electricity electricity unspecified tariff", "Electricity community", "Electricity not community", "Electricity this is for backwards compatibility only and should not be used", "electricity", "electricity (community)"] }, + { label: "Biomass", dbValues: ["Bulk wood pellets", "Wood chips", "Wood logs", "Biomass community", "Biomass this is for backwards compatibility only and should not be used", "biomass (community)", "wood logs"] }, + { label: "Coal", dbValues: ["Anthracite", "House coal not community", "House coal this is for backwards compatibility only and should not be used", "Smokeless coal", "Coal community", "house coal", "smokeless coal"] }, + { label: "Dual Fuel (Mineral + Wood)", dbValues: ["Dual fuel mineral wood", "dual fuel (mineral and wood)"] }, { label: "Biogas", dbValues: ["Biogas not community"] }, { label: "Biodiesel", dbValues: ["Heat from boilers using biodiesel from any biomass source community"] }, { label: "B30d (Biodiesel blend)", dbValues: ["B30d community"] }, { label: "B30k (Biodiesel blend)", dbValues: ["B30k not community"] }, { label: "Community Heat Network", dbValues: ["From heat network data community"] }, { label: "No Heating System", dbValues: ["To be used only when there is no heatinghotwater system", "To be used only when there is no heatinghotwater system or data is from a community network"] }, - { label: "Unknown / No Data", dbValues: ["UNKNOWN", "NO DATA!"] }, + { label: "Unknown / No Data", dbValues: ["UNKNOWN", "NO DATA!", "Unknown"] }, ]; diff --git a/src/lib/services/descriptorResolution.test.ts b/src/lib/services/descriptorResolution.test.ts index 763ee857..8839ec7f 100644 --- a/src/lib/services/descriptorResolution.test.ts +++ b/src/lib/services/descriptorResolution.test.ts @@ -2,8 +2,11 @@ import { describe, expect, it } from "vitest"; import { resolveBuiltForm, resolveConstructionAge, + resolveMainFuel, resolvePropertyType, } from "./descriptorResolution"; +import { MAINFUEL_OPTIONS } from "@/app/utils/propertyFilters"; +import { MainFuelValues } from "@/app/db/schema/landlord_overrides"; describe("resolvePropertyType", () => { it("prefers a landlord override over the EPC-derived value", () => { @@ -279,6 +282,55 @@ describe("resolveConstructionAge", () => { }); }); +describe("resolveMainFuel", () => { + it("prefers a landlord override (the only main-fuel source for new-approach)", () => { + expect( + resolveMainFuel({ + override: "mains gas", + isNewApproach: true, + legacyMainFuel: null, + }), + ).toBe("mains gas"); + }); + + it("has no EPC main fuel for new-approach, so resolves to 'Unknown' without an override", () => { + expect( + resolveMainFuel({ + override: null, + isNewApproach: true, + legacyMainFuel: "Gas mains gas", // legacy value must NOT surface for new-approach + }), + ).toBe("Unknown"); + }); + + it("reads the legacy EPC main fuel for a legacy property", () => { + expect( + resolveMainFuel({ + override: null, + isNewApproach: false, + legacyMainFuel: "Gas mains gas", + }), + ).toBe("Gas mains gas"); + }); + + it("resolves to 'Unknown' when nothing is available (never null)", () => { + expect( + resolveMainFuel({ override: null, isNewApproach: false, legacyMainFuel: null }), + ).toBe("Unknown"); + }); +}); + +describe("MAINFUEL_OPTIONS coverage", () => { + // Every landlord main-fuel override value must fold into a filter option, or an + // overridden fuel would display but be unfilterable (ADR-0012). + it("covers every landlord main-fuel override value", () => { + const covered = new Set(MAINFUEL_OPTIONS.flatMap((o) => o.dbValues)); + for (const value of MainFuelValues) { + expect(covered).toContain(value); + } + }); +}); + /** Legacy-property helper: only year_built is set, cutoff is false. */ function legacyAge(year: string): string { return resolveConstructionAge({ diff --git a/src/lib/services/descriptorResolution.ts b/src/lib/services/descriptorResolution.ts index 5fd3a45d..cea0da3c 100644 --- a/src/lib/services/descriptorResolution.ts +++ b/src/lib/services/descriptorResolution.ts @@ -113,6 +113,26 @@ function bandLetterToLabel(letter: string): string { * a year below the earliest start is pre-1900 → band A. Mirrors the legacy branch * of resolvedConstructionAgeSql. */ +export interface MainFuelInputs { + /** property_overrides.override_value for component 'main_fuel', building_part 0. */ + override: string | null; + isNewApproach: boolean; + /** property_details_epc.mainfuel — the raw legacy EPC string. */ + legacyMainFuel: string | null; +} + +/** + * Main fuel raw value (folded to a label by the cell's resolveEnumLabel). The + * new EPC graph exposes no main-fuel string (documented gap in mainfuelSql), so + * for new-approach properties the landlord override is the only source; legacy + * reads the EPC string. Never null → 'Unknown'. Mirrors resolvedMainFuelSql. + */ +export function resolveMainFuel(i: MainFuelInputs): string { + if (i.override) return i.override; + const derived = i.isNewApproach ? null : i.legacyMainFuel; + return derived ?? "Unknown"; +} + function bucketLegacyYearToBand(yearBuilt: string | null): string | null { if (yearBuilt == null || !/^[0-9]+$/.test(yearBuilt)) return null; const year = parseInt(yearBuilt, 10); diff --git a/src/lib/services/epcSources.ts b/src/lib/services/epcSources.ts index 5f8466c3..e4743d70 100644 --- a/src/lib/services/epcSources.ts +++ b/src/lib/services/epcSources.ts @@ -319,6 +319,23 @@ export const builtFormOverrideJoin = sql` /** built_form: override → EPC-derived → 'Unknown'. Mirrors resolveBuiltForm. */ export const resolvedBuiltFormSql = sql`COALESCE(pobf.override_value, ${builtFormTypeSql}, 'Unknown')`; +/** LEFT JOIN exposing the main_fuel override snapshot (alias `pmf`). */ +export const mainFuelOverrideJoin = sql` + LEFT JOIN property_overrides pmf + ON pmf.property_id = p.id + AND pmf.override_component = 'main_fuel' + AND pmf.building_part = 0 +`; + +/** + * main fuel (raw value; folded to a label by the cell's resolveEnumLabel): + * override → EPC-derived → 'Unknown'. The new EPC graph has no main fuel, so + * mainfuelSql is NULL for new-approach — the override is the only new-approach + * source. Takes the legacy `property_details_epc` alias. Mirrors resolveMainFuel. + */ +export const resolvedMainFuelSql = (e: Alias) => + sql`COALESCE(pmf.override_value, ${mainfuelSql(e)}, 'Unknown')`; + /** * The main building part's construction age band for one epc_property alias, as * a correlated scalar subquery — properties can also have extension parts, so a From 78db9815badf1439be8ac49f28e2bc9a92712627 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Fri, 10 Jul 2026 23:56:42 +0000 Subject: [PATCH 06/11] feat(portfolio): remove Tenure from the property list column + filter Slice 5. Tenure was broken for new-approach (read p.tenure, NULL) but has no override component and its epc_property.tenure values don't reconcile with the filter dbValues, so it's removed from the list rather than fixed (ADR-0012): the optional column, the filter field, TENURE_OPTIONS, and the CSV column. The per-property building passport (resolvePropertyMeta/resolvePropertyDescriptors) still resolves tenure, and property.tenure is retained. Co-Authored-By: Claude Opus 4.8 (1M context) --- src/app/db/schema/property.ts | 1 - .../[slug]/components/PropertyFilters.tsx | 3 --- .../[slug]/components/PropertyTable.tsx | 8 +----- .../components/propertyTableColumns.tsx | 25 +------------------ src/app/portfolio/[slug]/utils.ts | 5 ---- src/app/utils/propertyFilters.ts | 10 -------- 6 files changed, 2 insertions(+), 50 deletions(-) diff --git a/src/app/db/schema/property.ts b/src/app/db/schema/property.ts index cb73917e..fb5bb51e 100644 --- a/src/app/db/schema/property.ts +++ b/src/app/db/schema/property.ts @@ -410,7 +410,6 @@ export interface PropertyWithRelations extends Record { // Optional columns (hidden by default) propertyType: string | null; builtForm: string | null; - tenure: string | null; yearBuilt: string | null; totalFloorArea: number | null; co2Emissions: number | null; diff --git a/src/app/portfolio/[slug]/components/PropertyFilters.tsx b/src/app/portfolio/[slug]/components/PropertyFilters.tsx index a89bc9ad..9bdf8b8e 100644 --- a/src/app/portfolio/[slug]/components/PropertyFilters.tsx +++ b/src/app/portfolio/[slug]/components/PropertyFilters.tsx @@ -13,7 +13,6 @@ import { EnumOption, PROPERTY_TYPE_OPTIONS, BUILT_FORM_OPTIONS, - TENURE_OPTIONS, YEAR_BUILT_OPTIONS, MAINFUEL_OPTIONS, PROVENANCE_OPTIONS, @@ -32,7 +31,6 @@ const FIELD_OPTIONS: { value: FilterField; label: string }[] = [ { value: "epcExpiryDate", label: "EPC Expiry Date" }, { value: "propertyType", label: "Property Type" }, { value: "builtForm", label: "Built Form" }, - { value: "tenure", label: "Tenure" }, { value: "yearBuilt", label: "Construction Age" }, { value: "provenance", label: "Provenance" }, { value: "floorArea", label: "Floor Area (m²)" }, @@ -74,7 +72,6 @@ const NUM_OPERATOR_OPTIONS: { value: FilterOperator; label: string }[] = [ const ENUM_FIELD_OPTIONS: Record = { propertyType: PROPERTY_TYPE_OPTIONS, builtForm: BUILT_FORM_OPTIONS, - tenure: TENURE_OPTIONS, yearBuilt: YEAR_BUILT_OPTIONS, provenance: PROVENANCE_OPTIONS, mainfuel: MAINFUEL_OPTIONS, diff --git a/src/app/portfolio/[slug]/components/PropertyTable.tsx b/src/app/portfolio/[slug]/components/PropertyTable.tsx index eaa8867b..f82bb843 100644 --- a/src/app/portfolio/[slug]/components/PropertyTable.tsx +++ b/src/app/portfolio/[slug]/components/PropertyTable.tsx @@ -22,11 +22,7 @@ import BulkUploadComingSoonModal from "@/app/components/portfolio/BulkUploadComi import { sapToEpc } from "@/app/utils"; import { columns } from "@/app/portfolio/[slug]/components/propertyTableColumns"; import { PropertyWithRelations } from "@/app/db/schema/property"; -import { - TENURE_OPTIONS, - MAINFUEL_OPTIONS, - EnumOption, -} from "@/app/utils/propertyFilters"; +import { MAINFUEL_OPTIONS, EnumOption } from "@/app/utils/propertyFilters"; import { OPTIONAL_COLUMN_IDS, OPTIONAL_COLUMN_LABELS, @@ -84,7 +80,6 @@ function exportToCsv(data: PropertyWithRelations[]) { "Plan Cost (£)", "Property Type", "Built Form", - "Tenure", "Construction Age", "Floor Area (m²)", "CO₂ Emissions (kg/m²/yr)", @@ -118,7 +113,6 @@ function exportToCsv(data: PropertyWithRelations[]) { p.totalRecommendationCost ? p.totalRecommendationCost.toFixed(2) : "", p.propertyType ?? "", p.builtForm ?? "", - resolveEnumLabel(TENURE_OPTIONS, p.tenure), p.yearBuilt ?? "", p.totalFloorArea != null ? p.totalFloorArea.toFixed(1) : "", p.co2Emissions != null ? p.co2Emissions.toFixed(1) : "", diff --git a/src/app/portfolio/[slug]/components/propertyTableColumns.tsx b/src/app/portfolio/[slug]/components/propertyTableColumns.tsx index f2aee81a..1a4df953 100644 --- a/src/app/portfolio/[slug]/components/propertyTableColumns.tsx +++ b/src/app/portfolio/[slug]/components/propertyTableColumns.tsx @@ -17,11 +17,7 @@ import { cn } from "@/lib/utils"; import { PropertyWithRelations } from "@/app/db/schema/property"; import { expectedEpcRating } from "./expectedEpc"; import { X } from "lucide-react"; -import { - EnumOption, - TENURE_OPTIONS, - MAINFUEL_OPTIONS, -} from "@/app/utils/propertyFilters"; +import { EnumOption, MAINFUEL_OPTIONS } from "@/app/utils/propertyFilters"; import { CurrentEpcTooltip } from "./CurrentEpcTooltip"; /* ----------------------------------------------------------------------- @@ -36,13 +32,6 @@ function resolveEnumLabel( return opt?.label ?? dbValue; } -function tenureBadgeClass(label: string): string { - if (label.toLowerCase().includes("owner")) return "bg-blue-50 text-blue-700"; - if (label.toLowerCase().includes("private")) return "bg-violet-50 text-violet-700"; - if (label.toLowerCase().includes("social")) return "bg-emerald-50 text-emerald-700"; - return "bg-slate-100 text-slate-500"; -} - function Pill({ children, className = "bg-slate-100 text-slate-600", @@ -168,7 +157,6 @@ export function DataTableFilterHeader({ export const OPTIONAL_COLUMN_IDS = [ "propertyType", "builtForm", - "tenure", "yearBuilt", "totalFloorArea", "co2Emissions", @@ -180,7 +168,6 @@ export type OptionalColumnId = (typeof OPTIONAL_COLUMN_IDS)[number]; const OPTIONAL_COLUMN_LABELS: Record = { propertyType: "Property Type", builtForm: "Built Form", - tenure: "Tenure", yearBuilt: "Construction Age", totalFloorArea: "Floor Area (m²)", co2Emissions: "CO₂ Emissions", @@ -421,16 +408,6 @@ const optionalColumns: ColumnDef[] = [ return val ? {val} : —; }, }, - { - id: "tenure", - accessorKey: "tenure", - header: () =>
Tenure
, - cell: ({ row }) => { - const label = resolveEnumLabel(TENURE_OPTIONS, row.original.tenure); - if (!label) return —; - return {label}; - }, - }, { id: "yearBuilt", accessorKey: "yearBuilt", diff --git a/src/app/portfolio/[slug]/utils.ts b/src/app/portfolio/[slug]/utils.ts index 7d32dd53..e30882c5 100644 --- a/src/app/portfolio/[slug]/utils.ts +++ b/src/app/portfolio/[slug]/utils.ts @@ -45,7 +45,6 @@ import { PropertyFilter, PROPERTY_TYPE_OPTIONS, BUILT_FORM_OPTIONS, - TENURE_OPTIONS, YEAR_BUILT_OPTIONS, MAINFUEL_OPTIONS, PROVENANCE_OPTIONS, @@ -56,7 +55,6 @@ import { EPC_TO_SAP_MIN, EPC_TO_SAP_MAX } from "@/app/utils/epc"; const ENUM_FIELD_DB_OPTIONS: Record = { propertyType: PROPERTY_TYPE_OPTIONS, builtForm: BUILT_FORM_OPTIONS, - tenure: TENURE_OPTIONS, yearBuilt: YEAR_BUILT_OPTIONS, provenance: PROVENANCE_OPTIONS, mainfuel: MAINFUEL_OPTIONS, @@ -586,7 +584,6 @@ function buildConditionSql(filter: PropertyFilter): ReturnType | nul case "propertyType": case "builtForm": - case "tenure": case "yearBuilt": case "provenance": case "mainfuel": { @@ -605,7 +602,6 @@ function buildConditionSql(filter: PropertyFilter): ReturnType | nul // Override-resolved to match the column + the Run filter (ADR-0012). propertyType: resolvedPropertyTypeSql, builtForm: resolvedBuiltFormSql, - tenure: sql`p.tenure`, yearBuilt: resolvedConstructionAgeSql, provenance: provenanceSignalSql, // Override-resolved (ADR-0012): the new EPC graph has no main fuel, so a @@ -811,7 +807,6 @@ export async function getProperties( -- See resolvePropertyType / ADR-0012. ${resolvedPropertyTypeSql} AS "propertyType", ${resolvedBuiltFormSql} AS "builtForm", - p.tenure AS tenure, -- Construction Age is band-native and override-resolved (ADR-0012); the -- "yearBuilt" key is retained to avoid a wide rename. See resolveConstructionAge. ${resolvedConstructionAgeSql} AS "yearBuilt", diff --git a/src/app/utils/propertyFilters.ts b/src/app/utils/propertyFilters.ts index 792d4d0b..0c906a85 100644 --- a/src/app/utils/propertyFilters.ts +++ b/src/app/utils/propertyFilters.ts @@ -8,7 +8,6 @@ export type FilterField = | "propertyRef" | "propertyType" | "builtForm" - | "tenure" | "yearBuilt" | "provenance" | "floorArea" @@ -92,15 +91,6 @@ export const BUILT_FORM_OPTIONS: EnumOption[] = [ { label: "Unknown", dbValues: ["Unknown"] }, ]; -export const TENURE_OPTIONS: EnumOption[] = [ - { label: "Owner-occupied", dbValues: ["Owner-occupied", "owner-occupied"] }, - { label: "Rented (Private)", dbValues: ["Rented (private)", "rental (private)", "rented (private)"] }, - { label: "Rented (Social)", dbValues: ["Rented (social)", "rental (social)"] }, - { label: "Not Defined", dbValues: ["Not defined - use in the case of a new dwelling for which the intended tenure in not known. It is not to be used for an existing dwelling"] }, - { label: "Unknown", dbValues: ["unknown"] }, - { label: "Not Recorded", dbValues: ["__null__"] }, -]; - // Provenance signal (ADR-0002). dbValues are the raw signal strings emitted by // provenanceSignalSql / resolveProvenanceSignal. export const PROVENANCE_OPTIONS: EnumOption[] = [ From b393b5efe01ba9765c7506593e2dba6df2824c62 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Sat, 11 Jul 2026 00:00:50 +0000 Subject: [PATCH 07/11] refactor(modelling): previewModellingRun consumes shared epcSources fragments MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Slice 6. The override → EPC-derived → 'Unknown' rule was inlined in previewModellingRun and duplicated by the portfolio list. Replace the inline COALESCE expressions and property_overrides joins with the shared resolvedPropertyTypeSql / resolvedBuiltFormSql / *OverrideJoin fragments — a byte-identical substitution — so the modelling preview and the list are provably in sync (the divergence ADR-0008 warns about is now structurally impossible). Co-Authored-By: Claude Opus 4.8 (1M context) --- src/lib/modellingRuns/server.ts | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/src/lib/modellingRuns/server.ts b/src/lib/modellingRuns/server.ts index 63f03c31..078abeb2 100644 --- a/src/lib/modellingRuns/server.ts +++ b/src/lib/modellingRuns/server.ts @@ -11,7 +11,12 @@ import { parseRunConfig, selectionSummary, } from "./model"; -import { builtFormTypeSql, propertyTypeSql } from "@/lib/services/epcSources"; +import { + builtFormOverrideJoin, + propertyTypeOverrideJoin, + resolvedBuiltFormSql, + resolvedPropertyTypeSql, +} from "@/lib/services/epcSources"; // Distributor entry point on the Model backend. It fans the run out to // workers, attaching execution sub-tasks to the task_id we pass (ADR-0008). @@ -147,14 +152,14 @@ export async function previewModellingRun(args: { per_scenario: { scenario_id: string; n: number }[] | null; }>(sql` WITH resolved AS ( + -- Resolution rule shared with the portfolio list via epcSources fragments + -- so the preview and the list can never drift (ADR-0008, ADR-0012). SELECT p.id, p.postcode, - COALESCE(pot.override_value, ${propertyTypeSql}, 'Unknown') AS ptype, - COALESCE(pobf.override_value, ${builtFormTypeSql}, 'Unknown') AS bform + ${resolvedPropertyTypeSql} AS ptype, + ${resolvedBuiltFormSql} AS bform FROM property p - LEFT JOIN property_overrides pot ON pot.property_id = p.id - AND pot.override_component = 'property_type' AND pot.building_part = 0 - LEFT JOIN property_overrides pobf ON pobf.property_id = p.id - AND pobf.override_component = 'built_form_type' AND pobf.building_part = 0 + ${propertyTypeOverrideJoin} + ${builtFormOverrideJoin} LEFT JOIN epc_property epl ON epl.property_id = p.id AND epl.source = 'lodged' LEFT JOIN epc_property epp ON epp.property_id = p.id AND epp.source = 'predicted' WHERE p.portfolio_id = ${pid} AND p.marked_for_deletion = false From 4f40c0b13dc2ad5f7e8e79bc59c2158db11e3078 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 13 Jul 2026 13:36:26 +0000 Subject: [PATCH 08/11] feat(portfolio): smaller list fetch, larger export, wall/roof/heating columns MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three property-list improvements: 1. Fetch 250 rows initially, not 1000 (DEFAULT_LIMIT/DISPLAY_LIMIT). The table paginates at 7/page and isn't bulk-scrolled, so the join-heavy query stays fast; load-more still pulls the rest on demand. 2. Export now fetches its own batch (up to EXPORT_LIMIT) for the current filters instead of dumping the on-screen rows — so the small display fetch doesn't cap it — and the cap is raised 1000 -> 6000. Export button shows a loading state. 3. Add Wall Type / Roof Type / Heating System optional columns, resolved through the same override precedence as the others (override -> EPC epc_energy_element description, lodged over predicted -> legacy property_details_epc -> 'Unknown'). Per-part components (wall/roof) read the main building part. New generic resolveOverrideDescriptor (+5 tests) with SQL twins in epcSources. Columns are display-only (no filter — the wall/roof override vocab is too large for chips) and included in the CSV export. Co-Authored-By: Claude Opus 4.8 (1M context) --- src/app/api/properties/route.ts | 5 +- src/app/db/schema/property.ts | 3 ++ .../[slug]/components/PropertyTable.tsx | 41 ++++++++++++-- .../components/propertyTableColumns.tsx | 40 ++++++++++++++ src/app/portfolio/[slug]/utils.ts | 14 +++++ src/lib/services/descriptorResolution.test.ts | 53 +++++++++++++++++++ src/lib/services/descriptorResolution.ts | 25 +++++++++ src/lib/services/epcSources.ts | 49 +++++++++++++++++ 8 files changed, 224 insertions(+), 6 deletions(-) diff --git a/src/app/api/properties/route.ts b/src/app/api/properties/route.ts index bf0239a5..6726e8be 100644 --- a/src/app/api/properties/route.ts +++ b/src/app/api/properties/route.ts @@ -2,7 +2,10 @@ import { NextRequest, NextResponse } from "next/server"; import { getProperties, getPropertiesCount } from "@/app/portfolio/[slug]/utils"; import { FilterGroups } from "@/app/utils/propertyFilters"; -const DEFAULT_LIMIT = 1000; +// The property list paginates at 7 rows/page and isn't scrolled through in bulk, +// so a small initial page keeps the (join-heavy) query fast. Export fetches its +// own larger batch on demand (see EXPORT_LIMIT in PropertyTable). +const DEFAULT_LIMIT = 250; export async function POST(req: NextRequest) { const body = await req.json(); diff --git a/src/app/db/schema/property.ts b/src/app/db/schema/property.ts index fb5bb51e..783a88ab 100644 --- a/src/app/db/schema/property.ts +++ b/src/app/db/schema/property.ts @@ -414,6 +414,9 @@ export interface PropertyWithRelations extends Record { totalFloorArea: number | null; co2Emissions: number | null; mainfuel: string | null; + wallType: string | null; + roofType: string | null; + heatingSystem: string | null; lexiscore: number | null; } diff --git a/src/app/portfolio/[slug]/components/PropertyTable.tsx b/src/app/portfolio/[slug]/components/PropertyTable.tsx index f82bb843..4e788d22 100644 --- a/src/app/portfolio/[slug]/components/PropertyTable.tsx +++ b/src/app/portfolio/[slug]/components/PropertyTable.tsx @@ -56,7 +56,7 @@ import { Button } from "@/app/shadcn_components/ui/button"; /* ---------------------------------------- Export helpers ----------------------------------------- */ -const EXPORT_LIMIT = 1000; +const EXPORT_LIMIT = 6000; function resolveEnumLabel( options: EnumOption[], @@ -84,6 +84,9 @@ function exportToCsv(data: PropertyWithRelations[]) { "Floor Area (m²)", "CO₂ Emissions (kg/m²/yr)", "Main Fuel", + "Wall Type", + "Roof Type", + "Heating System", ]; const rows = data.map((p) => { @@ -117,6 +120,9 @@ function exportToCsv(data: PropertyWithRelations[]) { p.totalFloorArea != null ? p.totalFloorArea.toFixed(1) : "", p.co2Emissions != null ? p.co2Emissions.toFixed(1) : "", resolveEnumLabel(MAINFUEL_OPTIONS, p.mainfuel), + p.wallType ?? "", + p.roofType ?? "", + p.heatingSystem ?? "", ]; }); @@ -403,7 +409,7 @@ export default function PropertyTable({ }); const totalCount = allResponse?.total ?? 0; - const DISPLAY_LIMIT = 1000; + const DISPLAY_LIMIT = 250; const LOAD_MORE_SIZE = 100; // ── Extra rows (lazy-loaded pages beyond the initial 1 000) ────────────── @@ -494,6 +500,31 @@ export default function PropertyTable({ if (isOnLastPage && hasMore) loadMore(); }, [isOnLastPage, hasMore, loadMore]); + // Export fetches its own batch (up to EXPORT_LIMIT) for the current filters, + // independent of the on-screen page — the display fetch is deliberately small. + const [isExporting, setIsExporting] = useState(false); + const handleExport = useCallback(async () => { + if (isExporting) return; + setIsExporting(true); + try { + const res = await fetch("/api/properties", { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ + portfolioId, + filters: allFilterGroups, + limit: EXPORT_LIMIT, + offset: 0, + }), + }); + if (!res.ok) return; + const json: { data: PropertyWithRelations[] } = await res.json(); + exportToCsv(json.data); + } finally { + setIsExporting(false); + } + }, [isExporting, portfolioId, allFilterGroups]); + /* ---------------------------------------- Delete preview state ----------------------------------------- */ @@ -615,12 +646,12 @@ export default function PropertyTable({ ) : ( )} diff --git a/src/app/portfolio/[slug]/components/propertyTableColumns.tsx b/src/app/portfolio/[slug]/components/propertyTableColumns.tsx index 1a4df953..11db3e2d 100644 --- a/src/app/portfolio/[slug]/components/propertyTableColumns.tsx +++ b/src/app/portfolio/[slug]/components/propertyTableColumns.tsx @@ -161,6 +161,9 @@ export const OPTIONAL_COLUMN_IDS = [ "totalFloorArea", "co2Emissions", "mainfuel", + "wallType", + "roofType", + "heatingSystem", ] as const; export type OptionalColumnId = (typeof OPTIONAL_COLUMN_IDS)[number]; @@ -172,6 +175,9 @@ const OPTIONAL_COLUMN_LABELS: Record = { totalFloorArea: "Floor Area (m²)", co2Emissions: "CO₂ Emissions", mainfuel: "Main Fuel", + wallType: "Wall Type", + roofType: "Roof Type", + heatingSystem: "Heating System", }; export { OPTIONAL_COLUMN_LABELS }; @@ -451,8 +457,42 @@ const optionalColumns: ColumnDef[] = [ return label ? {label} : —; }, }, + { + id: "wallType", + accessorKey: "wallType", + header: () =>
Wall Type
, + cell: ({ row }) => , + }, + { + id: "roofType", + accessorKey: "roofType", + header: () =>
Roof Type
, + cell: ({ row }) => , + }, + { + id: "heatingSystem", + accessorKey: "heatingSystem", + header: () =>
Heating System
, + cell: ({ row }) => , + }, ]; +/** Long free-text descriptor (wall/roof/heating): truncate with a full-value tooltip; muted when Unknown. */ +function DescriptorCell({ value }: { value: string | null }) { + const val = value ?? "Unknown"; + return ( +
+ {val} +
+ ); +} + export const columns: ColumnDef[] = [ ...coreColumns, ...optionalColumns, diff --git a/src/app/portfolio/[slug]/utils.ts b/src/app/portfolio/[slug]/utils.ts index e30882c5..2622f1c6 100644 --- a/src/app/portfolio/[slug]/utils.ts +++ b/src/app/portfolio/[slug]/utils.ts @@ -38,6 +38,12 @@ import { resolvedConstructionAgeSql, mainFuelOverrideJoin, resolvedMainFuelSql, + wallTypeOverrideJoin, + resolvedWallTypeSql, + roofTypeOverrideJoin, + resolvedRoofTypeSql, + mainHeatingOverrideJoin, + resolvedHeatingSystemSql, } from "@/lib/services/epcSources"; import { FilterGroups, @@ -817,6 +823,11 @@ export async function getProperties( -- Main Fuel: override → EPC-derived → 'Unknown'. The new EPC graph has no -- main fuel, so overrides are the only new-approach source (ADR-0012). ${resolvedMainFuelSql(sql`epc`)} AS mainfuel, + -- Wall / roof / heating: override → EPC element description → legacy → 'Unknown' + -- (ADR-0012). Per-part components (wall/roof) read the main building part. + ${resolvedWallTypeSql(sql`epc`)} AS "wallType", + ${resolvedRoofTypeSql(sql`epc`)} AS "roofType", + ${resolvedHeatingSystemSql(sql`epc`)} AS "heatingSystem", p.lexiscore AS lexiscore FROM property p -- LATERAL one-row lookups keep this query at "N properties × index probe" @@ -856,6 +867,9 @@ export async function getProperties( ${builtFormOverrideJoin} ${constructionAgeOverrideJoin} ${mainFuelOverrideJoin} + ${wallTypeOverrideJoin} + ${roofTypeOverrideJoin} + ${mainHeatingOverrideJoin} WHERE p.portfolio_id = ${portfolioId} -- Unmatched (no-UPRN) properties live in the "Needs attention" tab until -- resolved, so they're excluded from the main table. diff --git a/src/lib/services/descriptorResolution.test.ts b/src/lib/services/descriptorResolution.test.ts index 8839ec7f..533d3935 100644 --- a/src/lib/services/descriptorResolution.test.ts +++ b/src/lib/services/descriptorResolution.test.ts @@ -3,6 +3,7 @@ import { resolveBuiltForm, resolveConstructionAge, resolveMainFuel, + resolveOverrideDescriptor, resolvePropertyType, } from "./descriptorResolution"; import { MAINFUEL_OPTIONS } from "@/app/utils/propertyFilters"; @@ -331,6 +332,58 @@ describe("MAINFUEL_OPTIONS coverage", () => { }); }); +describe("resolveOverrideDescriptor (wall / roof / heating free-text)", () => { + const base = { + override: null, + isNewApproach: true, + lodgedDescription: null, + predictedDescription: null, + legacyDescription: null, + }; + + it("prefers a landlord override over the EPC description", () => { + expect( + resolveOverrideDescriptor({ + ...base, + override: "Cavity wall, filled cavity", + lodgedDescription: "Solid brick, as built, no insulation (assumed)", + }), + ).toBe("Cavity wall, filled cavity"); + }); + + it("uses the lodged EPC description when there is no override", () => { + expect( + resolveOverrideDescriptor({ + ...base, + lodgedDescription: "Pitched, 250 mm loft insulation", + }), + ).toBe("Pitched, 250 mm loft insulation"); + }); + + it("falls back to the predicted EPC description when there is no lodged one", () => { + expect( + resolveOverrideDescriptor({ ...base, predictedDescription: "Flat, insulated (assumed)" }), + ).toBe("Flat, insulated (assumed)"); + }); + + it("reads the legacy description for a legacy property", () => { + expect( + resolveOverrideDescriptor({ + ...base, + isNewApproach: false, + lodgedDescription: "ignored for legacy", + legacyDescription: "Boiler and radiators, mains gas", + }), + ).toBe("Boiler and radiators, mains gas"); + }); + + it("resolves to 'Unknown' when nothing is available; ignores legacy for new-approach", () => { + expect( + resolveOverrideDescriptor({ ...base, legacyDescription: "must not surface" }), + ).toBe("Unknown"); + }); +}); + /** Legacy-property helper: only year_built is set, cutoff is false. */ function legacyAge(year: string): string { return resolveConstructionAge({ diff --git a/src/lib/services/descriptorResolution.ts b/src/lib/services/descriptorResolution.ts index cea0da3c..4ebb3242 100644 --- a/src/lib/services/descriptorResolution.ts +++ b/src/lib/services/descriptorResolution.ts @@ -113,6 +113,31 @@ function bandLetterToLabel(letter: string): string { * a year below the earliest start is pre-1900 → band A. Mirrors the legacy branch * of resolvedConstructionAgeSql. */ +export interface OverrideDescriptorInputs { + /** property_overrides.override_value for this component, building_part 0. */ + override: string | null; + isNewApproach: boolean; + /** epc_energy_element.description for the element on the lodged EPC. */ + lodgedDescription: string | null; + predictedDescription: string | null; + /** The legacy property_details_epc column (walls / roof / heating). */ + legacyDescription: string | null; +} + +/** + * Free-text component descriptor (wall type / roof type / heating system): + * override → EPC-derived (lodged over predicted, from epc_energy_element) → + * legacy property_details_epc column → 'Unknown'. No code→label mapping — the + * value is the description string itself. Mirrors resolvedWallTypeSql etc. + */ +export function resolveOverrideDescriptor(i: OverrideDescriptorInputs): string { + if (i.override) return i.override; + const derived = i.isNewApproach + ? i.lodgedDescription ?? i.predictedDescription + : i.legacyDescription; + return derived ?? "Unknown"; +} + export interface MainFuelInputs { /** property_overrides.override_value for component 'main_fuel', building_part 0. */ override: string | null; diff --git a/src/lib/services/epcSources.ts b/src/lib/services/epcSources.ts index e4743d70..8d40c226 100644 --- a/src/lib/services/epcSources.ts +++ b/src/lib/services/epcSources.ts @@ -336,6 +336,55 @@ export const mainFuelOverrideJoin = sql` export const resolvedMainFuelSql = (e: Alias) => sql`COALESCE(pmf.override_value, ${mainfuelSql(e)}, 'Unknown')`; +// ─── Free-text component descriptors: wall / roof / heating (ADR-0012) ──────── +// override → EPC-derived (epc_energy_element description, lodged over predicted) +// → legacy property_details_epc column → 'Unknown'. Mirrors resolveOverrideDescriptor. + +/** One EPC source's description for an energy element, as a correlated subquery. */ +const epcElementDescSql = (ep: Alias, elementType: string) => sql`( + SELECT ee.description FROM epc_energy_element ee + WHERE ee.epc_property_id = ${ep}.id AND ee.element_type = ${elementType} + LIMIT 1 +)`; + +/** EPC-or-legacy element description: new = lodged over predicted; legacy = the property_details_epc column. */ +const epcElementSql = (elementType: string, legacyCol: Alias) => sql`CASE + WHEN ${isNewApproachSql} THEN (CASE WHEN epl.id IS NOT NULL THEN ${epcElementDescSql(sql`epl`, elementType)} ELSE ${epcElementDescSql(sql`epp`, elementType)} END) + ELSE ${legacyCol} +END`; + +/** LEFT JOIN exposing the wall_type override snapshot at the main building part (alias `pwall`). */ +export const wallTypeOverrideJoin = sql` + LEFT JOIN property_overrides pwall + ON pwall.property_id = p.id + AND pwall.override_component = 'wall_type' + AND pwall.building_part = 0 +`; +/** LEFT JOIN exposing the roof_type override snapshot at the main building part (alias `proof`). */ +export const roofTypeOverrideJoin = sql` + LEFT JOIN property_overrides proof + ON proof.property_id = p.id + AND proof.override_component = 'roof_type' + AND proof.building_part = 0 +`; +/** LEFT JOIN exposing the main_heating_system override snapshot (alias `pheat`). */ +export const mainHeatingOverrideJoin = sql` + LEFT JOIN property_overrides pheat + ON pheat.property_id = p.id + AND pheat.override_component = 'main_heating_system' + AND pheat.building_part = 0 +`; + +/** wall type: override → EPC 'wall' description → legacy walls → 'Unknown'. Mirrors resolveOverrideDescriptor. */ +export const resolvedWallTypeSql = (e: Alias) => + sql`COALESCE(pwall.override_value, ${epcElementSql("wall", sql`${e}.walls`)}, 'Unknown')`; +/** roof type: override → EPC 'roof' description → legacy roof → 'Unknown'. */ +export const resolvedRoofTypeSql = (e: Alias) => + sql`COALESCE(proof.override_value, ${epcElementSql("roof", sql`${e}.roof`)}, 'Unknown')`; +/** heating system: override → EPC 'main_heating' description → legacy heating → 'Unknown'. */ +export const resolvedHeatingSystemSql = (e: Alias) => + sql`COALESCE(pheat.override_value, ${epcElementSql("main_heating", sql`${e}.heating`)}, 'Unknown')`; + /** * The main building part's construction age band for one epc_property alias, as * a correlated scalar subquery — properties can also have extension parts, so a From 0131586a686453f03ac4623fd19ab0645221270b Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 13 Jul 2026 13:37:11 +0000 Subject: [PATCH 09/11] docs(context): note Wall/Roof/Heating join the override-resolved list columns Co-Authored-By: Claude Opus 4.8 (1M context) --- CONTEXT.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTEXT.md b/CONTEXT.md index da92a07b..d4d8b670 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -192,7 +192,7 @@ _Avoid_: estimation notification, banner (those are component names) ## Flagged ambiguities -- The portfolio **Property table**'s **Property Type**, **Built Form**, **Construction Age** and **Main Fuel** columns and filters show the **landlord-override-resolved** value — the same precedence the modelling **Run filter** applies (override fact → EPC-derived → **Unknown**) — not the raw `epc_property` value or the legacy property-row column. See [ADR-0012](./docs/adr/0012-portfolio-list-resolves-descriptors-via-override-precedence.md). Two label subtleties follow from this: **Construction Age** is an RdSAP band (e.g. "1900–1929", "before 1900"), never a single year (it was formerly the "Year Built" column); and **Main Fuel** is override-only for new-approach properties, so it reads **Unknown** unless a landlord override supplies it. +- The portfolio **Property table**'s **Property Type**, **Built Form**, **Construction Age**, **Main Fuel**, **Wall Type**, **Roof Type** and **Heating System** columns show the **landlord-override-resolved** value — the same precedence the modelling **Run filter** applies (override fact → EPC-derived → **Unknown**) — not the raw `epc_property` value or the legacy property-row column. See [ADR-0012](./docs/adr/0012-portfolio-list-resolves-descriptors-via-override-precedence.md). The first four are also **filters**; Wall/Roof/Heating are display-only (their override vocabularies are too large for a chip filter) and read the **main building part** for the per-part ones (wall/roof). Two label subtleties: **Construction Age** is an RdSAP band (e.g. "1900–1929", "before 1900"), never a single year (it was formerly the "Year Built" column); and **Main Fuel** is override-only for new-approach properties, so it reads **Unknown** unless a landlord override supplies it. - The UI surfaces labelled **"Current EPC"** and **"Current Efficiency State"** (property table, building-passport card) show **Effective performance**, not Lodged — despite the glossary advising against "current performance" for Effective. The label is a product choice ("current" = the property's true present-day modelled state); the underlying datum is still **Effective performance**. The **"Lodged EPC"** column/badge is the only surface showing **Lodged performance**, and only when a real certificate exists (`source = lodged`). - "Upload" is used in the codebase to mean both the file-on-S3 and the BulkUpload row. We standardise on **BulkUpload** for the row; the file is just "the source file." - "Onboarding" appears in some route paths (`bulk_onboarding_inputs/...`) but isn't part of this glossary — we use **BulkUpload** end-to-end. From 3b04de622bd385008fe9471b61781d74fe0fbe9d Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 13 Jul 2026 14:30:50 +0000 Subject: [PATCH 10/11] feat(portfolio): Main Fuel EPC fallback + filterable Wall/Roof/Heating MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Main Fuel no longer reads "Unknown" for new-approach properties. The new EPC graph stores the main fuel as a JSONB scalar on epc_main_heating_detail.main_fuel_type — a numeric RdSAP main_fuel code (~95%) or an EPR text string (~5%). mainfuelSql now reads it (lodged over predicted) instead of NULL; resolvedMainFuelSql keeps the override → EPC → 'Unknown' precedence. The RdSAP code table (Model epc_codes.csv, cross-checked against prod) + the EPR strings fold into MAINFUEL_OPTIONS, so the column label and the filter work unchanged. resolveMainFuel gains the EPC branch as its pure twin; coverage tests assert every code + string + override value is folded. Verified: portfolio 824 now resolves code 26 → "Mains Gas". Wall Type / Roof Type / Heating System become filterable. Their resolved values are free-text (override vocab / epc_energy_element.description / legacy column) with high cardinality, so exact dbValues can't scale — each coarse bucket is a set of case-insensitive needles matched by prefix (wall/roof) or substring (heating). classifyDescriptor is the pure twin of the SQL bucketing in buildConditionSql; the "Unknown" bucket matches the complement of every known needle. Buckets validated against the full prod vocabulary (heating Unknown = 28/640k rows). Wired through FilterField, FIELD_OPTIONS, buildConditionSql, EPC_JOIN_FILTER_FIELDS and the count-query override joins; columns stay free-text display. Coverage tests assert every Wall/Roof/MainHeatingSystem override value buckets to a real, non-Unknown label. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../[slug]/components/PropertyFilters.tsx | 27 ++- src/app/portfolio/[slug]/utils.ts | 87 +++++++- src/app/utils/propertyFilters.ts | 193 +++++++++++++++++- src/lib/services/descriptorResolution.test.ts | 163 +++++++++++++-- src/lib/services/descriptorResolution.ts | 16 +- src/lib/services/epcSources.ts | 30 ++- 6 files changed, 474 insertions(+), 42 deletions(-) diff --git a/src/app/portfolio/[slug]/components/PropertyFilters.tsx b/src/app/portfolio/[slug]/components/PropertyFilters.tsx index 9bdf8b8e..a77295b8 100644 --- a/src/app/portfolio/[slug]/components/PropertyFilters.tsx +++ b/src/app/portfolio/[slug]/components/PropertyFilters.tsx @@ -10,12 +10,14 @@ import { FilterField, FilterOperator, DatePreset, - EnumOption, PROPERTY_TYPE_OPTIONS, BUILT_FORM_OPTIONS, YEAR_BUILT_OPTIONS, MAINFUEL_OPTIONS, PROVENANCE_OPTIONS, + WALL_TYPE_OPTIONS, + ROOF_TYPE_OPTIONS, + HEATING_SYSTEM_OPTIONS, } from "@/app/utils/propertyFilters"; /* ----------------------------------------------------------------------- @@ -36,6 +38,9 @@ const FIELD_OPTIONS: { value: FilterField; label: string }[] = [ { value: "floorArea", label: "Floor Area (m²)" }, { value: "co2Emissions", label: "CO₂ Emissions (kg/m²/yr)" }, { value: "mainfuel", label: "Main Fuel" }, + { value: "wallType", label: "Wall Type" }, + { value: "roofType", label: "Roof Type" }, + { value: "heatingSystem", label: "Heating System" }, ]; const EPC_OPERATOR_OPTIONS: { value: FilterOperator; label: string }[] = [ @@ -69,12 +74,18 @@ const NUM_OPERATOR_OPTIONS: { value: FilterOperator; label: string }[] = [ { value: "num_equals", label: "= (equals)" }, ]; -const ENUM_FIELD_OPTIONS: Record = { - propertyType: PROPERTY_TYPE_OPTIONS, - builtForm: BUILT_FORM_OPTIONS, - yearBuilt: YEAR_BUILT_OPTIONS, - provenance: PROVENANCE_OPTIONS, - mainfuel: MAINFUEL_OPTIONS, +// Options need only a `label` for the dropdown: the exact-enum fields carry +// EnumOption[] (dbValues used server-side), the free-text descriptor fields carry +// DescriptorBucket[] (needles used server-side). Both satisfy `{ label }`. +const ENUM_FIELD_OPTIONS: Record = { + propertyType: PROPERTY_TYPE_OPTIONS, + builtForm: BUILT_FORM_OPTIONS, + yearBuilt: YEAR_BUILT_OPTIONS, + provenance: PROVENANCE_OPTIONS, + mainfuel: MAINFUEL_OPTIONS, + wallType: WALL_TYPE_OPTIONS, + roofType: ROOF_TYPE_OPTIONS, + heatingSystem: HEATING_SYSTEM_OPTIONS, }; /* ----------------------------------------------------------------------- @@ -278,7 +289,7 @@ function EnumMultiDropdown({ selectedLabels, onChange, }: { - options: EnumOption[]; + options: { label: string }[]; selectedLabels: string[]; onChange: (labels: string[]) => void; }) { diff --git a/src/app/portfolio/[slug]/utils.ts b/src/app/portfolio/[slug]/utils.ts index 2622f1c6..c789b6b3 100644 --- a/src/app/portfolio/[slug]/utils.ts +++ b/src/app/portfolio/[slug]/utils.ts @@ -55,6 +55,9 @@ import { MAINFUEL_OPTIONS, PROVENANCE_OPTIONS, EnumOption, + DESCRIPTOR_FILTER_CONFIG, + DescriptorBucket, + DescriptorMatch, } from "@/app/utils/propertyFilters"; import { EPC_TO_SAP_MIN, EPC_TO_SAP_MAX } from "@/app/utils/epc"; @@ -519,6 +522,50 @@ function buildEpcSapCondition( return null; } +/** An ILIKE pattern for a needle: prefix ("cavity%") or substring ("%mains gas%"). */ +function descriptorLikePattern(needle: string, match: DescriptorMatch): string { + return match === "prefix" ? `${needle}%` : `%${needle}%`; +} + +/** + * WHERE fragment matching the free-text wall/roof/heating column against the + * selected coarse buckets (mirrors classifyDescriptor). A non-empty bucket + * matches any of its needles; the "Unknown" bucket matches the complement — + * rows matching NO known needle (which includes the resolved 'Unknown' + * terminal). NULL is impossible here (resolvedXSql COALESCEs to 'Unknown'). + */ +function buildDescriptorBucketCondition( + col: ReturnType, + buckets: DescriptorBucket[], + match: DescriptorMatch, + selectedLabels: string[], +): ReturnType | null { + const allNeedles = buckets.flatMap((b) => b.needles); + const parts: ReturnType[] = []; + + for (const label of selectedLabels) { + const bucket = buckets.find((b) => b.label === label); + if (!bucket) continue; + if (bucket.needles.length === 0) { + // "Unknown" = the complement of every known needle. + if (allNeedles.length === 0) return null; + const nots = allNeedles.map( + (n) => sql`${col} NOT ILIKE ${descriptorLikePattern(n, match)}`, + ); + parts.push(sql`(${sql.join(nots, sql` AND `)})`); + } else { + const likes = bucket.needles.map( + (n) => sql`${col} ILIKE ${descriptorLikePattern(n, match)}`, + ); + parts.push(sql`(${sql.join(likes, sql` OR `)})`); + } + } + + if (parts.length === 0) return null; + if (parts.length === 1) return parts[0]; + return sql`(${sql.join(parts, sql` OR `)})`; +} + function buildConditionSql(filter: PropertyFilter): ReturnType | null { switch (filter.field) { case "address": @@ -667,6 +714,36 @@ function buildConditionSql(filter: PropertyFilter): ReturnType | nul if (filter.operator === "num_equals") return sql`${carbon} = ${n}`; return null; } + + case "wallType": + case "roofType": + case "heatingSystem": { + // Free-text descriptors: bucket the resolved value by prefix/substring + // needles (ADR-0012), mirroring classifyDescriptor. The resolved SQL folds + // override → EPC element description → legacy → 'Unknown'. + if (filter.operator !== "enum_one_of") return null; + + let selectedLabels: string[]; + try { + selectedLabels = JSON.parse(filter.value); + } catch { + return null; + } + if (selectedLabels.length === 0) return null; + + const { buckets, match } = DESCRIPTOR_FILTER_CONFIG[filter.field]; + const colMap: Record> = { + wallType: resolvedWallTypeSql(sql`epc`), + roofType: resolvedRoofTypeSql(sql`epc`), + heatingSystem: resolvedHeatingSystemSql(sql`epc`), + }; + return buildDescriptorBucketCondition( + colMap[filter.field], + buckets, + match, + selectedLabels, + ); + } } return null; } @@ -710,6 +787,11 @@ const EPC_JOIN_FILTER_FIELDS = new Set([ "propertyType", "builtForm", "yearBuilt", + // Free-text descriptors (ADR-0012): resolved SQL reads the EPC element + // description subquery + the override join. + "wallType", + "roofType", + "heatingSystem", ]); const PLAN_JOIN_FILTER_FIELDS = new Set(["expectedEpc"]); @@ -739,7 +821,10 @@ export async function getPropertiesCount( ${propertyTypeOverrideJoin} ${builtFormOverrideJoin} ${constructionAgeOverrideJoin} - ${mainFuelOverrideJoin}` + ${mainFuelOverrideJoin} + ${wallTypeOverrideJoin} + ${roofTypeOverrideJoin} + ${mainHeatingOverrideJoin}` : sql``; const planJoin = needsPlanJoin ? sql`LEFT JOIN LATERAL ( diff --git a/src/app/utils/propertyFilters.ts b/src/app/utils/propertyFilters.ts index 0c906a85..f07f135f 100644 --- a/src/app/utils/propertyFilters.ts +++ b/src/app/utils/propertyFilters.ts @@ -12,7 +12,10 @@ export type FilterField = | "provenance" | "floorArea" | "co2Emissions" - | "mainfuel"; + | "mainfuel" + | "wallType" + | "roofType" + | "heatingSystem"; export type FilterOperator = | "contains" @@ -109,12 +112,72 @@ export const YEAR_BUILT_OPTIONS: EnumOption[] = [ "2007–2011", "2012–2022", "2023 onwards", "Unknown", ].map((label) => ({ label, dbValues: [label] })); -// dbValues fold two vocabularies onto one user-facing label: the raw EPC -// mainfuel strings (legacy) AND the landlord main_fuel override values -// ("mains gas", "LPG (bulk)", … — see MainFuelValues). resolvedMainFuelSql emits -// whichever applies; both must resolve to the same chip (ADR-0012). A -// descriptorResolution.test asserts every override value is covered here. -export const MAINFUEL_OPTIONS: EnumOption[] = [ +// RdSAP `main_fuel` code → coarse label. The new EPC graph stores the main fuel +// as this numeric code on epc_main_heating_detail.main_fuel_type (see mainfuelSql +// / resolveMainFuel), so every code that can appear must fold into a filter +// option or an overridden/EPC-derived fuel would display but be unfilterable +// (ADR-0012). Codes are the authoritative Model `epc_codes.csv` `main_fuel` table +// (RdSAP), cross-checked against production epc rows: 26→mains gas, 29/39→electric, +// 0/20/25/31/57→"Community scheme", 5/15/33→coal, 6→wood logs, 9→dual fuel. All +// "(community)"/heat-network codes bucket to "Community Heat Network" — being on a +// communal scheme is the salient fact; the scheme's own fuel is a secondary detail. +export const RDSAP_MAIN_FUEL_CODE_LABELS: Record = { + // Mains gas + "1": "Mains Gas", "26": "Mains Gas", + // LPG + "2": "LPG", "3": "LPG", "17": "LPG", "27": "LPG", "38": "LPG", + // Oil + "4": "Oil", "28": "Oil", + // Electricity + "10": "Electricity", "29": "Electricity", "39": "Electricity", + // Coal / mineral solid fuel (anthracite, smokeless, house coal) + "5": "Coal", "14": "Coal", "15": "Coal", "33": "Coal", + // Biomass / wood (logs, pellets, chips) + "6": "Biomass", "7": "Biomass", "8": "Biomass", "12": "Biomass", "16": "Biomass", + // Dual fuel + "9": "Dual Fuel (Mineral + Wood)", + // Biogas + "13": "Biogas", "51": "Biogas", + // Bioethanol + "19": "Bioethanol", "76": "Bioethanol", + // Biodiesel (from biomass / used cooking oil / vegetable / rape seed oil) + "34": "Biodiesel", "35": "Biodiesel", "36": "Biodiesel", "37": "Biodiesel", + "71": "Biodiesel", "72": "Biodiesel", "73": "Biodiesel", "74": "Biodiesel", + // Biodiesel blend + "18": "B30k (Biodiesel blend)", "75": "B30k (Biodiesel blend)", + // Waste combustion + "11": "Waste Combustion", + // Community / heat-network (the dwelling is on a communal scheme) + "0": "Community Heat Network", "20": "Community Heat Network", "21": "Community Heat Network", + "22": "Community Heat Network", "23": "Community Heat Network", "24": "Community Heat Network", + "25": "Community Heat Network", "30": "Community Heat Network", "31": "Community Heat Network", + "32": "Community Heat Network", "41": "Community Heat Network", "42": "Community Heat Network", + "43": "Community Heat Network", "44": "Community Heat Network", "45": "Community Heat Network", + "46": "Community Heat Network", "47": "Community Heat Network", "48": "Community Heat Network", + "49": "Community Heat Network", "50": "Community Heat Network", "52": "Community Heat Network", + "53": "Community Heat Network", "54": "Community Heat Network", "55": "Community Heat Network", + "56": "Community Heat Network", "57": "Community Heat Network", "58": "Community Heat Network", + "99": "Community Heat Network", +}; + +// EPR free-text main_fuel_type strings (Elmhurst/PASHub, ~5% of new-approach +// rows) folded to the same labels. The empty-string sentinel is NULLIF'd to NULL +// upstream and resolves to 'Unknown'. +export const EPR_MAIN_FUEL_STRING_LABELS: Record = { + "Mains gas": "Mains Gas", + "Electricity": "Electricity", + "Bulk LPG": "LPG", + "Heating oil": "Oil", +}; + +// dbValues fold several vocabularies onto one user-facing label: the raw legacy +// EPC mainfuel strings (property_details_epc), the landlord main_fuel override +// values ("mains gas", "LPG (bulk)", … — see MainFuelValues), the new-approach +// RdSAP numeric codes (RDSAP_MAIN_FUEL_CODE_LABELS) and the EPR strings +// (EPR_MAIN_FUEL_STRING_LABELS). resolvedMainFuelSql emits whichever applies; all +// must resolve to the same chip (ADR-0012). descriptorResolution.test asserts +// every override value, RdSAP code and EPR string is covered here. +const MAINFUEL_OPTIONS_BASE: EnumOption[] = [ { label: "Mains Gas", dbValues: ["Gas mains gas", "Mains gas community", "Mains gas not community", "Mains gas this is for backwards compatibility only and should not be used", "mains gas", "mains gas (community)"] }, { label: "LPG", dbValues: ["Bottled lpg", "Lpg community", "Lpg not community", "Lpg this is for backwards compatibility only and should not be used", "LPG (bulk)", "bottled LPG", "LPG special condition"] }, { label: "Oil", dbValues: ["Oil heating oil", "Oil community", "Oil not community", "Oil this is for backwards compatibility only and should not be used", "oil"] }, @@ -123,10 +186,126 @@ export const MAINFUEL_OPTIONS: EnumOption[] = [ { label: "Coal", dbValues: ["Anthracite", "House coal not community", "House coal this is for backwards compatibility only and should not be used", "Smokeless coal", "Coal community", "house coal", "smokeless coal"] }, { label: "Dual Fuel (Mineral + Wood)", dbValues: ["Dual fuel mineral wood", "dual fuel (mineral and wood)"] }, { label: "Biogas", dbValues: ["Biogas not community"] }, + { label: "Bioethanol", dbValues: [] }, { label: "Biodiesel", dbValues: ["Heat from boilers using biodiesel from any biomass source community"] }, { label: "B30d (Biodiesel blend)", dbValues: ["B30d community"] }, { label: "B30k (Biodiesel blend)", dbValues: ["B30k not community"] }, + { label: "Waste Combustion", dbValues: [] }, { label: "Community Heat Network", dbValues: ["From heat network data community"] }, { label: "No Heating System", dbValues: ["To be used only when there is no heatinghotwater system", "To be used only when there is no heatinghotwater system or data is from a community network"] }, { label: "Unknown / No Data", dbValues: ["UNKNOWN", "NO DATA!", "Unknown"] }, ]; + +/** Fold the RdSAP code + EPR string vocabularies into each label's dbValues. */ +export const MAINFUEL_OPTIONS: EnumOption[] = MAINFUEL_OPTIONS_BASE.map((opt) => { + const codes = Object.entries(RDSAP_MAIN_FUEL_CODE_LABELS) + .filter(([, label]) => label === opt.label) + .map(([code]) => code); + const strings = Object.entries(EPR_MAIN_FUEL_STRING_LABELS) + .filter(([, label]) => label === opt.label) + .map(([str]) => str); + return { ...opt, dbValues: [...opt.dbValues, ...codes, ...strings] }; +}); + +/* ----------------------------------------------------------------------- + Free-text descriptor buckets: Wall Type / Roof Type / Heating System (ADR-0012) + + Unlike the enum fields above (exact dbValues), the resolved wall/roof/heating + value is a free-text description (override vocabulary, EPC + epc_energy_element.description, or legacy property_details_epc column) with + HIGH cardinality — 88+ distinct EPC wall descriptions, 245 legacy, plus + insulation-thickness variants, "Average thermal transmittance …" U-values and + encoding corruption. Exact matching can't scale, so each coarse bucket is a set + of case-insensitive NEEDLES matched by prefix (wall/roof) or substring + (heating, whose fuel follows a comma). classifyDescriptor is the pure twin of + the SQL bucketing in the portfolio query's buildConditionSql — keep in sync. + Ordering matters: the first bucket whose needle matches wins. +------------------------------------------------------------------------ */ + +export interface DescriptorBucket { + label: string; + /** Lowercase needles, matched per the field's DescriptorMatch mode. Empty = the Unknown terminal (matches the complement). */ + needles: string[]; +} + +export type DescriptorMatch = "prefix" | "contains"; + +// Wall construction. Prefix-matched on the leading construction term. "Stone" +// folds sandstone/limestone + granite/whinstone; "Average thermal transmittance …" +// (a U-value, not a construction type) and corruption fall through to Unknown. +export const WALL_TYPE_OPTIONS: DescriptorBucket[] = [ + { label: "Cavity", needles: ["cavity"] }, + { label: "Solid Brick", needles: ["solid brick"] }, + { label: "Timber Frame", needles: ["timber frame"] }, + { label: "Stone", needles: ["sandstone", "granite"] }, + { label: "System Built", needles: ["system built"] }, + { label: "Park Home", needles: ["park home"] }, + { label: "Cob", needles: ["cob"] }, + { label: "Curtain Wall", needles: ["curtain"] }, + { label: "Basement", needles: ["basement"] }, + { label: "Unknown", needles: [] }, +]; + +// Roof form. Prefix-matched. "Premises Above" folds the "(another dwelling +// above)" / "(other premises above)" family (a flat with a dwelling above it, so +// no own roof). U-values fall through to Unknown. +export const ROOF_TYPE_OPTIONS: DescriptorBucket[] = [ + { label: "Pitched", needles: ["pitched"] }, + { label: "Flat", needles: ["flat"] }, + { label: "Room-in-Roof", needles: ["roof room"] }, + { label: "Thatched", needles: ["thatched"] }, + { label: "Premises Above", needles: ["(another", "(other", "(same", "another dwelling", "another premises", "other premises"] }, + { label: "Unknown", needles: [] }, +]; + +// Heating system. Substring-matched (the fuel follows a comma, e.g. "Boiler and +// radiators, mains gas"). Order is load-bearing: Heat Pump / Community / Solid +// Fuel / Oil / LPG are checked before the gas + electric buckets so a +// heat-pump/community/wood/oil/LPG system isn't miscaught by "gas"/"electric"; +// Gas Room Heater before Gas Boiler; Electric Storage before Direct Electric. +export const HEATING_SYSTEM_OPTIONS: DescriptorBucket[] = [ + { label: "Heat Pump", needles: ["heat pump"] }, + { label: "Community", needles: ["community"] }, + { label: "Solid Fuel", needles: ["coal", "anthracite", "smokeless", "wood", "dual fuel", "biomass", "solid fuel"] }, + // NB: bare "oil" is a substring of "b(oil)er" — match the fuel token only (", oil" + // as in "Boiler and radiators, oil") and the leading override "Oil room heater". + { label: "Oil", needles: [", oil", "oil room heater"] }, + { label: "LPG", needles: ["lpg"] }, + { label: "Gas Room Heater", needles: ["gas room heater", "room heaters, mains gas", "room heaters, gas"] }, + { label: "Gas Boiler", needles: ["mains gas", "gas boiler", "gas cpsu"] }, + { label: "Electric Storage", needles: ["storage heater"] }, + { label: "Direct Electric", needles: ["electric"] }, + { label: "Unknown", needles: [] }, +]; + +/** Filter config per descriptor field: its buckets + how needles are matched. */ +export const DESCRIPTOR_FILTER_CONFIG: Record< + "wallType" | "roofType" | "heatingSystem", + { buckets: DescriptorBucket[]; match: DescriptorMatch } +> = { + wallType: { buckets: WALL_TYPE_OPTIONS, match: "prefix" }, + roofType: { buckets: ROOF_TYPE_OPTIONS, match: "prefix" }, + heatingSystem: { buckets: HEATING_SYSTEM_OPTIONS, match: "contains" }, +}; + +/** + * Classify a resolved free-text descriptor into its coarse filter bucket: the + * first bucket with a matching needle, else "Unknown". Pure twin of the SQL + * bucketing in buildConditionSql (portfolio query) — keep in sync. + */ +export function classifyDescriptor( + value: string | null | undefined, + buckets: DescriptorBucket[], + match: DescriptorMatch, +): string { + if (value == null) return "Unknown"; + const v = value.toLowerCase(); + for (const bucket of buckets) { + for (const needle of bucket.needles) { + if (match === "prefix" ? v.startsWith(needle) : v.includes(needle)) { + return bucket.label; + } + } + } + return "Unknown"; +} diff --git a/src/lib/services/descriptorResolution.test.ts b/src/lib/services/descriptorResolution.test.ts index 533d3935..1c487586 100644 --- a/src/lib/services/descriptorResolution.test.ts +++ b/src/lib/services/descriptorResolution.test.ts @@ -6,8 +6,22 @@ import { resolveOverrideDescriptor, resolvePropertyType, } from "./descriptorResolution"; -import { MAINFUEL_OPTIONS } from "@/app/utils/propertyFilters"; -import { MainFuelValues } from "@/app/db/schema/landlord_overrides"; +import { + MAINFUEL_OPTIONS, + RDSAP_MAIN_FUEL_CODE_LABELS, + EPR_MAIN_FUEL_STRING_LABELS, + WALL_TYPE_OPTIONS, + ROOF_TYPE_OPTIONS, + HEATING_SYSTEM_OPTIONS, + DESCRIPTOR_FILTER_CONFIG, + classifyDescriptor, +} from "@/app/utils/propertyFilters"; +import { + MainFuelValues, + WallTypeValues, + RoofTypeValues, + MainHeatingSystemValues, +} from "@/app/db/schema/landlord_overrides"; describe("resolvePropertyType", () => { it("prefers a landlord override over the EPC-derived value", () => { @@ -284,52 +298,89 @@ describe("resolveConstructionAge", () => { }); describe("resolveMainFuel", () => { - it("prefers a landlord override (the only main-fuel source for new-approach)", () => { + const base = { + override: null, + isNewApproach: true, + lodgedMainFuel: null, + predictedMainFuel: null, + legacyMainFuel: null, + }; + + it("prefers a landlord override over the EPC-derived value", () => { expect( - resolveMainFuel({ - override: "mains gas", - isNewApproach: true, - legacyMainFuel: null, - }), + resolveMainFuel({ ...base, override: "mains gas", lodgedMainFuel: "26" }), ).toBe("mains gas"); }); - it("has no EPC main fuel for new-approach, so resolves to 'Unknown' without an override", () => { + it("reads the new-approach EPC main-fuel code (lodged) as a raw value", () => { + expect(resolveMainFuel({ ...base, lodgedMainFuel: "26" })).toBe("26"); + }); + + it("falls back to the predicted EPC main fuel when there is no lodged one", () => { + expect(resolveMainFuel({ ...base, predictedMainFuel: "29" })).toBe("29"); + }); + + it("prefers the lodged EPC main fuel over the predicted one", () => { expect( - resolveMainFuel({ - override: null, - isNewApproach: true, - legacyMainFuel: "Gas mains gas", // legacy value must NOT surface for new-approach - }), + resolveMainFuel({ ...base, lodgedMainFuel: "26", predictedMainFuel: "29" }), + ).toBe("26"); + }); + + it("passes an EPR string main fuel through unchanged", () => { + expect(resolveMainFuel({ ...base, lodgedMainFuel: "Mains gas" })).toBe("Mains gas"); + }); + + it("resolves to 'Unknown' for a new-approach property with no EPC main fuel; ignores the legacy row", () => { + expect( + resolveMainFuel({ ...base, legacyMainFuel: "Gas mains gas" }), ).toBe("Unknown"); }); it("reads the legacy EPC main fuel for a legacy property", () => { expect( resolveMainFuel({ - override: null, + ...base, isNewApproach: false, + lodgedMainFuel: "26", // must be ignored for legacy legacyMainFuel: "Gas mains gas", }), ).toBe("Gas mains gas"); }); it("resolves to 'Unknown' when nothing is available (never null)", () => { - expect( - resolveMainFuel({ override: null, isNewApproach: false, legacyMainFuel: null }), - ).toBe("Unknown"); + expect(resolveMainFuel({ ...base, isNewApproach: false })).toBe("Unknown"); }); }); describe("MAINFUEL_OPTIONS coverage", () => { + const covered = new Set(MAINFUEL_OPTIONS.flatMap((o) => o.dbValues)); + const labels = new Set(MAINFUEL_OPTIONS.map((o) => o.label)); + // Every landlord main-fuel override value must fold into a filter option, or an // overridden fuel would display but be unfilterable (ADR-0012). it("covers every landlord main-fuel override value", () => { - const covered = new Set(MAINFUEL_OPTIONS.flatMap((o) => o.dbValues)); for (const value of MainFuelValues) { expect(covered).toContain(value); } }); + + // Every RdSAP main_fuel code the new EPC graph can store must fold into a + // filter option under a real label (a typo'd label would be silently dropped + // by the MAINFUEL_OPTIONS merge — assert both the code AND its label). + it("folds every RdSAP main_fuel code under a real label", () => { + for (const [code, label] of Object.entries(RDSAP_MAIN_FUEL_CODE_LABELS)) { + expect(labels).toContain(label); + expect(covered).toContain(code); + } + }); + + // Every EPR free-text main_fuel string must fold in too. + it("folds every EPR main-fuel string under a real label", () => { + for (const [str, label] of Object.entries(EPR_MAIN_FUEL_STRING_LABELS)) { + expect(labels).toContain(label); + expect(covered).toContain(str); + } + }); }); describe("resolveOverrideDescriptor (wall / roof / heating free-text)", () => { @@ -384,6 +435,80 @@ describe("resolveOverrideDescriptor (wall / roof / heating free-text)", () => { }); }); +describe("classifyDescriptor — wall / roof / heating buckets", () => { + const wall = (v: string) => + classifyDescriptor(v, WALL_TYPE_OPTIONS, DESCRIPTOR_FILTER_CONFIG.wallType.match); + const roof = (v: string) => + classifyDescriptor(v, ROOF_TYPE_OPTIONS, DESCRIPTOR_FILTER_CONFIG.roofType.match); + const heat = (v: string) => + classifyDescriptor(v, HEATING_SYSTEM_OPTIONS, DESCRIPTOR_FILTER_CONFIG.heatingSystem.match); + + // Every landlord override value must fall into a real (non-Unknown) bucket, or + // an overridden component would display but be unfilterable (ADR-0012). The + // literal "Unknown" is the sole exception — it maps to the Unknown bucket. + const assertVocabCovered = ( + values: readonly string[], + classify: (v: string) => string, + labels: Set, + ) => { + for (const value of values) { + const bucket = classify(value); + expect(labels).toContain(bucket); + if (value !== "Unknown") expect(bucket).not.toBe("Unknown"); + } + }; + + it("buckets every WallType override value", () => { + assertVocabCovered(WallTypeValues, wall, new Set(WALL_TYPE_OPTIONS.map((b) => b.label))); + }); + it("buckets every RoofType override value", () => { + assertVocabCovered(RoofTypeValues, roof, new Set(ROOF_TYPE_OPTIONS.map((b) => b.label))); + }); + it("buckets every MainHeatingSystem override value", () => { + assertVocabCovered( + MainHeatingSystemValues, + heat, + new Set(HEATING_SYSTEM_OPTIONS.map((b) => b.label)), + ); + }); + + // Representative EPC / legacy descriptions (from production) bucket as expected. + it("buckets representative EPC/legacy WALL descriptions", () => { + expect(wall("Cavity wall, filled cavity")).toBe("Cavity"); + expect(wall("Solid brick, as built, no insulation (assumed)")).toBe("Solid Brick"); + expect(wall("Timber frame, as built, insulated (assumed)")).toBe("Timber Frame"); + expect(wall("Sandstone or limestone, as built, no insulation (assumed)")).toBe("Stone"); + expect(wall("Granite or whinstone, as built, no insulation (assumed)")).toBe("Stone"); + expect(wall("System built, with external insulation")).toBe("System Built"); + // A bare U-value is not a construction type → Unknown. + expect(wall("Average thermal transmittance 0.21 W/m²K")).toBe("Unknown"); + }); + + it("buckets representative EPC/legacy ROOF descriptions", () => { + expect(roof("Pitched, 200 mm loft insulation")).toBe("Pitched"); + expect(roof("Flat, no insulation (assumed)")).toBe("Flat"); + expect(roof("Roof room(s), insulated (assumed)")).toBe("Room-in-Roof"); + expect(roof("(another dwelling above)")).toBe("Premises Above"); + expect(roof("(other premises above)")).toBe("Premises Above"); + expect(roof("Another Premises Above")).toBe("Premises Above"); + expect(roof("Average thermal transmittance 0.11 W/m²K")).toBe("Unknown"); + }); + + it("buckets representative EPC/legacy HEATING descriptions", () => { + expect(heat("Boiler and radiators, mains gas")).toBe("Gas Boiler"); + expect(heat("Room heaters, mains gas")).toBe("Gas Room Heater"); + expect(heat("Community scheme")).toBe("Community"); + expect(heat("Community scheme, biomass")).toBe("Community"); + expect(heat("Electric storage heaters")).toBe("Electric Storage"); + expect(heat("Room heaters, electric")).toBe("Direct Electric"); + expect(heat("Air source heat pump, radiators, electric")).toBe("Heat Pump"); + // "oil" is a substring of "boiler" — a gas boiler must NOT bucket as Oil. + expect(heat("Boiler and radiators, oil")).toBe("Oil"); + expect(heat("Boiler and radiators, dual fuel (mineral and wood)")).toBe("Solid Fuel"); + expect(heat("Boiler and radiators, lpg")).toBe("LPG"); + }); +}); + /** Legacy-property helper: only year_built is set, cutoff is false. */ function legacyAge(year: string): string { return resolveConstructionAge({ diff --git a/src/lib/services/descriptorResolution.ts b/src/lib/services/descriptorResolution.ts index 4ebb3242..da166b64 100644 --- a/src/lib/services/descriptorResolution.ts +++ b/src/lib/services/descriptorResolution.ts @@ -142,19 +142,25 @@ export interface MainFuelInputs { /** property_overrides.override_value for component 'main_fuel', building_part 0. */ override: string | null; isNewApproach: boolean; + /** epc_main_heating_detail.main_fuel_type scalar on the lodged EPC — an RdSAP code ("26") or an EPR string ("Mains gas"). */ + lodgedMainFuel: string | null; + predictedMainFuel: string | null; /** property_details_epc.mainfuel — the raw legacy EPC string. */ legacyMainFuel: string | null; } /** - * Main fuel raw value (folded to a label by the cell's resolveEnumLabel). The - * new EPC graph exposes no main-fuel string (documented gap in mainfuelSql), so - * for new-approach properties the landlord override is the only source; legacy - * reads the EPC string. Never null → 'Unknown'. Mirrors resolvedMainFuelSql. + * Main fuel raw value (folded to a label by the cell's resolveEnumLabel): + * override → EPC-derived (new: epc_main_heating_detail scalar, lodged over + * predicted; legacy: the property_details_epc string) → 'Unknown'. The + * new-approach scalar is an RdSAP `main_fuel` code or an EPR string; both fold + * into MAINFUEL_OPTIONS. Never null → 'Unknown'. Mirrors resolvedMainFuelSql. */ export function resolveMainFuel(i: MainFuelInputs): string { if (i.override) return i.override; - const derived = i.isNewApproach ? null : i.legacyMainFuel; + const derived = i.isNewApproach + ? i.lodgedMainFuel ?? i.predictedMainFuel + : i.legacyMainFuel; return derived ?? "Unknown"; } diff --git a/src/lib/services/epcSources.ts b/src/lib/services/epcSources.ts index 8d40c226..42174e97 100644 --- a/src/lib/services/epcSources.ts +++ b/src/lib/services/epcSources.ts @@ -192,9 +192,35 @@ export const estimatedSql = (e: Alias) => export const isExpiredSql = (e: Alias) => sql`CASE WHEN ${isNewApproachSql} THEN (epl.registration_date IS NOT NULL AND epl.registration_date < (CURRENT_DATE - INTERVAL '10 years')) ELSE ${e}.is_expired END`; -/** Main fuel. GAP for new properties (no clean equivalent) → NULL. Legacy: e.mainfuel. */ +/** + * The main-fuel scalar for one epc_property alias, read from the JSONB + * `epc_main_heating_detail.main_fuel_type` as a correlated subquery. The stored + * scalar is either a numeric RdSAP `main_fuel` code ("26", "29", … — ~95% of + * rows, from the gov-EPC API) or an EPR text string ("Mains gas", "Bulk LPG", … + * from Elmhurst/PASHub). `#>> '{}'` extracts the JSON scalar as text; NULLIF + * drops the empty-string sentinel to NULL. Codes/strings are folded to a label + * downstream by MAINFUEL_OPTIONS — the fragment emits the raw value, like + * resolvedMainFuelSql's legacy branch. (No index on epc_property_id yet — see + * ix_epc_main_heating_detail_epc_property in the schema.) + */ +const epcMainFuelSql = (ep: Alias) => sql`( + SELECT NULLIF(mhd.main_fuel_type #>> '{}', '') + FROM epc_main_heating_detail mhd + WHERE mhd.epc_property_id = ${ep}.id + LIMIT 1 +)`; + +/** + * Main fuel raw value. New: the epc_main_heating_detail scalar (lodged over + * predicted) — an RdSAP code or an EPR string, folded to a label by + * MAINFUEL_OPTIONS. Legacy: e.mainfuel. Mirrors the new-approach branch of + * resolveMainFuel. + */ export const mainfuelSql = (e: Alias) => - sql`CASE WHEN ${isNewApproachSql} THEN NULL ELSE ${e}.mainfuel END`; + sql`CASE + WHEN ${isNewApproachSql} THEN (CASE WHEN epl.id IS NOT NULL THEN ${epcMainFuelSql(sql`epl`)} ELSE ${epcMainFuelSql(sql`epp`)} END) + ELSE ${e}.mainfuel + END`; // ───────────────────────────────────────────────────────────────────────────── // RdSAP code → label maps for the descriptive fields the new pipeline stores as From 0873fbc5c121b52a407f788060e5edbb1e5d9118 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 13 Jul 2026 14:38:42 +0000 Subject: [PATCH 11/11] docs(context): Main Fuel reads EPC detail; Wall/Roof/Heating filter by bucket Updates the portfolio Property-table flagged-ambiguity note for the two changes in the previous commit: Main Fuel now reads epc_main_heating_detail beneath the override (no longer override-only for new-approach), and Wall/Roof/Heating are now filterable via coarse prefix/substring buckets while still displaying the full free-text description. Co-Authored-By: Claude Opus 4.8 (1M context) --- CONTEXT.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTEXT.md b/CONTEXT.md index d4d8b670..777ac779 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -192,7 +192,7 @@ _Avoid_: estimation notification, banner (those are component names) ## Flagged ambiguities -- The portfolio **Property table**'s **Property Type**, **Built Form**, **Construction Age**, **Main Fuel**, **Wall Type**, **Roof Type** and **Heating System** columns show the **landlord-override-resolved** value — the same precedence the modelling **Run filter** applies (override fact → EPC-derived → **Unknown**) — not the raw `epc_property` value or the legacy property-row column. See [ADR-0012](./docs/adr/0012-portfolio-list-resolves-descriptors-via-override-precedence.md). The first four are also **filters**; Wall/Roof/Heating are display-only (their override vocabularies are too large for a chip filter) and read the **main building part** for the per-part ones (wall/roof). Two label subtleties: **Construction Age** is an RdSAP band (e.g. "1900–1929", "before 1900"), never a single year (it was formerly the "Year Built" column); and **Main Fuel** is override-only for new-approach properties, so it reads **Unknown** unless a landlord override supplies it. +- The portfolio **Property table**'s **Property Type**, **Built Form**, **Construction Age**, **Main Fuel**, **Wall Type**, **Roof Type** and **Heating System** columns show the **landlord-override-resolved** value — the same precedence the modelling **Run filter** applies (override fact → EPC-derived → **Unknown**) — not the raw `epc_property` value or the legacy property-row column. See [ADR-0012](./docs/adr/0012-portfolio-list-resolves-descriptors-via-override-precedence.md). All seven are also **filters**, and the per-part ones (wall/roof) read the **main building part**. Wall/Roof/Heating filter by **coarse bucket** rather than the full vocabulary — their resolved values are free-text (override vocab / EPC `epc_energy_element.description` / legacy column) with hundreds of variants, so a fixed set of prefix/substring buckets (Wall → Cavity / Solid Brick / Timber Frame / Stone / …; Roof → Pitched / Flat / Room-in-Roof / …; Heating → Gas Boiler / Community / Electric Storage / Heat Pump / …) matches them, while the columns still **display the full description**. `classifyDescriptor` is the pure twin of the SQL bucketing. Two label subtleties: **Construction Age** is an RdSAP band (e.g. "1900–1929", "before 1900"), never a single year (it was formerly the "Year Built" column); and **Main Fuel** for new-approach properties reads the EPC's `epc_main_heating_detail.main_fuel_type` (an RdSAP `main_fuel` code or an EPR string, folded to a coarse label) beneath any landlord override — it only reads **Unknown** when neither is present. - The UI surfaces labelled **"Current EPC"** and **"Current Efficiency State"** (property table, building-passport card) show **Effective performance**, not Lodged — despite the glossary advising against "current performance" for Effective. The label is a product choice ("current" = the property's true present-day modelled state); the underlying datum is still **Effective performance**. The **"Lodged EPC"** column/badge is the only surface showing **Lodged performance**, and only when a real certificate exists (`source = lodged`). - "Upload" is used in the codebase to mean both the file-on-S3 and the BulkUpload row. We standardise on **BulkUpload** for the row; the file is just "the source file." - "Onboarding" appears in some route paths (`bulk_onboarding_inputs/...`) but isn't part of this glossary — we use **BulkUpload** end-to-end.