Update the proposal doc to match what was actually built:
- client_id is uuid -> organisation (Client maps to organisation).
- property_id FKs are bigint (Property has a bigint PK).
- status/priority columns documented as text (values TBC).
- Drop the free-text uploaded_by column; note the existing bigint
FK -> user is reused.
- Document the new minimal project_type table.
- Record that a "projects" file_source value was added.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduce the Projects domain model as Drizzle table definitions
(schema-only; no migration generated/run yet).
New tables in src/app/db/schema/projects/projects.ts: project,
project_type, project_property, contractor, workstream,
project_workstream, project_workstream_stage,
project_workstream_contractor, document_type,
project_workstream_evidence_requirement, work_order.
Also:
- uploaded_files: add "projects" file_source value and three nullable
bigint FK columns linking evidence to the Projects module.
- relations.ts: wire the Projects relations.
- db.ts: register the new schema module.
Deviations from schema.md, decided during implementation:
- client_id is uuid -> organisation (Client maps to organisation).
- property_id FKs are bigint (Property has a bigint PK, not integer).
- TBC status/priority columns modelled as text for now.
- No new uploaded_by column; reuse the existing bigint FK -> user.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The card rendered a blank value and "NaN%": Postgres returns column names
verbatim, so `AS without_epc` came back as `without_epc` while the code read
`withoutEpc` — undefined, and `undefined / total * 100` is NaN.
`db.execute<T>()` asserts the row shape rather than checking it, so tsc, lint and
the full suite all passed on the broken query. The previous field names (`estimated`,
`actual`) were single words and happened to survive the round trip; renaming to
camelCase did not. Quoted aliases are the existing convention (see utils.ts).
Verified against portfolio 839: keys come back as [withoutEpc, withEpc], and the
card renders 29 / "0.5% have no EPC on record".
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Impeccable critique (P1): the escalation was a bare blue text-link on the bar's
pale-blue tint, no persistent underline — invisible next to the navy primary,
and it's the exact action users hunt for. Promote it to a secondary button
(white fill + brand-navy border + check icon, fills on hover) so it stands out
without competing with "Choose a tag"; demote "Clear selection" to a legible
neutral underlined link (was blue-on-blue).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
"Homes Without an EPC" and "Expired EPCs" were both derived from estimatedSql,
which asks "was this picture gap-filled?". That conflates two different questions:
provenance (was it modelled?) and coverage (does a certificate exist?).
They used to agree, because a gap-fill implied no certificate. Model ADR-0054 ends
that: the gov EPC API only serves certs registered since 2012, so a pre-2012
dwelling is gap-filled even though a real, stale certificate exists — and gets the
new source='expired'. On portfolio 824, 375 of the 403 "without an EPC" homes have
exactly that: a real certificate, just an out-of-date one. A different, and far more
actionable, problem than never having been certified.
Two defects fell out of the same conflation:
- Every query matched source='predicted' literally, and `source` is plain text
with no enum or CHECK. An 'expired' row matches neither the lodged nor the
predicted alias, so the home would have read as having a VALID CURRENT cert —
silently gone from both cards, no Estimated badge, nothing to catch it.
- isExpiredSql read epl.registration_date alone, but 714 of 824's 4,873 lodged
certs carry only inspection_date. All 714 counted as current; 170 are in fact
>10 years old. lodgementDateSql already coalesced the two; expiry didn't.
The cards now partition on coverage: withoutEpcSql = no lodged AND no historical
record; isExpiredSql = a cert of either kind, out of date. estimatedSql keeps its
own meaning and spans the whole predicted slot, so an expired home is estimated AND
has an EPC — both true. The `AND estimated = false` guard is gone from the Expired
card: an expired home IS estimated, so it excluded the very homes being counted.
Legacy portfolios are untouched (632: 8/1662 before and after). 824 moves to
913 → 1,083 expired immediately (the inspection_date fix), then to 25 / 1,461 once
Model writes expired rows. Safe to deploy BEFORE Model — the predicates just find
none. The reverse was not safe, which is why this goes first.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
With a 7-row page size, the header "select all" (toggleAllPageRowsSelected)
only selected the visible 7 — so it never reached "all loaded", the condition
that reveals the "Select all N matching" escalation. It now selects every loaded
row (toggleAllRowsSelected):
- when everything matching is already loaded, that IS the whole set (tag via ids);
- when more match than are loaded (e.g. 250 loaded of 10,000), the escalation
appears and routes through mode:"filter" — the server resolves and tags the
entire matching set (getFilteredPropertyIds, chunked), so memory is never the
ceiling and no huge id list is sent from the client.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The header checkbox only selects loaded rows, so bulk-tagging a portfolio with
more properties than are loaded (250-row window) silently tagged just the page.
Bulk mode is now explicit (Gmail pattern): nothing selected = no target (Choose
a tag is disabled), so a tag can't be applied to everything by accident. Tick
rows to target them; once every loaded row is ticked and more match, a "Select
all N matching" escalation switches the target to the whole matching set,
resolved server-side via the existing assignments filter mode (no backend
change). Any manual checkbox change drops the escalation back to the explicit
selection; changing filters / leaving the mode clears it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Surface Design Constraints, Planning Comments, Planning Status, and
Planning Suggested Approach in the Live Reporting properties table,
following the same read-path wiring as the previous planning columns
(query mapping, HubspotDeal type, column defs, hidden-by-default toggle,
CSV export, test fixtures). DB columns already exist via migration 0267.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Surface the four planning columns from hubspot_deal_data — Planning
Authority, Designated Area, Article 4 PD Rights, Listed Building — as
optional columns in the Live Reporting properties table.
The DB columns already exist (migration 0267); this wires them through
the read path only: query mapping, HubspotDeal type, column defs, the
column-visibility toggle (hidden by default), and CSV export. Rendered
as plain text since they are text columns in HubSpot.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The not-modelled banner and the toolbar both rendered a navy "Run modelling"
CTA whenever a portfolio had properties but no plans — two identical buttons.
The banner owns the action in that state, so the toolbar's Run modelling is now
hidden while the banner is up (it returns once everything is modelled or the
banner is dismissed). One primary CTA at a time, as intended.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The selection checkbox column was persistently visible. Now it appears only
when a bulk assign/remove mode is active (launched from the Tags menu) via a
select-less column set, and the selection clears on entering and leaving the
mode. The default table has no checkbox column; per-property tagging still uses
the inline Tags-cell popover, and the bulk bar's target still follows selection
→ filter → whole portfolio while the mode is open.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Design pass on the portfolio table (critique: 27/40; P0 was the unmodelled state).
- onboard: a mixed-safe "not modelled yet" nudge above the table — shown whenever
any loaded property lacks a plan (so a part-modelled portfolio still gets it),
wired to Run modelling. Turns a wall of empty cells into a next step without
hiding real data for the rows that ARE modelled.
- harden: one empty-cell vocabulary (EmptyCell) replacing the four divergent
renderings (blank div, slate-300 "—", italic "No cost", "Unknown"); AA-legible
and aria-labelled. Empty EPC bubble is now a labelled placeholder ring, not an
invisible div. Export failures surface an inline error instead of dying silently.
- a11y: the quick-filter clear is no longer a <span role=button> nested inside a
<button> — pill is a container with sibling open/clear buttons.
- distill: exactly one navy CTA at a time — Add properties leads an empty
portfolio, Run modelling leads a populated one; the other demotes to neutral.
- colorize: black tokens in PropertyFilters + the Edit-Columns checkboxes now use
the brand navy/brandmidblue scale (was raw black, drifting off the toolbar).
- cleanup: removed the unreachable delete-property modal (no DELETE endpoint,
onDeleteProperty never called). Real property deletion is its own slice.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replaces the isolated "Assign tags from file" button with a single Tags ▾
dropdown in the toolbar (grouped with Export / Add properties), offering the
three bulk paths: upload from file, assign a tag, remove a tag.
Assign/Remove enter a lightweight contextual mode (TagActionBar) whose target
follows the live table — the row selection if any, else the active filter set,
else the whole portfolio — so ticking rows or changing filters re-scopes it in
place, and it supports removal, not just assignment. Row checkboxes stay visible
at all times; a "N selected" chip now sits by the results count. The upload
modal gains downloadable CSV templates (Property Ref / UPRN).
Removes TagBulkBar (superseded by TagsMenu + TagActionBar).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Reject `tagIds` in the modelling-runs trigger route (400) until the distributor
resolves tag_ids — the RunModellingClient guard was UI-only, so a direct POST
(or a future caller) could dispatch a tagged run that silently models the
unscoped superset (preview≠run, ADR-0008). Defence-in-depth now.
- Narrow the tags POST catch to Postgres 23505 (new isUniqueViolation helper) so
transient/other DB errors surface as 500, not a misleading duplicate-name 409.
- Wrap the PATCH rename in the same unique-violation handler — the pre-check is
racy, so a concurrent rename collision now returns 409, not an uncaught 500.
- TagsCard reuses usePortfolioTags / portfolioTagsKey / Tag instead of a copied
query + key, so every tag surface stays in sync if the key changes.
- Scope DataTable getRowId to selection-enabled tables only, leaving other
consumers (plan table) on TanStack's default row identity.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The shared DataTable is also used by the plan table, whose PlanWithTotals rows
have no `id` (they carry planId/propertyId). A bare String(row.id) keyed every
plan row as "undefined" — colliding React keys. Key by id when present, else by
index (restoring the pre-change default for idless consumers).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Impeccable pass. Replaces the solid full-saturation chips (a heavy-accent
inconsistency vs the settings card's soft chips) with a single quiet vocabulary
per PRODUCT.md: a soft tint of the Tag's freeform hex, a colour dot, and dark
hue-preserving ink whose contrast on the tint is guaranteed ≥4.5:1 for any
colour (new tagChipTokens, TDD — yellow and every hue verified). Settings,
table, filter, bulk bar, upload, and run UI now share one TagChip. Also drops a
stray dev scratch file and gitignores the pattern.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a Tags FilterColumn (per-portfolio options from GET /tags, colour dots,
counts) feeding filters.tagIds; folds it into filterCount/clearFilters and
removes the "Tags are coming" copy. The in-app preview already counts by tag
membership, so the selection previews exactly. The run button is guarded while
tags are selected — the distributor can't resolve tag_ids yet (backend ask,
ADR-0013), so previewing is honest but running stays disabled until it lands.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds row multi-select to the property table (new checkbox column + stable
id-keyed selection surviving pagination) with a bulk bar that tags the
selection ({mode:"properties"}) or every property matching the active filter
({mode:"filter"}, whole set resolved server-side). Plus a Bulk tag assignment
upload modal: client-side SheetJS → parseTagIdentifierRows → pick a target tag
→ {mode:"identifiers"} → match summary (matched / already-tagged / unmatched).
Selection is filter-scoped and clears on filter change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a "Tags" field to PropertyFilters with per-portfolio options fetched
via GET /tags plus a synthetic "Untagged" bucket. Emits enum_one_of with
value = JSON of tag ids (+ "__untagged__"); the server SQL already resolves
this (any-of, ADR-0013). Condition chips label by tag name and render
"(deleted tag)" for dangling ids.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Chips with "+N" overflow in a default-visible, toggle-able column; the
cell doubles as an assign popover that toggles membership via
POST …/assignments {mode:"properties"}, optimistic across every cached
property-table page and reconciled on settle. Freeform-hex chips get an
auto-contrast foreground (new pure colour util, TDD). Shared usePortfolioTags
hook keeps settings/table/filter in sync.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Backend, end to end (ADR-0013):
- getProperties now returns each property's tags (name-ordered JSON); the
PropertyWithRelations type gains a `tags: PropertyTag[]` field.
- A "tags" property-table filter (EXISTS/NOT-EXISTS on property_tag, any-of +
an "Untagged" bucket) — PK-indexed by property_id, so no join added.
- getFilteredPropertyIds resolves the whole matching set for filter-driven Bulk
tag assignment; the assignments route's `filter` mode uses it.
- Run filter: previewModellingRun joins property_tag for tag_ids; the two
modelling routes accept tagIds. Distributor ask documented in
docs/backend-asks/tag-ids-run-filter.md.
UI:
- Settings → Tags: create (name + colour), edit, delete-with-count-confirm,
list with property counts (TanStack v4). Sidebar link added.
Typecheck clean; 431 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Run filter gains an optional tag_ids key (ADR-0013): normaliseRunFilters
coerces/dedupes ids, buildRunRecord + parseRunConfig round-trip it, and
selectionSummary counts it. Pure, TDD (9 new cases).
- parseTagIdentifierRows (pure, TDD): a CSV/Excel 2D array → the identifier
column to match on (landlord_property_id or uprn, header-detected), trimmed,
deduped, capped at 10k.
- Routes (write-gated via shared getPortfolioRole/canWriteTags):
GET/POST /tags (list w/ counts, create), PATCH/DELETE /tags/[id] (edit,
delete+cascade), POST /tags/[id]/assignments (properties add/remove +
identifiers bulk-match with a summary). Names validated by validateTagDraft.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
validateTagDraft: trims + requires a name, caps it at 30 chars, rejects a
case-insensitive duplicate against the portfolio's existing tag names (the
caller excludes the tag's own name on edit), and validates + lowercases a
freeform hex colour (#rgb or #rrggbb). The DB's case-insensitive unique index
remains the ultimate guard; this gives a friendly pre-check for the tag routes
and settings UI. 8 behaviours, built red-green. ADR-0013.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Captures the grill-with-docs outcome: flat app-owned Tags, four assignment
paths (incl. in-app bulk upload, no FastAPI), tags as a Run-filter key extending
ADR-0008, hard-delete + cascade, reporting-by-tag deferred to the reporting
redesign. Design only — no implementation yet.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Flat, portfolio-owned, many-to-many label for grouping Properties for bulk
actions. Captured during the tagging grill-with-docs session.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The schema substrate for the portfolio tagging system (ADR-0013): a flat,
portfolio-owned Tag (name + freeform hex colour, case-insensitively unique per
portfolio) and the many-to-many property_tag membership. Both FKs cascade so a
tag delete drops its memberships and a property/portfolio delete drops its tags.
A tag_id-leading index backs the reverse lookup (chips, filter, run resolution).
Migration only — not yet applied to prod. Feature code stacks on this branch.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Updates the portfolio Property-table flagged-ambiguity note for the two changes
in the previous commit: Main Fuel now reads epc_main_heating_detail beneath the
override (no longer override-only for new-approach), and Wall/Roof/Heating are
now filterable via coarse prefix/substring buckets while still displaying the
full free-text description.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The portfolio list resolves Main Fuel per row via a correlated subquery on
epc_main_heating_detail.epc_property_id (mainfuelSql, ADR-0012). The table has
no index on that column, so the subquery seq-scans it once per property in the
count/export path. At ~41k rows it stays in memory (a 32k-property Main-Fuel
count measures ~1.2s), but this index makes it an index probe (~50ms) and
guards against larger new-approach portfolios.
Migration only — not yet applied to prod. Companion to PR #384.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Main Fuel no longer reads "Unknown" for new-approach properties. The new EPC
graph stores the main fuel as a JSONB scalar on
epc_main_heating_detail.main_fuel_type — a numeric RdSAP main_fuel code (~95%)
or an EPR text string (~5%). mainfuelSql now reads it (lodged over predicted)
instead of NULL; resolvedMainFuelSql keeps the override → EPC → 'Unknown'
precedence. The RdSAP code table (Model epc_codes.csv, cross-checked against
prod) + the EPR strings fold into MAINFUEL_OPTIONS, so the column label and the
filter work unchanged. resolveMainFuel gains the EPC branch as its pure twin;
coverage tests assert every code + string + override value is folded.
Verified: portfolio 824 now resolves code 26 → "Mains Gas".
Wall Type / Roof Type / Heating System become filterable. Their resolved values
are free-text (override vocab / epc_energy_element.description / legacy column)
with high cardinality, so exact dbValues can't scale — each coarse bucket is a
set of case-insensitive needles matched by prefix (wall/roof) or substring
(heating). classifyDescriptor is the pure twin of the SQL bucketing in
buildConditionSql; the "Unknown" bucket matches the complement of every known
needle. Buckets validated against the full prod vocabulary (heating Unknown =
28/640k rows). Wired through FilterField, FIELD_OPTIONS, buildConditionSql,
EPC_JOIN_FILTER_FIELDS and the count-query override joins; columns stay
free-text display. Coverage tests assert every Wall/Roof/MainHeatingSystem
override value buckets to a real, non-Unknown label.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three property-list improvements:
1. Fetch 250 rows initially, not 1000 (DEFAULT_LIMIT/DISPLAY_LIMIT). The table
paginates at 7/page and isn't bulk-scrolled, so the join-heavy query stays
fast; load-more still pulls the rest on demand.
2. Export now fetches its own batch (up to EXPORT_LIMIT) for the current filters
instead of dumping the on-screen rows — so the small display fetch doesn't cap
it — and the cap is raised 1000 -> 6000. Export button shows a loading state.
3. Add Wall Type / Roof Type / Heating System optional columns, resolved through
the same override precedence as the others (override -> EPC epc_energy_element
description, lodged over predicted -> legacy property_details_epc -> 'Unknown').
Per-part components (wall/roof) read the main building part. New generic
resolveOverrideDescriptor (+5 tests) with SQL twins in epcSources. Columns are
display-only (no filter — the wall/roof override vocab is too large for chips)
and included in the CSV export.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The property-list query (getProperties) resolves each property's latest default
plan via a LATERAL. The existing plan indexes lead with (portfolio_id,
scenario_id, …); with scenario_id unconstrained the lookup scans the whole
portfolio's plans per property. On portfolio 796 (~32k properties) EXPLAIN ANALYZE
showed this LATERAL at 11.5s of an 11.9s query (2.97M shared buffer hits,
11.5ms × 1000 loops).
idx_plan_property_default_latest leads with property_id (partial on is_default)
so the lookup becomes a direct index seek + LIMIT 1 — expected ~12s -> sub-second.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Slice 6. The override → EPC-derived → 'Unknown' rule was inlined in
previewModellingRun and duplicated by the portfolio list. Replace the inline
COALESCE expressions and property_overrides joins with the shared
resolvedPropertyTypeSql / resolvedBuiltFormSql / *OverrideJoin fragments — a
byte-identical substitution — so the modelling preview and the list are provably
in sync (the divergence ADR-0008 warns about is now structurally impossible).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Slice 5. Tenure was broken for new-approach (read p.tenure, NULL) but has no
override component and its epc_property.tenure values don't reconcile with the
filter dbValues, so it's removed from the list rather than fixed (ADR-0012):
the optional column, the filter field, TENURE_OPTIONS, and the CSV column. The
per-property building passport (resolvePropertyMeta/resolvePropertyDescriptors)
still resolves tenure, and property.tenure is retained.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Slice 4. The new EPC graph exposes no main-fuel string (documented gap), so for
new-approach properties the landlord override is the only source. resolveMainFuel
returns the raw value (override → legacy EPC → 'Unknown'); the cell's
resolveEnumLabel folds it to a chip. MAINFUEL_OPTIONS.dbValues extended with the
override vocabulary so overridden fuels are both displayable and filterable —
guarded by a coverage test over MainFuelValues. resolvedMainFuelSql mirrors it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Slice 3. Construction age is modelled as an RdSAP band, not a year (ADR-0012) —
the only representation that faithfully carries an 'Unknown' override. resolve-
ConstructionAge owns override(letter)→label / EPC band(letter)→label / legacy
year→band bucketing / 'Unknown', with 10 unit tests incl. band boundaries.
resolvedConstructionAgeSql mirrors it (emits the band label). YEAR_BUILT_OPTIONS
becomes the band labels + Unknown, guarded against label/en-dash drift by a new
epcSources.test. Column + filter + CSV header renamed to 'Construction Age'
(internal 'yearBuilt' key retained to avoid a wide rename).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Slice 2. Same shape as slice 1: resolveBuiltForm + builtFormOverrideJoin /
resolvedBuiltFormSql, wired into getProperties SELECT, the filter colMap, and the
count query's join set (builtForm added to EPC_JOIN_FILTER_FIELDS). BUILT_FORM_OPTIONS
gains Unknown (Not Recorded now matches only an explicit override of that literal).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Slice 1 of the portfolio-list descriptor fix. Property Type was read from the
raw property row (p.property_type), which is NULL for new-approach properties
(updated_at >= 2026-06-01), so the column and filter broke for new portfolios.
- New pure resolver resolvePropertyType (descriptorResolution.ts, mirrors the
provenance.ts split) owning override → EPC-derived (lodged over predicted,
RdSAP codes mapped, legacy fallback) → 'Unknown', with 9 unit tests.
- SQL twin resolvedPropertyTypeSql + propertyTypeOverrideJoin in epcSources.ts.
- Wired into getProperties SELECT, the filter colMap, and the count query's
conditional join set (propertyType added to EPC_JOIN_FILTER_FIELDS).
- PROPERTY_TYPE_OPTIONS gains Park home + Unknown.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
