The reporting 'where the money goes' panel joins the latest plan per
property to its recommendations ON plan_id, filters to the default active
set, and sums estimated_cost / counts property_id per measure_type. The
only plan_id index (idx_recommendation_plan_id) is plain, so the aggregate
heap-fetches every recommendation to apply the filter and read those
columns — hundreds of thousands of fetches on large portfolios (434),
which times the panel out.
idx_recommendation_plan_default carries plan_id + measure_type +
property_id + estimated_cost as key columns (drizzle-orm has no INCLUDE),
partial on default = true AND already_installed = false, so the aggregate
runs index-only. Added to the schema and generated via drizzle-kit.
OPS: build CONCURRENTLY out-of-band on prod before the migration runs so it
no-ops there (0258 pattern); the migration uses IF NOT EXISTS for fresh
environments. See the migration file header.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Foundation for the "Addresses" review tab: captures the user's confirmed
UPRN per combiner row, keyed by source_row_id (no property.id exists
pre-finalise). At dispatchFinaliser these are overlaid onto the combiner
CSV so the finaliser builds identity + property_overrides from the
confirmed UPRN in one pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per-user portfolio workspace layer for the home-page redesign: starred
portfolios (starred_at, non-null = starred, orderable) and personal
folders (name + position, drag-reorderable). Composite FK
(folder_id, user_id) -> (id, user_id) makes cross-user folder
assignment impossible at the database level. Config rows are created
lazily via upsert on (user_id, portfolio_id); portfolioUsers remains
pure access control.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The address2uprn lookup already returns the EPC certificate number
alongside the UPRN, but only the UPRN is persisted today. This adds
a nullable column so a follow-up change can start saving it.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Mirrors sub_task.inputs (JSON as text): the task's request inputs. For a
modelling run (ADR-0008, feature/bulk-trigger-modelling) the task row itself
carries the run's config — portfolio id, scenario ids, filters — with
sub_tasks holding the per-property execution work. Column-only change so it
can be applied to the database ahead of the feature merging.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New provenance value for the deterministic OS Places classification-code
mapping written by the postcode-search journey (ADR-0007, on
feature/add-properties-by-postcode). Enum-only change so it can be
applied to the database ahead of the feature merging.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Generated with drizzle-kit generate, then hand-edited to IF NOT EXISTS:
the index already exists on the live DB (created CONCURRENTLY on
2026-07-02), so the migration must no-op there while still building the
index in fresh environments. Snapshot + journal are updated so the next
generated migration doesn't re-emit this index.
Applying it (drizzle-kit migrate) remains a human action.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plan deletes cascade to recommendation rows, and each cascaded delete
fires an FK check against plan_recommendations.recommendation_id. With
~26M rows and no index on that column the check seq-scans per row,
making any plan deletion path time out.
The index already exists on the live DB (created CONCURRENTLY on
2026-07-02 as idx_plan_recommendations_recommendation_id, 74s build);
this brings the Drizzle schema in line so it stops drifting. No
migration is included — generating/applying it is left as a human
action, and it must reuse the existing index name (IF NOT EXISTS) so
it no-ops against prod.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Hyde 796 landlord descriptions include '100% secondary glazing (sap 9.94)'
(39 properties), currently flattened to 'Single glazing' (U = 4.8 instead of
secondary's 2.9). The calculator already supports secondary glazing (SAP10
code 5) — only this FE-owned enum and the model repo's GlazingType lacked a
member. Additive ALTER TYPE ... ADD VALUE only, generated via drizzle-kit.
Pairs with the model-repo GlazingType.SECONDARY change (Model#1416).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Overhaul the recommendation plan_id/material_* backfill for the 21M-row prod
table (see docs/runbooks/recommendation-denormalization-backfill.md):
- Single combined pass: set plan_id + all four material_* columns in one write
per row (was two full passes, rewriting every row twice). Guard + COALESCE keep
it idempotent and resumable — done rows are never rewritten.
- Session tuning: synchronous_commit=off, tunable work_mem / maintenance_work_mem.
- Range + phase controls (BACKFILL_START_ID/END_ID/SKIP_CARDINALITY/ONLY_FINALIZE)
to run disjoint id-range workers in parallel when the disk isn't the bottleneck.
- Percent-complete + ETA in the progress log.
- Finalize builds the two new indexes plain by default (faster in a quiet
window) with a CONCURRENTLY toggle.
The real speed-up in prod was dropping the 3 secondary indexes for the backfill
then rebuilding — the table is packed (fillfactor 100) so updates can't go HOT
and otherwise maintain every index per row. Index drop/rebuild + vacuum are kept
as standalone SQL so ops controls exactly when the app loses/regains them:
- sql/drop-recommendation-backfill-indexes.sql
- sql/rebuild-recommendation-backfill-indexes.sql (1GB maintenance_work_mem — safe
on the 4GB instance)
- sql/vacuum-recommendation.sql
run-sql.ts runs .sql files statement-by-statement, autocommitted (so VACUUM /
CREATE INDEX CONCURRENTLY work), for environments without the psql CLI. Wired up
as npm run db:run-sql and backfill:recommendation-restore.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The modelling_e2e Lambda reads the active catalogue once per invocation
(SELECT * FROM material WHERE is_active ORDER BY id). Without an index this
was a full seq scan every time (~23ms), paid by all 32 concurrent Lambdas at
startup. Partial index on id WHERE is_active returns the active rows already
id-ordered — no sort, no filter.
Verified on dev: seq scan 23.2ms -> index scan 4.2ms.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A recursive persistence-fidelity guard surfaced 7 fields the SAP calculator
reads but the epc_property projection had no column for, so they were silently
lost on save->reload (same bug class as the PV arrays / floor flags). All
additive, nullable except the PV-diverter bool; no backfill (defaults match
the domain defaults).
- epc_main_heating_detail: community_heating_boiler_fuel_type,
community_heating_chp_fraction (community-heated CHP).
- epc_building_part: alt_wall_1_is_sheltered, alt_wall_2_is_sheltered, and
wall_insulation_thermal_conductivity (jsonb to preserve Union[int,str]).
- epc_property: energy_pv_diverter_present (NOT NULL DEFAULT false),
heating_cylinder_volume_measured_l, ventilation_air_permeability_ap50_m3_h_m2.
Includes generated migration 0246_minor_lady_bullseye.sql + snapshot/journal.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add EPC persistence schema for two round-trip-fidelity gaps so the Model
backend can store the full EpcPropertyData faithfully:
- New child table epc_photovoltaic_array (one row per PV array), mirroring
the epc_window child-table pattern.
- epc_floor_dimension: add is_exposed_floor and is_above_partially_heated_space
boolean flags (NOT NULL DEFAULT false; additive, no backfill).
Includes generated migration 0245_magenta_nomad.sql + snapshot/journal.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a NOT NULL DEFAULT false `marked_for_deletion` boolean to the
`property` table. Hard deletes are expensive (cascading child rows,
indexes), so a row is flagged here and a separate cron job performs the
physical delete later; reads can filter on the flag in the interim.
A constant default makes this a metadata-only ALTER on Postgres 11+ (no
table rewrite); existing rows backfill to false.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Theme 1 — EPC card provenance. New resolveProvenanceSignal (pure, unit-
tested) + resolveProvenance expose provenanceSignal ("estimated" /
"remodelled" / "none") and the lodged headline on property-meta. The
building-passport card shows a "Lodged EPC" badge on re-modelled
properties (even within-band), never on predicted ones; the existing
"estimated from nearby homes" banner already covers predicted.
Theme 2 — portfolio table. "Current EPC" + reporting now read the
effective baseline (effectiveSapSql/effectiveEpcBandSql); a dedicated
"Lodged EPC" column reads the source-gated lodged value (blank for
predicted); a "Predicted" pill marks estimated rows; a provenance filter
is added and the broken currentEpc/lodgedEpc filters (on null row
columns) point at the effective/lodged baselines. Expected EPC now uses
the default plan's post rating (expectedEpcRating, unit-tested) instead
of current + Σ rec SAP, which double-counted across plans (732385: A→C);
the same rec aggregate is scoped to the default plan, fixing the Cost
column's 2x double-count.
See ADR-0002. Pure logic covered by provenance.test.ts + expectedEpc.test.ts;
SQL/query/UI verified against live properties 732385 and 729529.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
