Generated with drizzle-kit generate, then hand-edited to IF NOT EXISTS:
the index already exists on the live DB (created CONCURRENTLY on
2026-07-02), so the migration must no-op there while still building the
index in fresh environments. Snapshot + journal are updated so the next
generated migration doesn't re-emit this index.
Applying it (drizzle-kit migrate) remains a human action.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plan deletes cascade to recommendation rows, and each cascaded delete
fires an FK check against plan_recommendations.recommendation_id. With
~26M rows and no index on that column the check seq-scans per row,
making any plan deletion path time out.
The index already exists on the live DB (created CONCURRENTLY on
2026-07-02 as idx_plan_recommendations_recommendation_id, 74s build);
this brings the Drizzle schema in line so it stops drifting. No
migration is included — generating/applying it is left as a human
action, and it must reuse the existing index name (IF NOT EXISTS) so
it no-ops against prod.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
"Total carbon saved" on the reporting scenario overlay went negative
(portfolio 796: -10,673 t/yr, £/tCO2 -8,195). Cause: the baseline side
(getTotals → carbonSql) summed bp.lodged_co2_emissions_t_per_yr, which
is NULL for predicted-only properties (12,064 of 796's 31,919 — no
mirrored lodged estimate is written), so they contributed nothing to
the baseline. The scenario side counted their plans' post_co2_emissions
(21,046 t) in full, dragging saved below zero. The same subtraction fed
£-per-tonne-CO2 and average-per-unit.
carbonSql and energyConsumptionSql were left on lodged_* when ADR-0002
(effective performance is the canonical current, superseding "reporting
stays on lodged") migrated the SAP fragments — the same reporting cards
were already mixing effective SAP with lodged carbon. Move both to
effective_*, which is populated for every property with a baseline row
and is the figure the plans' post_* values were scored against. All
call sites (reporting averages/totals, both scenario metrics routes,
portfolio table CO2 column/filter) are current-state surfaces, so none
wanted lodged. Legacy branch untouched.
Verified live on 796: baseline and scenario totals computed in one
snapshot with the shared fragment give saved = +4,059 t/yr (a plan
regeneration was writing 796 mid-diagnosis, so values drift run-to-run;
pre-fix the same snapshot maths gives ~ -13k and worsening as plans
land). Legacy portfolio 433 totals unchanged.
No regression-test seam exists for cross-query consistency (no DB test
harness); consistency is structural — both sides compose the same
carbonSql fragment.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
getCountByAgeBand and getCountByPropertyType read year_built /
property_type straight off the property row, which is NULL for
new-approach properties — every one fell into the Unknown bucket on
/portfolio/[slug]/reporting. Branch them like their already-migrated
siblings (getCountByEpcBand, getEstimatedCounts):
- propertyTypeSql: epc_property (lodged, else predicted) with RdSAP
code→label mapping and dwelling_type fallback, mirroring
resolvePropertyDescriptors; legacy stays on property.property_type.
- constructionYearSql: the main building part's construction age band
mapped to a representative start year (correlated subquery, since
extension parts would multiply a plain JOIN), so both paths share the
same year→band bucketing; legacy stays on numeric year_built.
- The RdSAP label maps move into the shared-fragments section as the
single source of truth; AGE_BAND_YEARS becomes CONSTRUCTION_AGE_BANDS
carrying label + representative year. Band I (1996–2002) straddles the
1976–1999 / 2000+ boundary and buckets by its start year.
Verified live against portfolio 814 (all 338 properties resolve, buckets
match the epc_building_part distribution exactly) and legacy portfolios
419/433 (output identical to the pre-fix SQL).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Hyde 796 landlord descriptions include '100% secondary glazing (sap 9.94)'
(39 properties), currently flattened to 'Single glazing' (U = 4.8 instead of
secondary's 2.9). The calculator already supports secondary glazing (SAP10
code 5) — only this FE-owned enum and the model repo's GlazingType lacked a
member. Additive ALTER TYPE ... ADD VALUE only, generated via drizzle-kit.
Pairs with the model-repo GlazingType.SECONDARY change (Model#1416).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Overhaul the recommendation plan_id/material_* backfill for the 21M-row prod
table (see docs/runbooks/recommendation-denormalization-backfill.md):
- Single combined pass: set plan_id + all four material_* columns in one write
per row (was two full passes, rewriting every row twice). Guard + COALESCE keep
it idempotent and resumable — done rows are never rewritten.
- Session tuning: synchronous_commit=off, tunable work_mem / maintenance_work_mem.
- Range + phase controls (BACKFILL_START_ID/END_ID/SKIP_CARDINALITY/ONLY_FINALIZE)
to run disjoint id-range workers in parallel when the disk isn't the bottleneck.
- Percent-complete + ETA in the progress log.
- Finalize builds the two new indexes plain by default (faster in a quiet
window) with a CONCURRENTLY toggle.
The real speed-up in prod was dropping the 3 secondary indexes for the backfill
then rebuilding — the table is packed (fillfactor 100) so updates can't go HOT
and otherwise maintain every index per row. Index drop/rebuild + vacuum are kept
as standalone SQL so ops controls exactly when the app loses/regains them:
- sql/drop-recommendation-backfill-indexes.sql
- sql/rebuild-recommendation-backfill-indexes.sql (1GB maintenance_work_mem — safe
on the 4GB instance)
- sql/vacuum-recommendation.sql
run-sql.ts runs .sql files statement-by-statement, autocommitted (so VACUUM /
CREATE INDEX CONCURRENTLY work), for environments without the psql CLI. Wired up
as npm run db:run-sql and backfill:recommendation-restore.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The shadcn TooltipContent rendered inline, missing the Portal that upstream
shadcn ships. On the Current Efficiency card the info tooltip lives inside a
`relative z-10` div while the Lodged EPC badge is a sibling at `z-20`, so the
popup's own `z-50` only competed within the z-10 subtree and the badge painted
over it (also latently clipped by the card's `overflow-hidden`). Wrapping
TooltipContent in TooltipPrimitive.Portal renders it at <body>, escaping both
the stacking context and the overflow clip — fixes all 9 tooltips.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The merge of main renamed epc_property.energyMainsGas ->
energyGasConnectionAvailable (migration 0252), but the new-approach
resolver read the old property name, which main never touched so it
survived the merge with no conflict and broke the build.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The modelling_e2e Lambda reads the active catalogue once per invocation
(SELECT * FROM material WHERE is_active ORDER BY id). Without an index this
was a full seq scan every time (~23ms), paid by all 32 concurrent Lambdas at
startup. Partial index on id WHERE is_active returns the active rows already
id-ordered — no sort, no filter.
Verified on dev: seq scan 23.2ms -> index scan 4.2ms.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A recursive persistence-fidelity guard surfaced 7 fields the SAP calculator
reads but the epc_property projection had no column for, so they were silently
lost on save->reload (same bug class as the PV arrays / floor flags). All
additive, nullable except the PV-diverter bool; no backfill (defaults match
the domain defaults).
- epc_main_heating_detail: community_heating_boiler_fuel_type,
community_heating_chp_fraction (community-heated CHP).
- epc_building_part: alt_wall_1_is_sheltered, alt_wall_2_is_sheltered, and
wall_insulation_thermal_conductivity (jsonb to preserve Union[int,str]).
- epc_property: energy_pv_diverter_present (NOT NULL DEFAULT false),
heating_cylinder_volume_measured_l, ventilation_air_permeability_ap50_m3_h_m2.
Includes generated migration 0246_minor_lady_bullseye.sql + snapshot/journal.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add EPC persistence schema for two round-trip-fidelity gaps so the Model
backend can store the full EpcPropertyData faithfully:
- New child table epc_photovoltaic_array (one row per PV array), mirroring
the epc_window child-table pattern.
- epc_floor_dimension: add is_exposed_floor and is_above_partially_heated_space
boolean flags (NOT NULL DEFAULT false; additive, no backfill).
Includes generated migration 0245_magenta_nomad.sql + snapshot/journal.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The scenario measures modal came up empty because both measures routes
gated on EXISTS against plan_recommendations — a retired join table with
no rows for new-approach plans (25.7M legacy rows, none for e.g.
portfolio 812's plans), so the query returned zero measures.
Read the denormalised model instead: drive from the latest plan per
property (default or scenario) and JOIN recommendation by the indexed
property_id, scoped to the plan via recommendation.plan_id. Portfolio 812
default now returns 5 measures (solar_pv 54 homes/£266k, …) where it
returned 0. Also removes the stale commented query block that referenced
the retired plan_recommendations / recommendation_materials tables.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The breakdown chart looped only over baseline bands (epcBands), which the
query returns only for bands that have properties. So a band reached ONLY
after the scenario — e.g. A/B when no property starts that high — had no
row, and its scenario bar never rendered (portfolio 812: scenario yields
7 A + 2 B, baseline has none). Iterate a canonical A–G+Unknown order over
baseline ∪ scenario instead, so post-scenario-only bands still appear.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Display the UPRN beneath the selected address on the remote assessment
page so it's available for quicker debugging. The value is already on
the selected address object from the postcode lookup; it just wasn't
rendered.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a NOT NULL DEFAULT false `marked_for_deletion` boolean to the
`property` table. Hard deletes are expensive (cascading child rows,
indexes), so a row is flagged here and a separate cron job performs the
physical delete later; reads can filter on the flag in the interim.
A constant default makes this a metadata-only ALTER on Postgres 11+ (no
table rewrite); existing rows backfill to false.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The modelling writes a target-level post_sap (e.g. ~C) even for homes
already above the scenario target, so plans can carry cost while modelling
post_sap BELOW the effective baseline. That skewed three reporting
surfaces. Three TEMP (demo) guards, all keyed on the effective baseline
(ADR-0002); revert once the Model team fixes the sub-baseline plans:
1. EPC band chart: post-scenario SAP clamped to GREATEST(baseline, post)
so already-compliant properties aren't shown "improving" down a band
(portfolio 796: EPC B 4,244 -> 1,660 wrongly, now 4,479).
2. n_units_upgraded + cost: exclude plans whose post_sap is below the
effective baseline (not real upgrades) -- 796: 10,283 -> 9,765, -£1.28M.
3. total_sap_uplift / £-per-SAP: baseline is the effective SAP, not the
null-for-new-approach current_sap_points, and counts genuine gains
only -- uplift 0 -> 89,724, so £/SAP £0 -> £536.
Also fixes the no-plan branch to use the effective baseline instead of
the null current_sap_points (Unknown-band leakage).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
getPropertiesCount returns the portfolio's total property count for
pagination, but it dragged the whole read model through the COUNT: the
property_details_epc + property_baseline_performance + two epc_property
joins, plus a correlated default-plan LATERAL that ran once per property
(31k+ plan lookups for a large portfolio). None of those joins change a
COUNT (none multiply rows), so for an unfiltered load they were pure cost
— pushing the query to ~14.7s, past Vercel's 15s limit (intermittent
timeout on /api/properties).
Join only what an active filter references: no filters -> plain COUNT over
property; EPC/provenance filter -> add the epc-graph joins; Expected-EPC
filter -> add the plan LATERAL. Unfiltered count 14,667ms -> 93ms
(portfolio 796); provenance-filtered 156ms. getProperties is unchanged
(its LIMIT 1000 already bounds the LATERALs).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
