- The run's config now lives on the task row's new `inputs` column
(migration PR #361) instead of a config subtask — sub_tasks belong
entirely to the distributor's execution work.
- Distributor path corrected to /v1/modelling/trigger-run.
- Sub-task granularity is batches, not properties: history shows progress
as a percentage under the status chip; the Plans column always shows the
plan count (batch counts are never comparable to property/plan numbers).
- ADR-0008 amended accordingly.
Live verification of the task write waits for PR #361 to be applied.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mirrors sub_task.inputs (JSON as text): the task's request inputs. For a
modelling run (ADR-0008, feature/bulk-trigger-modelling) the task row itself
carries the run's config — portfolio id, scenario ids, filters — with
sub_tasks holding the per-property execution work. Column-only change so it
can be applied to the database ahead of the feature merging.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The distributor gets {task_id, portfolio_id, scenario_ids, filters} only —
previewed count and triggered_by are app provenance, already durable on the
config subtask keyed by the same task_id.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
previewModellingRun now resolves Run filters in one grouped SQL statement
(override snapshot -> EPC-derived via propertyTypeSql/new builtFormTypeSql
-> Unknown) and intersects the matched set with plan for per-scenario
already-modelled counts. The Model-backend ask shrinks to the distributor
alone; the app's preview is the reference implementation of the resolution
rule (ADR-0008 amended).
Verified live against 814: all=340 matched / 338 already modelled in
scenario 1271; B93 8SU=2 resolving House/Detached through override
snapshots; Bungalow/Unknown=0.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Builds the bulk-trigger-modelling journey per ADR-0008 on the TDD'd core:
- Server module: triggerModellingRun (app-authored task + config subtask,
BulkUpload trigger convention; dispatch to /v1/plan/trigger-run; failed
dispatch marks the task failed), previewModellingRun (dry-run count proxy
to /v1/plan/preview-run), listModellingRuns (one grouped portfolio-scoped
query; config parsed back, progress from execution-subtask counts),
scenarioIdsWithActiveRuns (third badge state).
- Routes: POST/GET /api/portfolio/[id]/modelling-runs, POST .../preview,
GET /api/portfolio/[id]/postcodes (grouped distinct + counts).
- Page /portfolio/[slug]/modelling/run to the approved mockup in the
scenarios-tab design language: scenario picker with the three badges and
in-flight warning, three filter columns (postcodes from DB, canonical
enums + Unknown), engine-counted summary tray with re-model callouts,
large-run confirm (10k gate), done state with ETA, Recent runs table
with live progress. TanStack Query v4; no useEffect/useMemo.
- "Run modelling" button beside Add properties in PropertyTable.
Backend contract (Model team): the distributor at /v1/plan/trigger-run
accepts {task_id, portfolio_id, scenario_ids, filters, ...} and attaches
execution subtasks to the given task; /v1/plan/preview-run returns
{matched_properties, per_scenario[]} through the same resolution code path.
Verified live against portfolio 814: postcodes, validation errors, unknown
scenarios, dispatch failure marks the task failed and history shows it;
simulated backend execution subtasks drove status to
{in_progress, ready 2, total 4} and the scenario badge to "Modelling in
progress" in the rendered page; portfolio page shows the button. tsc, lint
and all 365 vitest tests pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The DB schema, value enums, and migrations already defined all nine landlord
override categories, but the application layer was hardwired to the original
four (property_type, built_form_type, wall_type, roof_type). The other five
(main_fuel, glazing, construction_age_band, water_heating, main_heating_system)
were classified and written to their tables by the backend, but the frontend
never read them or offered edit dropdowns — so the verify panel rendered them
as "not classified", and any Unknown in those categories slipped past the
ADR-0006 finalise gate.
Replace the per-category switch statements (duplicated across ~6 sites) with a
single CATEGORY_TABLES registry (categories.ts) as the source of truth. Adding a
category is now one registry row. A completeness test asserts the registry,
CLASSIFIER_FIELDS, and the property_overrides override_component enum never
drift apart — the exact hole that left five categories unwired.
- categories.ts: registry + derived CATEGORY_VALUES + isLandlordCategory guard
- landlordOverrides/server.ts: getLandlordOverrides reads all 9 (fixes the
latent crash where the page mapped 9 fields against a 4-key result)
- bulkUpload/server.ts: lookupOverrides / unknownForField / getUnknownOverrides
/ upsertUserOverride / validation now registry-driven, all 9
- OnboardingProgress.tsx: edit dropdowns use the shared 9-category CATEGORY_VALUES
- landlord-overrides page: SourceBadge now renders os_places (ADR-0007) distinctly
instead of mislabelling it "classifier"
Enum value sets are already in sync FE<->Model (verified); only member order
drifts on 3 enums, which is cosmetic and left untouched (Postgres cannot reorder
enums). Follow-up (backend): Model's override_source SAEnum still lacks os_places.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Amends ADR-0008 after challenge: the Modelling run reuses the BulkUpload
trigger convention (app inserts the tasks row + config subtask with inputs
JSON, passes task_id to the distributor) instead of a dedicated
modelling_run table. No migration; status and progress come from the task
system by construction; dispatch failure marks the task failed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The dispatch confirmation sets a time expectation only; the email promise
stays out of the UI until a completion notification exists (run_id -> task
correlation provides the signal to build it on).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Grilling-session outcomes for the bulk-trigger-modelling feature: the
ModellingRun/Run-filter glossary entries and the decision record — insert-only
run rows, filters (not ids) to the distributor lambda, backend-owned filter
resolution with a shared-code-path count endpoint, run_id correlation into
task records for derived in-flight state, warn-don't-block concurrency.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Left uncommitted by the previous session; restores the "### Building parts"
heading it had accidentally swallowed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Order address candidates naturally so Flat 2 sorts before Flat 10
(TDD'd in the domain core; numeric locale compare).
- Disable spell check / autocomplete / autocorrect on the postcode input.
- Long result lists and the tray's postcode chips now scroll inside
their containers instead of growing the page.
- gitignore docs/wip/** and untrack the postcode-search handover doc.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Completes the add-properties-by-postcode journey (ADR-0007) on top of the
TDD'd domain core:
- GET /api/portfolio/[id]/properties/search?postcode= — postcodes.io for
validation + geography (admin_district, parliamentary_constituency) every
time; OS Places through the 30-day postcode_search read-through cache
(refresh=1 bypasses it). Returns candidates with alreadyInPortfolio flags
from one grouped uprn query. Cache reads accept legacy space-less keys;
writes use the canonical spaced form.
- POST /api/portfolio/[id]/properties — re-derives facts server-side from
classification codes (client propertyType never trusted), then one
transaction: property rows via uq_property_portfolio_uprn ON CONFLICT DO
NOTHING, vocabulary upserts with os_places provenance (DO NOTHING preserves
user rows; snapshots mirror the resolved value), property_overrides
snapshots (building_part 0, original description = OS code), and
property_details_spatial per-property coordinates. Returns {added, skipped}.
- OS Places fetches now request output_srs=EPSG:4326 so records carry
LAT/LNG (no mapper read the BNG X/Y coordinates).
- /portfolio/[slug]/properties/add built to the approved mockup: search card
with cache-age line, per-postcode result cards, cross-postcode selection
tray, done state. TanStack Query v4, no useEffect/useMemo.
- "Add properties" dropdown beside Export in PropertyTable (search by
postcode + bulk excel import "Soon"); toolbar AddNew dropdown retired.
- New display/cache helpers (describeCandidate, postcodeCacheKeys,
cacheAgeDays) TDD'd in the domain core.
Verified live against portfolio 814: search (cache + refresh), submit
(added 2, all four write layers checked in DB), dedup re-submit (skipped 2),
alreadyInPortfolio flags, page + dropdown render.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New provenance value for the deterministic OS Places classification-code
mapping written by the postcode-search journey (ADR-0007, on
feature/add-properties-by-postcode). Enum-only change so it can be
applied to the database ahead of the feature merging.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
TDD'd (9 tests): postcode normalisation to canonical form; the ADR-0007
deterministic OS classification mapping (RD02/03/04/06/07; other
residential codes selectable with no facts, Unknown never written;
RD10 + non-RD unselectable); 30-day cache freshness for postcode_search;
DPA-preferred UPRN-deduped address candidates with postcode-stripped
title-case addresses; buildWritePlan producing the property row
(incl. local_authority/constituency from postcodes.io — closing the
no-source gap) and 0-2 property_overrides facts keyed to the OS code.
Adds 'os_places' to OverrideSourceValues — enum migration still needs
generating via generate_migration.sh before deploy.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The docs/adr/** ignore rule silently ate ADRs 0004-0006 that CONTEXT.md
still references. Remove it so decisions actually land in the repo.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Internal wall and floor insulation (and room-in-roof) involve major
internal works, and landlords often exclude them. Encode that domain
knowledge once as DISRUPTIVE_MEASURES in the scenario model (with
plain-language reasons; guarded by a key-validity test) and surface it
three ways:
- measures step: allowed disruptive chips carry an amber "disruptive"
tag + amber border, with the reason in the tooltip; a legend explains
the marking
- "Low disruption" quick-start now excludes exactly this set, so the
preset and the badges cannot drift apart
- review step: an amber nudge lists any disruptive measures still
allowed before saving
New CONTEXT.md term: Disruptive measure.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Design feedback:
- Target-band picker reads A→G left-to-right (reading order; the
gallery/detail meters stay G→A as they mirror the building passport's
efficiency gauge, which fills low→high).
- Unselected picker chips are now light band-colour tints with one
consistent dark ink — no ink has legal contrast across all seven band
colours at full saturation, so the old full-colour+opacity fade made
letter colours look inconsistent and washed out. Only the selected
chip goes full colour, with luminance-correct ink (shared bandInk()).
- Excluded measure chips recede (gray, struck through) instead of
filling dark navy — "Exclude all" no longer paints a wall of blue;
the measures still in play are the visually dominant ones. State is
never colour-only: ✓/✕ icon + strikethrough carry it too.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Swap the native confirm() popup for the app's shadcn Dialog pattern
(as in Settings → Danger Zone): the destructive button stays disabled
until the user types "delete" (case-insensitive), with Enter-to-submit
and a proper explanation of what's removed. On a delete error the
dialog closes so the inline error is visible.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Users could only create scenarios buried inside the CSV-upload modal, at
trigger time. This adds a first-class Scenarios tab: a gallery of the
portfolio's scenarios, a read-only detail page, and a 3-step creation
journey (goal → measures → review) that saves app-authored scenario rows
per ADR-0003.
Domain rules (TDD'd in src/lib/scenarios/model.test.ts, 19 tests):
- goal semantics: "Increasing EPC" requires a target band; all other
goals are maximise-within-budget with goal_value forced null
- exclusions-only measure model: normalised (dedupe+sort), unknown keys
rejected, all-excluded rejected, empty set valid; serialised to the
backend's brace text format (parses the legacy space-padded variant)
- insert mapping: multi_plan=true, is_default=false, budget per property
- exact-config duplicate detection ignoring name (review-step warning)
- status derived from plan existence — no stored status column
Queries follow the ADR's discipline: gallery status = one grouped
COUNT(DISTINCT property_id) per portfolio; detail = one scoped count
(index-only, powers the blocked-delete message); delete = single atomic
DELETE guarded on NOT EXISTS(plans). Rename is the only other mutation —
configuration is immutable.
Verified live on portfolio 814: create (400 on missing band, 201 on CO₂
draft), derived Awaiting/Modelled in the gallery, rename, delete of the
draft (200) and blocked delete of the modelled scenario (409, "338
properties…" matching the portfolio's true count).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Generated with drizzle-kit generate, then hand-edited to IF NOT EXISTS:
the index already exists on the live DB (created CONCURRENTLY on
2026-07-02), so the migration must no-op there while still building the
index in fresh environments. Snapshot + journal are updated so the next
generated migration doesn't re-emit this index.
Applying it (drizzle-kit migrate) remains a human action.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plan deletes cascade to recommendation rows, and each cascaded delete
fires an FK check against plan_recommendations.recommendation_id. With
~26M rows and no index on that column the check seq-scans per row,
making any plan deletion path time out.
The index already exists on the live DB (created CONCURRENTLY on
2026-07-02 as idx_plan_recommendations_recommendation_id, 74s build);
this brings the Drizzle schema in line so it stops drifting. No
migration is included — generating/applying it is left as a human
action, and it must reuse the existing index name (IF NOT EXISTS) so
it no-ops against prod.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
"Total carbon saved" on the reporting scenario overlay went negative
(portfolio 796: -10,673 t/yr, £/tCO2 -8,195). Cause: the baseline side
(getTotals → carbonSql) summed bp.lodged_co2_emissions_t_per_yr, which
is NULL for predicted-only properties (12,064 of 796's 31,919 — no
mirrored lodged estimate is written), so they contributed nothing to
the baseline. The scenario side counted their plans' post_co2_emissions
(21,046 t) in full, dragging saved below zero. The same subtraction fed
£-per-tonne-CO2 and average-per-unit.
carbonSql and energyConsumptionSql were left on lodged_* when ADR-0002
(effective performance is the canonical current, superseding "reporting
stays on lodged") migrated the SAP fragments — the same reporting cards
were already mixing effective SAP with lodged carbon. Move both to
effective_*, which is populated for every property with a baseline row
and is the figure the plans' post_* values were scored against. All
call sites (reporting averages/totals, both scenario metrics routes,
portfolio table CO2 column/filter) are current-state surfaces, so none
wanted lodged. Legacy branch untouched.
Verified live on 796: baseline and scenario totals computed in one
snapshot with the shared fragment give saved = +4,059 t/yr (a plan
regeneration was writing 796 mid-diagnosis, so values drift run-to-run;
pre-fix the same snapshot maths gives ~ -13k and worsening as plans
land). Legacy portfolio 433 totals unchanged.
No regression-test seam exists for cross-query consistency (no DB test
harness); consistency is structural — both sides compose the same
carbonSql fragment.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
getCountByAgeBand and getCountByPropertyType read year_built /
property_type straight off the property row, which is NULL for
new-approach properties — every one fell into the Unknown bucket on
/portfolio/[slug]/reporting. Branch them like their already-migrated
siblings (getCountByEpcBand, getEstimatedCounts):
- propertyTypeSql: epc_property (lodged, else predicted) with RdSAP
code→label mapping and dwelling_type fallback, mirroring
resolvePropertyDescriptors; legacy stays on property.property_type.
- constructionYearSql: the main building part's construction age band
mapped to a representative start year (correlated subquery, since
extension parts would multiply a plain JOIN), so both paths share the
same year→band bucketing; legacy stays on numeric year_built.
- The RdSAP label maps move into the shared-fragments section as the
single source of truth; AGE_BAND_YEARS becomes CONSTRUCTION_AGE_BANDS
carrying label + representative year. Band I (1996–2002) straddles the
1976–1999 / 2000+ boundary and buckets by its start year.
Verified live against portfolio 814 (all 338 properties resolve, buckets
match the epc_building_part distribution exactly) and legacy portfolios
419/433 (output identical to the pre-fix SQL).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Hyde 796 landlord descriptions include '100% secondary glazing (sap 9.94)'
(39 properties), currently flattened to 'Single glazing' (U = 4.8 instead of
secondary's 2.9). The calculator already supports secondary glazing (SAP10
code 5) — only this FE-owned enum and the model repo's GlazingType lacked a
member. Additive ALTER TYPE ... ADD VALUE only, generated via drizzle-kit.
Pairs with the model-repo GlazingType.SECONDARY change (Model#1416).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Overhaul the recommendation plan_id/material_* backfill for the 21M-row prod
table (see docs/runbooks/recommendation-denormalization-backfill.md):
- Single combined pass: set plan_id + all four material_* columns in one write
per row (was two full passes, rewriting every row twice). Guard + COALESCE keep
it idempotent and resumable — done rows are never rewritten.
- Session tuning: synchronous_commit=off, tunable work_mem / maintenance_work_mem.
- Range + phase controls (BACKFILL_START_ID/END_ID/SKIP_CARDINALITY/ONLY_FINALIZE)
to run disjoint id-range workers in parallel when the disk isn't the bottleneck.
- Percent-complete + ETA in the progress log.
- Finalize builds the two new indexes plain by default (faster in a quiet
window) with a CONCURRENTLY toggle.
The real speed-up in prod was dropping the 3 secondary indexes for the backfill
then rebuilding — the table is packed (fillfactor 100) so updates can't go HOT
and otherwise maintain every index per row. Index drop/rebuild + vacuum are kept
as standalone SQL so ops controls exactly when the app loses/regains them:
- sql/drop-recommendation-backfill-indexes.sql
- sql/rebuild-recommendation-backfill-indexes.sql (1GB maintenance_work_mem — safe
on the 4GB instance)
- sql/vacuum-recommendation.sql
run-sql.ts runs .sql files statement-by-statement, autocommitted (so VACUUM /
CREATE INDEX CONCURRENTLY work), for environments without the psql CLI. Wired up
as npm run db:run-sql and backfill:recommendation-restore.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
