The switch was the wrong control. project_workstream_contractor stores both
flags per (workstream, organisation) pair, so one switch stood for n booleans:
it could report a per-workstream split but never author one, and clicking it
destroyed the split to stay decidable. The schema's central capability was
unreachable from the only screen that configures it.
Add/edit dialog is now a grid: one row per workstream the *project* covers —
not only the ones this contractor holds — with a tick to assign and a tick per
permission. Unassigned rows stay visible and greyed, so the shape of the
project and its uncovered workstreams are part of the decision. Un-ticking a
workstream clears its permissions, so re-ticking never silently restores a
grant. Column headers carry an all/none shortcut.
The main table no longer shows permissions at all: contractor, workstreams,
Actions. Clicking a row (or its disclosure button, which carries
aria-expanded/aria-controls for keyboards and screen readers) opens a sub-row
listing each assignment with its own two permissions, read-only. Several may
be open at once. Actions stay per contractor — Edit and Remove act on the
whole set.
Wire shape follows: POST/PATCH take assignments[{projectWorkstreamId,
updateStagesPermission, uploadDocumentsPermission}]. A workstream named twice
is a 400 — two answers to one question. ContractorGroup drops its
per-contractor summary and permissionState is gone; nothing needs the flags
reduced to one answer, and reducing them is how a contractor allowed on one
workstream comes to look allowed everywhere. Updates are grouped by flag pair,
so twenty workstreams still cost at most four UPDATEs.
ADR-0022 records this; ADR-0021's decision 1 is marked superseded, its other
two (removal never cascades; Continue leads to the import) stand.
The screen at /projects/[projectId]/setup/contractors — the wireframe's
06-contractors-permissions.html, one table row per contractor organisation
with its assigned workstreams and the two permission switches.
Revisitable, like the setup steps before it: step 5 of the first-run wizard
and the screen the hub (#444) links to afterwards, with ?from=hub the only
difference (it swaps Continue for a return to the hub). Assignments are read
back from the database on every visit, and every change persists immediately.
- Add contractor: searches *existing* organisations (creating one belongs to
the invite-company flow), multi-selects workstreams, sets both permissions.
- Edit and remove per contractor, reacting to the API's 409s — a warning the
user may confirm, or a refusal they cannot.
- Continue leads to the work-order import (#414), the wizard's terminal step,
with the readiness helper — not a re-derived rule — rendering what is still
outstanding rather than a disabled button that can't say why.
No useEffect: the form dialog is keyed and mounted only while open, so its
useState initialisers are the load step. router.refresh() alongside the query
invalidation keeps the readiness panel honest as the last gap is filled.
ADR-0021 records why permissions are per-assignment but shown per-contractor
(with a mixed state), and why a removal never cascades through work orders.
Step 2 of 3's screen, on top of the pure logic in the previous commit. Follows
wireframes 08 (value mapping locked) and 09 (unlocked). Writes nothing.
Where the step lives
Client state inside ImportUpload, not a /import/mapping route: the parsed rows
live in that component (#414), so a URL step would drop them and force a
re-upload on every Back. "Continue to mapping" — disabled since #414 —
now opens the mapping screen.
ImportMapping — the screen
Owns the two mapping layers and nothing else. Statuses, counts, which values
need review and whether Continue is enabled are all derived inline from
(file, columnMapping, valueMapping, targets) on each render, so re-pointing the
workstream column immediately re-groups the contractor and stage tables — they
were never a separate copy of the answer. No useEffect, no useMemo.
ColumnMappingTable — file column -> Ara field
One row per file column with the wireframe's matched / needs-review
affordances, an "e.g. …" sample from the preview, and a shadcn Select per row
offering every field plus "Don't import this column". Picking a field another
column already holds says so in the option hint rather than moving it
silently. Beneath the table, the requirement blockers: which required fields
are still unmapped and why. Unmapped *optional* fields are stated, never
blocking.
ValueMappingTables — values -> the project's own entities
Three tables. Workstream values map flat; contractor and stage values are
grouped under the workstream they are scoped to, because a contractor
assignment and a stage ladder belong to a workstream (ADR-0019) — the same
"Acme" under two workstreams is two questions, and each picker offers only
that workstream's assignments. Row counts per value, auto-matched marked as
such. Notes carry the cases with no row of their own: rows waiting on their
workstream mapping, blank contractors (an error downstream), and blank stages
(not an error — they start at the workstream's first stage).
Gating (AC)
The value tables are locked, exactly as wireframe 08 shows, until every
required column is mapped — there is nothing to list until we know which
column holds the workstream. Continue to validation is disabled until the same
condition holds, and names what is missing. Unmapped *values* never block: they
surface as "N of M rows will fail validation as mapped".
page.tsx
Loads the value-mapping targets server-side alongside readiness (read-only, and
skipped entirely for a gated project, which has nothing to map to). Container
widened to max-w-5xl for the mapping tables; the eyebrow now covers steps 1–2.
ADR-0020 — records that the import session, and the mapping persisted alongside
it, are client-held. Persisting the mapping without the rows buys nothing: the
rows are client-held too, so a refresh loses them either way and a restored
mapping has no file to apply to. ImportSession is JSON-clean so a future
server-side session can store it unchanged; the ADR names moving the file
server-side as the thing that would change the answer. It also records the
production-database constraint honestly, as a constraint and not as the reason.
Stepping back to the drop zone and forward again keeps the mapping: the session
is held above the mapping component, so that component may unmount freely.
225 tests pass across src/lib/projects; tsc and eslint clean. No component tests
— this repo runs vitest in a node environment with no DOM setup.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extends the #414 upload-and-parse work (does not rewrite it) with a per-row
contractor, an optional stage, and a setup-completeness gate on the import
page. New ADR-0019 records the design. Still writes nothing.
CHANGE 1 — template + parser expected fields
Final column order: Property identifier, Workstream, Contractor, Stage,
Work order reference (optional), Forecast end date (optional). Contractor is
required; Stage is optional (blank falls back to the workstream's first
stage at commit). buildImportTemplateCsv, its worked example and the field
docs are updated. The raw parser stays header-agnostic — required/optional is
enforced in mapping (#415) and validation/commit (#417), not here — with a
test proving a file missing the required Contractor column still parses.
CHANGE 2 — setup-completeness gate (the substantive feature)
src/lib/projects/import/readiness.ts: a pure computeImportReadiness(facts)
plus a read-only loadImportReadiness(projectId) round trip. Rule: every
project_workstream must have >=1 project_workstream_stage AND >=1
project_workstream_contractor, and the project must have >=1 workstream.
Evidence requirements (#412) are advisory and excluded from the gate. The
helper returns per-workstream detail (which are incomplete, and why) and is
built to be reused by #413's final wizard step — noted in a comment.
The import page loads readiness and, when not ready, renders an
ImportSetupIncomplete state (lists the incomplete workstreams with the reason,
links to project setup) instead of the drop zone — not a 404, not a silent
empty zone. When ready, the drop zone renders as before. The gate applies on
every entry: the route stays reachable standalone, and a comment notes #413
will also route the wizard into it (that wiring is out of scope here).
CHANGE 3 — ADR-0019
Records: import is the final, setup-gated wizard step; contractor is supplied
per row and must match a contractor already assigned to the workstream
(contractor creation is out of scope — the existing "invite company to Ara"
flow owns it); stage is optional per row with first-stage fallback. States
the trade-off vs the replaced #417 design, which derived the contractor from
the workstream's single assignee — abandoned because
project_workstream_contractor is a collection, so one workstream's properties
can be split across contractors, which a single-assignee lookup cannot express.
Notes on the readiness query
count(distinct project_workstream_stage.id) / (…contractor.id) over two left
joins — the distinct is load-bearing because joining both children at once
makes a stages x contractors cartesian product a plain count(*) would
over-count. count(distinct) returns a bigint that node-postgres hands back as
a string, so each count is coerced with Number(); a loader test drives a
mocked db to prove that coercion.
Production-DB safety
No migrations, seeds or writes. loadImportReadiness is read-only SELECTs and
is not executed here. Unit tests mock @/app/db/db (vi.mock) and open no
connection; the pure readiness logic is tested with in-memory fixtures.
Full suite: 575 passing. tsc and eslint clean. authz modules and
src/app/projects/page.tsx untouched (other branches own them).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Dropping `<input type="date">` for the dd/mm/yyyy mask also dropped its
calendar, leaving typing as the only way to enter a date. ADR-0019 accepted
that cost but named the remedy: add a picker *behind* `DateInput`, writing the
same ISO value, rather than revert to the native control. That was felt
immediately in use, so this does it.
`DateInput` now renders a calendar button inside the field's right edge and a
Calendar in a Radix popover. Typing and picking are equal routes to the same
value — picking fills the text, typing moves the grid — so neither is
privileged and the two cannot disagree. The trigger is `type="button"` so it
cannot submit the form, and carries an aria-label, a bare icon having no
accessible name.
The calendar wrapper is written by hand rather than taken from the shadcn
registry: the published template targets react-day-picker v8/v9, whose
`classNames` keys differ from the `UI` enum v10 uses. It defaults to the enGB
locale, which carries `weekStartsOn: 1`, so the grid starts on Monday without
being told to.
The delicate part is the `Date` boundary. A calendar works in `Date`s, and
both obvious conversions are wrong: `new Date("2026-03-01")` is midnight UTC,
which is 29 February west of Greenwich, and `toISOString()` shifts the day back
the other way. So `isoToLocalDate` and `localDateToIso` build and read local
parts, and they are the only two functions in `src/utils/dates.ts` that touch
`Date` at all — everything else stays string-only. The round trip is tested
across a full month rather than at a sample day, because the failure only
appears on some days in some zones; the suite was also run under
TZ=America/Los_Angeles and TZ=Pacific/Auckland.
Dependencies: react-day-picker and date-fns, the latter already present
transitively and now direct at v4. `@tremor/react` bundles react-day-picker v8
nested and npm kept it isolated on date-fns v3, so nothing existing changed
version; no Tremor DatePicker is used anywhere in src, so there is no second
copy in the bundle. The alternative was a hand-rolled month grid with no
dependency, rejected because keyboard navigation and ARIA on a calendar are
easy to get subtly wrong and not worth owning.
ADR-0019's "the native calendar picker is gone" consequence was now false and
has been rewritten.
TESTS: tsc --noEmit and ESLint clean; vitest 582 passing (53 files), 24 of them
for the date helpers. NOT verified in a browser — this project has no jsdom or
testing-library, and `next build` was not run because it would statically
render pages against a DATABASE_URL still pointing at production. The new
Cypress case, which picks 17 March from the grid and expects 17/03/2026 in the
field, is unrun like the rest of that spec.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The create-project form's date fields were `<input type="date">`, whose
display format follows the *browser's* locale rather than the document's. On a
US-configured browser they rendered mm/dd/yyyy inside an app that renders
en-GB everywhere else (~40 call sites). That is not cosmetic: 03/09/2026 is a
valid date under both readings, so a user cannot tell which one the box took
and no validation can catch it. The failure mode is silently wrong data, not
an error message. No attribute, `lang` value or CSS overrides the native
control, so the control had to go.
`DateInput` replaces it: a text field that masks to dd/mm/yyyy as you type,
whose value and onChange remain YYYY-MM-DD. The display format therefore stops
at the input's edge — the schema, the request body and the Drizzle `date`
column are untouched, and dates keep sorting lexicographically with no
timezone in play. The cost is the native calendar picker, which ADR-0019
argues is the right way round: a picker is a convenience, an ambiguous entry
format is a correctness problem. If one is wanted later it belongs behind
`DateInput`, not in a revert to the native control.
The parsing lives in `src/utils/dates.ts`, pure and Date-free — `new
Date("2026-03-01")` is midnight UTC, which is the previous day in a
negative-offset zone, so this compares and builds strings instead. Impossible
days are rejected explicitly because `Date` would roll 31/02 forward into
March rather than refuse it.
Two behaviours worth knowing, both covered by tests. A *finished* but
impossible date is handed to the schema as typed rather than reported as
empty: a field visibly holding 31/02/2026 while an optional-date schema reads
it as unfilled would tell the user nothing. Unfinished input does report
empty, so no error fires mid-keystroke. And the mask only re-adds a separator
while characters are being added — doing it during a backspace would make the
slash undeletable.
`DateInput` adjusts state during render rather than in a `useEffect` (the
documented alternative) to resync when the form sets the field from outside.
Per CLAUDE.md's utils split, dates.ts is generic and lives in the new
`src/utils/`; `src/app/utils.ts` stays for company-specific helpers (EPC
bands, SAP points) and `src/lib/utils.ts` is vendored Tremor/shadcn.
CLAUDE.md records the rule and the "never `<input type=\"date\">`"
prohibition so the next agent finds it before writing the wrong thing.
Five other date inputs remain wrong under this ADR — PibiSection,
PibiDatesEditor and PropertyFilters — left alone as unrelated to this branch
and recorded in ADR-0019 as convert-when-next-touched.
TESTS: tsc --noEmit and ESLint clean; vitest 576 passing (53 files), 18 of
them new for the date helpers. Cypress was NOT run — this environment's
DATABASE_URL still points at production. The spec's four date entries were
converted to day-first, an assertion added that 01/03/2026 crosses the wire as
2026-03-01, and a new case covers the mask and the impossible-date refusal;
all three are unverified.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Records that Ara Projects supersedes the HubSpot Live projects feature
long-term (coexisting for now, no v1 migration) and reserves the top-level
/projects namespace. Adds the canonical Ara Projects glossary to CONTEXT.md,
matching src/app/db/schema/projects/schema.md, and qualifies the ambiguous
'Project' references (HubSpot Project vs Ara Project).
Closes#405
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Static Stitch export previously only available locally; referenced by the
Ara Projects issues (#405-#429). README explains which parts are spec vs
placeholder.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- CONTEXT.md: add the Scenario overlay glossary term under Reporting.
- ADR-0015: the Scenario overlay is one read model behind both metrics endpoints.
- ADR-0016: the SAP↔EPC band threshold ladder has a single owner (@/lib/epc).
- ADR-0017: Likely downgrade is band-movement only; the SAP-05 signal is retired.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
"Homes Without an EPC" and "Expired EPCs" were both derived from estimatedSql,
which asks "was this picture gap-filled?". That conflates two different questions:
provenance (was it modelled?) and coverage (does a certificate exist?).
They used to agree, because a gap-fill implied no certificate. Model ADR-0054 ends
that: the gov EPC API only serves certs registered since 2012, so a pre-2012
dwelling is gap-filled even though a real, stale certificate exists — and gets the
new source='expired'. On portfolio 824, 375 of the 403 "without an EPC" homes have
exactly that: a real certificate, just an out-of-date one. A different, and far more
actionable, problem than never having been certified.
Two defects fell out of the same conflation:
- Every query matched source='predicted' literally, and `source` is plain text
with no enum or CHECK. An 'expired' row matches neither the lodged nor the
predicted alias, so the home would have read as having a VALID CURRENT cert —
silently gone from both cards, no Estimated badge, nothing to catch it.
- isExpiredSql read epl.registration_date alone, but 714 of 824's 4,873 lodged
certs carry only inspection_date. All 714 counted as current; 170 are in fact
>10 years old. lodgementDateSql already coalesced the two; expiry didn't.
The cards now partition on coverage: withoutEpcSql = no lodged AND no historical
record; isExpiredSql = a cert of either kind, out of date. estimatedSql keeps its
own meaning and spans the whole predicted slot, so an expired home is estimated AND
has an EPC — both true. The `AND estimated = false` guard is gone from the Expired
card: an expired home IS estimated, so it excluded the very homes being counted.
Legacy portfolios are untouched (632: 8/1662 before and after). 824 moves to
913 → 1,083 expired immediately (the inspection_date fix), then to 25 / 1,461 once
Model writes expired rows. Safe to deploy BEFORE Model — the predicates just find
none. The reverse was not safe, which is why this goes first.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Backend, end to end (ADR-0013):
- getProperties now returns each property's tags (name-ordered JSON); the
PropertyWithRelations type gains a `tags: PropertyTag[]` field.
- A "tags" property-table filter (EXISTS/NOT-EXISTS on property_tag, any-of +
an "Untagged" bucket) — PK-indexed by property_id, so no join added.
- getFilteredPropertyIds resolves the whole matching set for filter-driven Bulk
tag assignment; the assignments route's `filter` mode uses it.
- Run filter: previewModellingRun joins property_tag for tag_ids; the two
modelling routes accept tagIds. Distributor ask documented in
docs/backend-asks/tag-ids-run-filter.md.
UI:
- Settings → Tags: create (name + colour), edit, delete-with-count-confirm,
list with property counts (TanStack v4). Sidebar link added.
Typecheck clean; 431 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Captures the grill-with-docs outcome: flat app-owned Tags, four assignment
paths (incl. in-app bulk upload, no FastAPI), tags as a Run-filter key extending
ADR-0008, hard-delete + cascade, reporting-by-tag deferred to the reporting
redesign. Design only — no implementation yet.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Records the decision to make the portfolio property-list override-aware for
Property Type / Built Form / Construction Age / Main Fuel, extending ADR-0008's
override → EPC-derived → Unknown precedence to a second surface via shared
epcSources fragments. Adds a CONTEXT.md Flagged-ambiguities note.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
7 Leyton Avenue (UPRN 100031252876) came back from OS Places as RH03
("HMO Not Further Divided") and rendered as "Non-residential", so it
couldn't be added from the postcode-search "add properties" journey.
classifyOsCode decided "residential?" purely on an RD prefix, dropping the
whole RH (HMO) family — even though RD07 (the RD-family HMO code) was
already mapped to House. Extend the residential check to RH and map
RH01/RH02/RH03 to House (built form unknowable), consistent with RD07.
Updates ADR-0007's enumerated mapping table to stay truthful.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add bulk document download as a row-selection flow on the Live Reporting
Documents tab: a "Bulk download" button flips the existing DocumentTable
into selection mode (per-row tickers + select-all across all filtered
pages); "Download selected" zips those properties' documents and emails a
link (shown in-app while the page stays open). Scoped by the tab's
existing Project/group selector.
Two-step trigger (ADR-0008 convention): insert an app-owned tasks row
(task_source "app:bulk_document_download", selection config JSON in
tasks.inputs, source_id = portfolio) then dispatch only { task_id } to
POST /v1/documents/bulk-download. Documents are matched by
landlord_property_id, so the ticked ids ARE the selection — a property
with no landlord id has no documents and is un-tickable.
Resilience: the client now parses responses defensively (a platform 504's
HTML body no longer surfaces as "Unexpected token 'A'…"), and the POST
route sets maxDuration=60 so the synchronous backend resolution has
headroom before the browser gets a 504.
Docs: CONTEXT.md gains Project + Bulk document download; ADR-0011 records
why the marker lives in task_source (worker re-reads the task) vs service.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Gross/Net cost, Project delivery, Current stock, Likely downgrade/upgrade
(band movement) added to CONTEXT.md from the reporting-redesign grilling
session. ADR-0010 records the compliance-window filter as a report-view
parameter, not a Scenario constraint. Also commits the previously
untracked ADR-0001/0002 docs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Design references and critique snapshot for the portfolio home redesign;
impeccable design skill added to devcontainer postCreateCommand on trial.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- The run's config now lives on the task row's new `inputs` column
(migration PR #361) instead of a config subtask — sub_tasks belong
entirely to the distributor's execution work.
- Distributor path corrected to /v1/modelling/trigger-run.
- Sub-task granularity is batches, not properties: history shows progress
as a percentage under the status chip; the Plans column always shows the
plan count (batch counts are never comparable to property/plan numbers).
- ADR-0008 amended accordingly.
Live verification of the task write waits for PR #361 to be applied.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
previewModellingRun now resolves Run filters in one grouped SQL statement
(override snapshot -> EPC-derived via propertyTypeSql/new builtFormTypeSql
-> Unknown) and intersects the matched set with plan for per-scenario
already-modelled counts. The Model-backend ask shrinks to the distributor
alone; the app's preview is the reference implementation of the resolution
rule (ADR-0008 amended).
Verified live against 814: all=340 matched / 338 already modelled in
scenario 1271; B93 8SU=2 resolving House/Detached through override
snapshots; Bungalow/Unknown=0.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Amends ADR-0008 after challenge: the Modelling run reuses the BulkUpload
trigger convention (app inserts the tasks row + config subtask with inputs
JSON, passes task_id to the distributor) instead of a dedicated
modelling_run table. No migration; status and progress come from the task
system by construction; dispatch failure marks the task failed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The dispatch confirmation sets a time expectation only; the email promise
stays out of the UI until a completion notification exists (run_id -> task
correlation provides the signal to build it on).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Grilling-session outcomes for the bulk-trigger-modelling feature: the
ModellingRun/Run-filter glossary entries and the decision record — insert-only
run rows, filters (not ids) to the distributor lambda, backend-owned filter
resolution with a shared-code-path count endpoint, run_id correlation into
task records for derived in-flight state, warn-don't-block concurrency.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Order address candidates naturally so Flat 2 sorts before Flat 10
(TDD'd in the domain core; numeric locale compare).
- Disable spell check / autocomplete / autocorrect on the postcode input.
- Long result lists and the tray's postcode chips now scroll inside
their containers instead of growing the page.
- gitignore docs/wip/** and untrack the postcode-search handover doc.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The docs/adr/** ignore rule silently ate ADRs 0004-0006 that CONTEXT.md
still references. Remove it so decisions actually land in the repo.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Users could only create scenarios buried inside the CSV-upload modal, at
trigger time. This adds a first-class Scenarios tab: a gallery of the
portfolio's scenarios, a read-only detail page, and a 3-step creation
journey (goal → measures → review) that saves app-authored scenario rows
per ADR-0003.
Domain rules (TDD'd in src/lib/scenarios/model.test.ts, 19 tests):
- goal semantics: "Increasing EPC" requires a target band; all other
goals are maximise-within-budget with goal_value forced null
- exclusions-only measure model: normalised (dedupe+sort), unknown keys
rejected, all-excluded rejected, empty set valid; serialised to the
backend's brace text format (parses the legacy space-padded variant)
- insert mapping: multi_plan=true, is_default=false, budget per property
- exact-config duplicate detection ignoring name (review-step warning)
- status derived from plan existence — no stored status column
Queries follow the ADR's discipline: gallery status = one grouped
COUNT(DISTINCT property_id) per portfolio; detail = one scoped count
(index-only, powers the blocked-delete message); delete = single atomic
DELETE guarded on NOT EXISTS(plans). Rename is the only other mutation —
configuration is immutable.
Verified live on portfolio 814: create (400 on missing band, 201 on CO₂
draft), derived Awaiting/Modelled in the gallery, rename, delete of the
draft (200) and blocked delete of the modelled scenario (409, "338
properties…" matching the portfolio's true count).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Overhaul the recommendation plan_id/material_* backfill for the 21M-row prod
table (see docs/runbooks/recommendation-denormalization-backfill.md):
- Single combined pass: set plan_id + all four material_* columns in one write
per row (was two full passes, rewriting every row twice). Guard + COALESCE keep
it idempotent and resumable — done rows are never rewritten.
- Session tuning: synchronous_commit=off, tunable work_mem / maintenance_work_mem.
- Range + phase controls (BACKFILL_START_ID/END_ID/SKIP_CARDINALITY/ONLY_FINALIZE)
to run disjoint id-range workers in parallel when the disk isn't the bottleneck.
- Percent-complete + ETA in the progress log.
- Finalize builds the two new indexes plain by default (faster in a quiet
window) with a CONCURRENTLY toggle.
The real speed-up in prod was dropping the 3 secondary indexes for the backfill
then rebuilding — the table is packed (fillfactor 100) so updates can't go HOT
and otherwise maintain every index per row. Index drop/rebuild + vacuum are kept
as standalone SQL so ops controls exactly when the app loses/regains them:
- sql/drop-recommendation-backfill-indexes.sql
- sql/rebuild-recommendation-backfill-indexes.sql (1GB maintenance_work_mem — safe
on the 4GB instance)
- sql/vacuum-recommendation.sql
run-sql.ts runs .sql files statement-by-statement, autocommitted (so VACUUM /
CREATE INDEX CONCURRENTLY work), for environments without the psql CLI. Wired up
as npm run db:run-sql and backfill:recommendation-restore.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Self-contained brief to start a fresh context implementing the per-Property fact
layer in the bulk_upload_finaliser: locked decisions, the four-input assembly,
the two open hazards (classifier-CSV↔combiner-output join key; property_id for
no-UPRN rows), candidate architectures, and a first-steps list. v1 (async
finalise writing `property`) is shipped and working.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the multiEntryOrdering jsonb column + interactive order picker: the
largest-count multi-entry sample is shown with a building-part dropdown per
file position (one Main building + Extensions), validated as a permutation.
A PATCH route persists { count: permutation } + confirmed, and Finalise is
disabled until the ordering is confirmed when the upload is multi-entry.
Migration for the new column is intentionally not included here — generated
after origin/main is merged (its multi_entry_summary migration lands first).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wires the X-SES-CONFIGURATION-SET header on outbound auth emails so SES
bounce/delivery events flow through dev-ses-config to the dev-ses-events
SNS topic. Replaces the fire-and-forget "EMAIL MAGIC LINK SENT" log
(which fired before the SMTP transaction and swallowed downstream errors)
with structured EMAIL_MAGIC_LINK_SUCCESS/_FAILURE logs carrying the
Nodemailer messageId, so app-side sends are now correlatable with SES
events.
Motivated by the Atkins / Sustainable Building UK silent-quarantine
incidents where we couldn't tell whether SES had even tried to send.
Plan doc at docs/wip/auth-email-code-fallback-plan.md tracks the
broader email-code-fallback design that PR 2 will implement.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
