The Model repo reads both fields from the gov-EPC API but neither had a
column here, so both were mapped and then silently discarded on save --
inert for any dwelling read back from Postgres.
rafter_insulation_thickness: property 749719 lodges "250mm", which bills
the roof on RdSAP 10 Table 16 col (2) at U 0.23. Dropped on the DB
round-trip it falls to the Table 18 col (2) unknown-thickness default of
2.30 -- the spec's *uninsulated* value, a 10x error:
quantity | lodged | dropped | with field
SAP | 73 (band C) | 63.22 | 72.65
primary energy kWh/m2/yr | 212 | 327.9 | 212.3
CO2 t/yr | 1.5 | 2.40 | 1.54
It is jsonb, not text, to match the sibling thickness columns: the value
is Union[str, int] -- "250mm" from the gov API, int mm from Site Notes --
and the Python round-trip test asserts int-vs-str survives exactly.
wall_is_basement: not a lodged U-value -- it selects WHICH RdSAP default
applies (RdSAP 10 5.17 / Table 23). Nullable is load-bearing, so it is
deliberately NOT notNull().default(false). Three states are distinct:
null "not stated" falls back to the gov-API code-6 heuristic, false
"explicitly system-built" suppresses it, true is a basement wall. Code 6
is canonically system-built and the Elmhurst site-notes mapper sets false
precisely to defeat the heuristic, so collapsing false to null INVERTS
the determination -- a system-built wall bills as a basement wall and,
via has_basement, drags the whole ground floor onto the Table 23
basement-floor U-value. Rare (54 of 67k building parts in the 2026
sweep) but silent and directional.
Deliberately NOT persisted: wall_u_value, roof_u_value, floor_u_value.
Those are full-SAP artefacts. We re-model every dwelling *as* an RdSAP
assessment, which derives U-values from the construction-default cascade
rather than honouring an assessor's measured value, so persisting them
would make the re-model a hybrid. Corpus check across 1,000
RdSAP-21.0.1 certs: roof_u_value 0, floor_u_value 0, wall_u_value
4 (0.4%).
Both columns nullable, additive, no backfill -- neither ever existed, so
there is nothing to restore. Affected properties must be re-ingested from
the gov-EPC API before their scores move.
Deploy ordering: this migration must land BEFORE the Model-side backend
deploys. The Python SQLModel now selects both columns, so without them
every EPC read fails outright with "column
epc_building_part.rafter_insulation_thickness does not exist" -- not a
graceful degradation.
Refs Hestia-Homes/Model#1661
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reorganises the authz lib along DDD lines by separating the persistence
boundary from the domain:
src/lib/projects/authz.ts domain — pure decisions
src/app/repositories/projects/authzRepository.ts persistence — Drizzle
The repository sits alongside src/app/db/ and mirrors the existing
db/schema/projects/ layout, so schema and repository stay symmetrical as
Ara Projects grows more of both.
Dependencies point one way: the repository imports the domain's fact types
and returns them, so the domain never learns a table shape and continues to
import no database client. The 34 guard tests still run with no connection.
No behaviour change — the queries and guards are untouched. Note this is a
new pattern for the codebase: the prevailing convention is a colocated
queries.ts/server.ts per lib folder, which 88 files still follow.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Code review: /onboarding is *user* onboarding, not Portfolio onboarding.
It captures the user's details and their required privacy-policy
acceptance (plus marketing opt-in) and touches no Portfolio at all —
confirmed by the form schema in src/app/onboarding/page.tsx, which has
no portfolio coupling and makes acceptedPrivacy mandatory.
The exemption added in e34f166f was therefore wrong: it let contractor
users into the app without the consent we are required to capture. A
contractor can complete onboarding perfectly well, so nobody needs the
bypass.
Removes the /projects exemption and isProjectsPath(). /projects stays in
the matcher, so the tree is still auth-guarded. Unit and Cypress tests
inverted to assert the corrected behaviour, plus a new case covering an
onboarded contractor reaching /projects.
Refs #406
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds migration 0274, a data-only custom SQL migration seeding the two
reference tables the setup wizard needs (#407):
- project_type: Retrofit, Planned Maintenance
- workstream: the canonical 8 from the UX wireframes, each with a
one-line description (the column is NOT NULL)
Note that #407 specifies the second project type as "New Build"; PR review
superseded that with "Planned Maintenance". The ticket text is stale, not
the code.
Idempotency uses INSERT ... SELECT ... WHERE NOT EXISTS rather than
ON CONFLICT DO NOTHING, because neither table has a unique constraint on
`name` — an ON CONFLICT (name) form would fail at runtime with "no unique
or exclusion constraint matching the ON CONFLICT specification". Re-running
is a no-op and leaves existing rows, including edited descriptions, alone.
The 0274 snapshot is a copy of 0273 with chained id/prevId, matching how
the previous data-only migration (0227) was recorded.
No UI for creating workstreams — deferred to #428.
UNVERIFIED AGAINST A LIVE DATABASE. This environment's DB credentials point
at production, so the migration was deliberately not executed. It was
verified statically only: both statements parse against the PostgreSQL
grammar (libpg_query), the target tables and columns were cross-checked
against the 0273 schema snapshot, every NOT NULL column without a default
is supplied, and the seeded values match the intended set exactly. A human
needs to run it against a non-production database and confirm both the
fresh-DB and re-run cases before it ships.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Ara Projects gets its own top-level route tree rather than living under
the /portfolio/[slug] chrome: Projects are organisation-scoped
(project.organisation_id) and contractor users from external orgs will
work in these screens without ever having a Portfolio.
- src/app/projects/layout.tsx: shell + top-level nav
- src/app/projects/page.tsx: list stub (real list lands with #408's
visibility rule)
- src/app/projects/[projectId]/{layout,page}.tsx: per-project nav
(Dashboard, Work orders, Import, Settings) + dashboard shell
- ProjectsNav: the Toolbar.tsx shadcn navigation-menu pattern, taking
its items as a prop so both tiers share one implementation
Middleware: adds /projects/:path* to the matcher and exempts the tree
from the portfolio onboarding redirect. A contractor is never
`onboarded`, so without the exemption they would be bounced to
/onboarding on every request with no way out. The decision is split into
a pure routeAuthenticatedRequest() so the rule is unit-testable without
minting a JWT.
Authorization is a thin seam in src/app/projects/authz.ts with TODOs
referencing #408 — the shell deliberately does not inline permission
logic, and user->organisation resolution does not exist yet either.
Tests: 8 unit tests over the middleware routing rules; Cypress smoke
test for the unauthenticated redirect, the authenticated render, the
contractor path, and the per-project shell.
Refs #406
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds src/lib/projects/authz.ts as the single source of permission truth
for every Ara Projects route handler — no route re-implements this inline.
Ara Projects are organisation-scoped, so none of the portfolio-scoped
team_portfolio_permissions machinery applies. Membership resolves through
team_members -> team -> org_id instead.
Role resolution is most-privileged-first: internal (@domna.homes) >
client (member of the owning org) > contractor (member of an org assigned
to any of the project's workstreams) > no access. project.domna_admin_access
gates the internal role only — with it off, a Domna user falls through to
whatever their own memberships earn them rather than losing access outright.
Client outranks contractor when one org is both owner and deliverer.
The two work-order guards check the permission flag on the *assignment*
rather than on the organisation: a contractor needs membership of that
work order's own assigned org plus the relevant flag, so holding a
permissive assignment on a sibling workstream grants nothing.
Structure follows the repo's model/queries split — authz.ts is pure and
imports no database client, so all 34 guard tests run on in-memory
fixtures with no connection. DB-backed fact loaders (getUserOrganisations
and friends) live in authzQueries.ts.
Note the guard signatures take the project facts explicitly —
canUpdateStage(user, project, workOrder) rather than the (user, workOrder)
in the ticket — since the role can only be resolved against the project.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
majorConditionDeals was computed in computeLiveTrackerData and threaded through
LiveTrackerProps → LiveTracker → AnalyticsView, but never rendered — the
intended "Awaab's Law card" was never built. With the rendered "Flagged at
Survey" count now keyed on the same hasMajorConditionIssue signal, this set is
redundant. Removed the const, the LiveTrackerProps field, the prop plumbing in
LiveTracker + AnalyticsView, and its two tests. hasMajorConditionIssue stays
(used by the survey-flag count).
Typecheck + lint clean; live-tracker tests (64) + full suite (500) green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
24 Charlesworth Street (deal 507693469922) has a major condition issue recorded
(major_condition_issue_description = "Damp in bathroom") but didn't appear in the
live tracker's condition-issue surfaces. Two causes, both now keyed on the issue
itself via a shared hasMajorConditionIssue(deal) predicate:
- majorConditionDeals filtered on dealstage === "3061261536" — a HubSpot stage
that was retired, so the set was always empty. (This set is also not yet
rendered in any card — noted for follow-up.)
- The rendered "Damp, Mould & Other Condition Issues → Flagged at Survey" count
(computeDampMouldRisk.surveyFlagDeals) keyed on major_condition_issue_evidence_s3_url,
which is null for this deal (its photos are in the raw HubSpot field). This is
the count the property was actually missing from.
There is no yes/no column; the description is the "Yes" signal (only filled when
an issue exists). Removed the now-dead MAJOR_CONDITION_STAGE_ID. TDD:
red→green→refactor; 66 live-tracker tests, full suite (502) + typecheck + lint green.
Also gitignore .sandcastle/ (untracked agent worktree scratch).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Money formatters: model.ts owns the single formatMoneyCompact + moneyFull;
components/primitives re-exports them (moneyCompact = formatMoneyCompact) and
the PDF's local money() is gone.
- ReportView: viewState.ts is the single definition; ScenarioCombobox re-exports
it instead of re-declaring.
- EPC_BANDS: the reporting API routes import the canonical @/lib/epc/bands
rather than the scenarios-model copy.
- compare/page uses countBelowBand(toBandCounts(...), "C") instead of a
hardcoded ["D","E","F","G"] reduce.
Typecheck + 95 reporting/epc tests green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Likely downgrade had two live definitions over near-disjoint populations: the
confidence strip used the legacy SAP-05 point comparison (legacy homes only),
while the data-quality page and drill-down used band movement (which yields 0
for legacy homes, whose lodged and effective band are the same column). So the
two surfaces could disagree for the same portfolio.
CONTEXT.md already calls the point-level signal "retired". This makes band
movement the single definition (lodged band vs effective band; real
certificates only):
- New shared SQL fragments likelyDowngradeSql / likelyUpgradeSql in
epcSources.ts — the SQL twin of model.classifyBandMovement.
- server.getLikelyDowngrades now counts band movement (was SAP-05, legacy-only),
so the confidence strip agrees with the data-quality page.
- getDataQualityMetrics and the drill-down route use the shared fragments.
Legacy portfolios now report likely-downgrades via band movement (0 where lodged
== effective), consistent everywhere. Typecheck + 86 reporting tests green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The block turning a ScenarioOverlay + current-stock totals into the ledger view
(carbonSaved, billSavings, cost-per-SAP, cost-per-carbon) was hand-written three
times — ReportingClientArea, the PDF page, and (partially) the Compare table —
where the ratios could silently disagree.
- model.ts gains deriveLedgerView() plus the atomic helpers amountSaved,
capitalOutlay, costPerSapPoint, costPerCarbonSaved. LedgerView moves here too
(re-exported from InvestmentLedger for existing imports).
- ReportingClientArea and pdf/page now call deriveLedgerView; their overlay
fetches are typed as ScenarioOverlay (candidate 3 wiring).
- Compare reuses the atomic helpers, keeping its null-for-"—" rendering.
- model.test.ts covers the derivation and its divide-by-zero guards.
Typecheck + 38 model tests green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The "After scenario" aggregate — the back half of Current → After — was
written twice: scenario/[scenarioId]/metrics (488 lines) and
scenario/default/metrics (316), differing only in plan scope. They drifted
independently (band thresholds, compliance-window predicate, ledger wiring all
duplicated) and sat outside the reporting folder.
Candidate 1+3+4 from the architecture review:
- New @/lib/reporting/overlay.ts owns all overlay SQL + shaping behind one
getScenarioOverlay(portfolioId, scope, filters, tags). The default
(recommended-plans) view is the scenario view with an inert filter set and no
EPC target, so it flows through the same code. Joins are now conditional on
what the active filters read, so an unfiltered overlay never joins the
lodged-certificate / legacy tables — strictly leaner than before. Dropped the
plans-only avg_*/total_* columns that were computed but never read.
- Both metrics routes are now thin adapters: parse+validate the request, call
the read model, return it (or 404).
- ScenarioOverlay is a single typed contract in ./model (client-safe), replacing
the untyped JSON re-typed across consumers. Dropped dead ScenarioOverlayMetrics
and MetricKey.
- Reporting reads leave the Next.js route folder for @/lib/reporting, matching
the repo's src/lib/<domain>/server.ts convention: databaseFunctions.ts +
measuresQuery.ts → server.ts; types.ts → types.ts; reportingScenarios.ts →
moved with its test. actions.ts stays as the "use server" seam.
- Deleted the dead reporting/utils.ts.
Behaviour-preserving (getLikelyDowngrades kept on the SAP-05 path here; retired
in the next commit). Typecheck + 89 reporting/epc tests green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 92/81/69/55/39/21 band floors were defined in five places across TS and
SQL — @/app/utils.sapToEpc, a hand-copied sapToEpcLetter in actions, the
EPC_TO_SAP_MIN/MAX ranges, and the BAND_BUCKETS_SQL/EPC_MIN_SAP in both
scenario-metrics routes. "What SAP score is band C" now lives once in
@/lib/epc/thresholds (BAND_MIN_SAP), with a SQL twin in thresholdsSql.
- lib/epc/thresholds.ts: BAND_MIN_SAP, sapToBand, EPC_TO_SAP_MIN/MAX (derived)
- lib/epc/thresholdsSql.ts: sapBandBucketsSql() built from the same ladder
- @/app/utils.sapToEpc + @/app/utils/epc now delegate to the canonical ladder
- deleted the hand-copied sapToEpcLetter in actions/recommendations.ts
- both metrics routes use sapBandBucketsSql / BAND_MIN_SAP
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The left column (EPC distribution + About box) and the right column
(Investment ledger) ended at slightly different heights. Make the two
report columns equal height (items-stretch) and let each column's bottom
box grow to fill — the About/Recommended brief and the ledger card both
flex-1 — so their bottom edges always align regardless of content length.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Post-PR structural changes to the Projects module schema (migration
regenerated; not yet applied):
- Delete the contractor table. Contractors are organisations:
project_workstream_contractor.contractor_id -> organisation_id
(uuid -> organisation).
- Rename project.client_id -> organisation_id (uuid -> organisation).
- uploaded_files: drop project_workstream_id and
project_workstream_contractor_id; keep
project_workstream_evidence_requirement_id; add nullable work_order_id
(evidence is per work order, not per project workstream).
- Rewire relations accordingly and regenerate migration 0273.
- Reconcile schema.md with all of the above.
Note: deleting contractor drops its phone_number / email_address, which
have no equivalent on organisation.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two requests in one: describe the scenario, and fill the blank space
under the EPC distribution (the left column was shorter than the ledger).
Both are solved by an "About this scenario" brief stacked beneath the EPC
distribution in the left column.
- New getScenarioConfig() reads the scenario's immutable config
(goal/target, housing type, budget, measure exclusions, fabric-first;
exclusions parsed from the `{a, b}` text column).
- The brief: a plain-language sentence ("what it takes to reach EPC C
across the portfolio's social homes…") plus a spec list (Goal, Stock,
Budget, Fabric-first, Excluded measures with human labels).
- Recommended plans gets a matching "About recommended plans" note.
- Tightened spacing so the fuller page still fits one A4 sheet.
Verified all three views (scenario / current stock / recommended) render
as one clean page.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The exported PDF split awkwardly — page 1 half-empty, the EPC + ledger
block pushed wholesale to page 2 — and rendered too wide. Cause: the
print CSS used `@page { margin: 0 }` plus a `transform: scale(0.94);
width: 106.4%` hack on .print-root, which renders wider than the sheet
and paginates badly (a scaled/oversized root can't break cleanly).
- Print CSS: real `@page { size: A4 portrait; margin: 12mm }`, drop the
transform-scale and width hack. (These classes are used only by the
report, so no other page is affected.)
- Constrain the report to A4 content width (max-w-190mm, centred) so it
can never render landscape/oversized, with on-screen padding that
collapses in print.
Result: cover + headline + KPI tiles + EPC distribution and Investment
ledger (side by side) + footer all fit one portrait page; current-stock
likewise. Verified by generating the real PDFs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The report PDF was effectively broken/unfinished: it crashed locally
(self-fetch to a hardcoded :3000), only worked for a numeric scenario
("No scenario selected" from Current stock / Recommended), ignored the
new URL-driven tag + scenario filters, and printed bare because @media
print stripped background colours — so it "looked rubbish".
Rebuild it as the true print register of the reporting screen:
- Carries the full view state (view + filters + tags) from the Download
report link, so the pack matches exactly what's on screen.
- Supports every view: Current stock (baseline-only), Recommended plans
(default segment) and any scenario; filters/tags threaded through.
- Polished, print-safe layout: branded cover with the portfolio name,
headline, KPI tiles, EPC distribution (div bars, not the button-based
ladder that print hides), and the restyled Investment ledger reused
verbatim. A "Filtered view" note when filters/tags are active.
- Fix colour printing (print-color-adjust: exact on .print-root) and
drop the app chrome (portfolio-name banner + copyright footer) in print.
- Robust base URL (respects PORT in dev).
Also fixes a real bug the Recommended report surfaced: the default
metrics route omitted gross_cost (→ "£NaN gross" in the ledger) and
computed net as construction−funding with a contingency-less per-home.
It now uses computeLedger like the [scenarioId] route — fixing the
on-screen Recommended ledger too.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The modelling engine emits costed plans for homes already at a scenario's
target EPC band (e.g. ~£300–600 of low-energy lighting / system tune-ups
on effective-C homes in a "Reach EPC C" scenario). Counting them inflated
"Homes upgraded" and its costs, and stopped the KPI reconciling with the
below-target band distribution (portfolio 796 / scenario 1268: 9492
upgraded vs 9039 homes below C).
Add an interim guard to the scenario metrics upgrade aggregate (count +
cost): for Increasing-EPC scenarios, drop any home whose effective band
already meets the target (effective band <= goal_value; lexical, A best).
Unknown/NULL effective band is kept. Verified against the DB: scenario
1268 goes 9492 → 8871 upgraded, −£282k of works on compliant homes.
This is a reporting-only band-aid; the root cause is in the engine —
tracked in Hestia-Homes/Model#1652. The default/recommended view has no
single target band, so the guard doesn't apply there.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Apply the new structured tooltip treatment to the scenario filters, so
their descriptions read like the cost figures': a summary line plus a
subtle footnote, in the same larger, rounded ⓘ card.
- Extract the tooltip body into a shared `tipContent.tsx` (TipBody +
TipSpec); costHelp now consumes it instead of its own copy.
- FilterChips: the two toggle chips use the shared `InfoDot`; the
"skip already-compliant" popover reuses the same structured copy.
Filter help now lives in one `FILTER_HELP` map.
Copy/present only — no filter behaviour changed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The production build's `react/jsx-key` lint failed on the HelpBody
elements built inside COST_HELP's Object.fromEntries map. Add a key —
they're constructed in an array literal even though they're stored in an
object, so the rule applies.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The cost explanations were one dense paragraph in a small (0.75rem)
tooltip. Restructure them: a one-line definition followed by scannable
Includes (green ✓) and Excludes (grey −) lists, with a subtle noted
footnote where useful. Bump the tooltip text to 0.82rem, widen and
re-pad it, and give it a softer rounded card look.
costHelp.ts → costHelp.tsx: entries are now structured specs
(summary / includes / excludes / note) rendered to tooltip bodies;
InfoDot and KpiBand's `tip` accept ReactNode. No copy meaning changed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Users couldn't tell what the cost values include or exclude — e.g. does
"gross per home" cover contingency, is the principal contractor's price
in there. Add plain-language "what's included / excluded" disclosure
across the scenario overlay, sourced from one canonical copy module
(costHelp.ts) so every surface says it the same way and matches the
domain glossary.
- New shared `InfoDot` primitive (the one ⓘ affordance).
- Investment ledger: ⓘ on Net cost, gross/home, Construction works,
Project delivery, Contingency, Bill savings, Carbon saved, and the two
value-for-money ratios — each stating what it does/doesn't cover
(e.g. construction = the principal contractor's works, excludes
delivery + contingency; gross = works + delivery + contingency).
- KPI band gains an optional `tip`; the "Gross cost /home" KPI uses it.
- "Where the money goes" now notes it's construction works only, so it
reconciles to the Construction works line, not gross cost.
Copy only — no figures or queries changed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The scenario Investment ledger was a flat monochrome list. Give it a
visual hierarchy while keeping the reporting design vocabulary (quiet
navy-on-white, semantic green for benefits):
- lead with a Net cost hero block — the headline figure a board commits
to — with gross/home and the gross · funding breakdown beneath it;
- turn the cost lines into a colour-keyed composition bar (navy → gold →
grey, echoing the data-quality evidence bar) with matching dots;
- give annual benefits green icon-chip rows (they're recurring savings)
and value-for-money its own iconography;
- add section and header icons throughout.
Purely presentational — same LedgerView data, no behaviour change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Follow-up to the layout tidy: the inset only applied to the issues table,
so it sat narrower than the cards above and below it. Constrain the whole
data-quality page at the container level (max-w-7xl) instead, so the
header and all three sections share the same edges, and dial the gutter
back from the earlier too-narrow max-w-6xl.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three small visual fixes to the data-quality page:
- vertically centre the rows of the top issues table (was top-aligned, so
the count/impact/link hugged the top of each tall row);
- inset that table (max-w-6xl, centred) so it isn't stretched full-width
for its sparse content;
- replace the button-like "Template coming soon" box with a subtle muted
"Coming soon" status pill (clock icon), so it no longer reads as a
broken/disabled button.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The whole reporting view (scenario `view`, the scenario filters, and the
tag filter) now lives in the URL, so it survives refresh and browser
back/forward. `ReportingClientArea` reads the full `ReportingViewState`
from `useSearchParams` as the single source of truth, replacing the
`useState` for view/filters.
A single `applyViewState` writer routes each change by what it affects:
- a changed tag filter re-renders the server baseline → `router.replace`
(a brief metrics Suspense skeleton is correct; the data changes);
- a `view` or scenario-`filter` change only touches the client-fetched
overlay → History API (`pushState` for a scenario switch so Back has an
entry, `replaceState` for rapid filter toggles), so the baseline
Suspense never re-flashes.
Adds a pure `tagFiltersEqual` (order-insensitive; `includeMode` only
counts with 2+ includes) to decide the server-vs-client route.
Verified in a headless browser on portfolio 796: view+filters persist
across refresh and back/forward; view/filter changes make zero server
round-trips (no baseline re-flash) while a tag change makes exactly one.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a view-wide tag filter to the reporting page. A "Tags" control in the
header lists the portfolio's tags; each can be Included (home must carry it) or
Excluded (home must not) — mutually exclusive per tag — with an Any/All switch
once two or more are included.
The filter is URL-driven (?includeTags=…&excludeTags=…&includeMode=all): the
server reads it and renders the baseline (current-stock figures) filtered, so it
survives refresh and back/forward. Changing it navigates and streams the
filtered figures back in via the Suspense boundaries. The scenario overlay,
measure allocation and drill-down all send the same tag params (in their
react-query keys), so the whole view stays consistent with the headline counts.
Scenario selection + scenario filters remain client state for now (instant, no
round-trip) — migrating those into the URL too is the documented next step.
Verified in a headless browser on #796: including the tag drops the homes count
31,398 → 607 (its exact membership) and writes ?includeTags=13 to the URL.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Threads the tag filter through every reporting query so the whole view reflects
the tagged subset: baseline aggregate, age-band + property-type counts, likely
downgrades, both scenario-metrics routes (all three property scans each), the
drill-down list, and the measures allocation. Each query ANDs
tagFilterCondition() into its WHERE; routes parse the filter from the URL via
the shared parseReportingViewState so they read exactly what the client wrote.
Defaults to the empty (no-op) filter, so existing callers/tests are unchanged.
Verified on #796/scenario 1292: include tag → n_units = the tag's membership
(607), exclude → the complement (30705), and the two EPC distributions
reconcile to the full portfolio total.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Foundation for filtering the reporting view by tags, URL-driven so the whole
view survives refresh and back/forward.
- viewState.ts: the reporting view as URL-serialisable state — scenario `view`,
the scenario filters, and the tag filter {include, exclude, includeMode}.
parse ⇄ serialize, defensively sanitised (numeric tag ids, dedupe,
include/exclude overlap → exclude wins, defaults omitted for clean URLs).
- tagFilterSql.ts: tagFilterCondition() → a SQL WHERE fragment. include=any
(EXISTS), include=all (COUNT DISTINCT = size), exclude (NOT EXISTS), empty =
TRUE (never filters everything out). Verified on real data (#796, #824):
include = membership count, exclude = complement, ANY = union, ALL =
intersection.
27 unit tests. No wiring yet — next: thread through the reporting queries + UI.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Switching from Current stock into a scenario has no previous overlay to keep,
so the scenario KPIs were momentarily empty — an empty-bar flash before the
figures loaded. That's the "no data yet" case, so it now shows the same
KpiBandSkeleton the initial page load uses (extracted + shared), and drops the
now-redundant "Loading scenario…" line there.
The dim + "Updating figures…" path is unchanged and stays for the cases where
data is already on screen (scenario→scenario switch, filter changes), where the
previous figures are deliberately kept via keepPreviousData. Net: skeleton = no
data yet; dim = refreshing existing data — one consistent convention.
Verified in a headless browser on #796: first entry shows the KPI skeleton +
aria-busy and no empty flash; a filter toggle still dims the figures with
"Updating figures…".
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The scenario overlay uses keepPreviousData, so switching scenario or changing
a filter left the previous figures on screen with nothing signalling they were
being recalculated. Now:
- The picker row shows a small spinner + "Updating…" beside the filter chips
while the overlay refetches — immediate feedback right at the controls. It
reads react-query's useIsFetching(["scenario-report"]) so the query stays
owned by MetricsBody (no lifting).
- The figures (KPI band + EPC distribution + ledger) dim (opacity-50, aria-busy)
while refetching with stale data on screen, and show a spinner + "Updating
figures…" / "Loading scenario…" line above them.
Verified end-to-end with a headless browser on portfolio 796: switching to an
"Increasing EPC" scenario and toggling a filter both surface the spinner,
"Updating…"/"Loading…" text, aria-busy and the dimmed figures.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The page previously awaited a Promise.all of every query in the server
component, so nothing painted until the slowest query returned. Now the page
kicks the queries off without awaiting and hands the promises to the client
area, which unwraps them with use() inside Suspense boundaries.
Result: the header + "Download report" paint on first byte; the scenario
picker streams in as soon as the (fast) scenarios query lands; the metrics
body (KPI band, EPC distribution, stock breakdown, confidence strip) streams
in when the baseline aggregate lands — each behind a skeleton fallback. The
current-stock "N homes" count sits in its own inner boundary so it never
holds back the picker.
All existing logic (KPIs, ledger, goal callout, drill-down, scenario overlay
via react-query, measure allocation) is preserved verbatim — just relocated
into the streamed MetricsBody. Verified end-to-end against portfolio 796
(authenticated, HTTP 200): shell + skeletons emit first, sections stream in.
Uses the same use(promise) pattern already shipping elsewhere in the app
(property pages unwrap params/searchParams the same way).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The overlay's dominant cost was a full sequential scan of property_details_epc
(the being-phased-out legacy table — ~600k rows, ~330ms warm, ~2.5s cold from
disk) on every query, because the LEFT JOIN was keyed on property_id alone.
property_details_epc has a (property_id, portfolio_id) composite index, so
adding `AND e.portfolio_id = <pid>` to the join turns the seq scan into an
index lookup. For migrated portfolios the legacy table holds ~0 rows, so the
join becomes near-free; for un-migrated ones it returns the same rows via the
index. EXPLAIN confirms 0 rows dropped and the exact (numeric) aggregates are
unchanged — only order-dependent ::float SUM accumulation jitters, which the
UI rounds away.
Applied to the scenario overlay (both [scenarioId] and default routes) and the
initial-load baseline aggregate. On portfolio #796: overlay handler ~1.6s →
~0.74s (warm; larger win cold), baseline query 790ms → 330ms.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reporting was slow on large portfolios (e.g. #796) because every metric
re-scanned property ⋈ property_details_epc ⋈ newApproachJoins, and the
scenario overlay ran five queries strictly serially.
Baseline (initial load): the six queries that were the identical scan —
total, averages, totals, EPC-band distribution, estimated/actual split and
expired count — collapse into one conditional-aggregation pass. Age bands
(correlated subqueries) and likely downgrades (legacy-only join) stay
separate and run in parallel. 8 queries → 3; ~1945ms → ~787ms on #796,
results byte-identical.
Scenario overlay: Query 2 (portfolio-after aggregates) and Query 3 (EPC
distribution) were near-identical per-property LATERAL scans, and Q3
shipped one row per property to Node just to bucket bands in JS. Merged
into a single scan that buckets bands in SQL (thresholds matched to
sapToEpc exactly), and the three data queries now run in one Promise.all
wave instead of serially. Applied to both the [scenarioId] and default
routes. Verified identical aggregates + band counts on #796's ~31k-home
scenario.
All ADR-0002 / ADR-0010 semantics (effective baselines, compliance window,
lodged toggle) preserved unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Record that, because evidence requirements reference the file_type enum
rather than a document_type table, adding a new required document kind is
a schema change (enum value) rather than a data insert.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Evidence requirements now reference the existing uploaded_files file_type
enum instead of a separate document_type reference table, so the required
and submitted document kinds share one taxonomy.
- Remove the document_type table and its relations.
- project_workstream_evidence_requirement.document_type_id ->
file_type (file_type enum, NOT NULL).
- Extract fileType/fileSource enums into uploaded_files_enums.ts so the
Projects module can reference fileType without an import cycle
(uploaded_files.ts imports the Projects tables for its FK columns).
uploaded_files.ts re-exports the enums for backwards compatibility.
- Regenerate migration 0273 (now 10 Projects tables, not 11).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
drizzle-kit recurses the schema directory and tries to compile every
file as JS/TS. The new projects/schema.md broke generation with a
SyntaxError. Narrow the glob to *.ts so docs can live alongside schema.
Generate migration 0273 for the Projects module: 11 new tables, the
three nullable uploaded_files FK columns, and the new "projects"
file_source enum value.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
landlord_property_id values like "07510027001" were being read via SheetJS
without raw:false, so CSV parsing coerced numeric-looking cells to numbers
and dropped leading zeros. Also wires up drag-and-drop on the bulk tag
upload dropzone, which was styled as one but only supported click-to-browse.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
AnalyticsView.tsx imported and rendered QueriesReviewPanel, but that
component file was never added in any commit -- the import has been
dangling since it was introduced, breaking the production build with
"Module not found: Can't resolve './QueriesReviewPanel'".
Removes the import and its render block. The Queries/Review-with-Landlord
row can be reintroduced once the component actually exists.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Update the proposal doc to match what was actually built:
- client_id is uuid -> organisation (Client maps to organisation).
- property_id FKs are bigint (Property has a bigint PK).
- status/priority columns documented as text (values TBC).
- Drop the free-text uploaded_by column; note the existing bigint
FK -> user is reused.
- Document the new minimal project_type table.
- Record that a "projects" file_source value was added.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduce the Projects domain model as Drizzle table definitions
(schema-only; no migration generated/run yet).
New tables in src/app/db/schema/projects/projects.ts: project,
project_type, project_property, contractor, workstream,
project_workstream, project_workstream_stage,
project_workstream_contractor, document_type,
project_workstream_evidence_requirement, work_order.
Also:
- uploaded_files: add "projects" file_source value and three nullable
bigint FK columns linking evidence to the Projects module.
- relations.ts: wire the Projects relations.
- db.ts: register the new schema module.
Deviations from schema.md, decided during implementation:
- client_id is uuid -> organisation (Client maps to organisation).
- property_id FKs are bigint (Property has a bigint PK, not integer).
- TBC status/priority columns modelled as text for now.
- No new uploaded_by column; reuse the existing bigint FK -> user.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
