Implements #418: `/projects/[projectId]` as a Server Component over SQL
aggregates, plus the shared derivation helpers that #419 and #422 are
instructed to reuse rather than reimplement.
## The shared interface — `src/lib/projects/derivations.ts`
Pure, no db client, no table shapes; unit-tested against in-memory
fixtures. This is the single source of truth for *complete*, *overdue*
and *evidence completeness*. Code against it sight-unseen:
buildStageLadders(stages: StageLadderEntry[]): StageLadders
Resolve the ladder once, then ask it questions. `StageLadderEntry` is
`{ id, projectWorkstreamId, order }` — one `project_workstream_stage`
row. Pass EVERY stage of every workstream you will ask about; a partial
ladder derives the wrong terminal stage.
Row-level, over `WorkOrderDerivationInput`
(`{ projectWorkstreamStageId, forecastEnd: string | Date | null }`):
progressOfWorkOrder(ladders, wo): "not_started" | "in_progress" | "complete" | null
isComplete(ladders, wo): boolean
isOverdue(ladders, wo, asOf: Date): boolean
Set-level, for pushing into SQL instead of streaming rows into JS:
ladders.stageIdsByProgress(progress): bigint[]
ladders.isTerminalStage(stageId): boolean
ladders.orderOfStage(stageId) / ladders.workstreamOfStage(stageId)
Evidence (advisory counts only — never a gate, per CONTEXT.md):
requiredEvidenceRequirementIds(ladders, requirements, stageId): bigint[]
evidenceCompleteness(required, satisfied): EvidenceCompleteness
sumEvidenceCompleteness(parts): EvidenceCompleteness
isMissingRequiredEvidence(completeness): boolean
`EvidenceCompleteness` is `{ required, satisfied, missing, ratio }` where
`ratio` is `null` — not `0` — when nothing is required, so "unconfigured"
and "0% met" stay distinguishable. `sumEvidenceCompleteness` sums counts
before dividing rather than averaging percentages.
Also exported: `toCalendarDay(value)`, which normalises a `date` column
value to `YYYY-MM-DD` UTC. Bind it as a SQL parameter rather than using
`CURRENT_DATE`, so a page and its queries measure against the same day
across midnight.
### Rules, as decided in the issue
- complete = stage is the ladder's terminal stage (highest `order`)
- overdue = `forecast_end` strictly before today AND not complete
- evidence = advisory counts of submissions against required requirements
### Edge cases pinned down (documented because two tickets depend on them)
- Single-stage ladder is terminal, not first: it reads `complete`.
- Every stage tied at the maximum `order` is terminal (and at the minimum,
first) — `order` is not unique in the schema, so this stays deterministic.
- Unknown/unloaded stage id derives `null` progress: not complete, and
still eligible to be overdue.
- Dates compare as calendar days in UTC. Due *today* is not overdue.
- A stage-scoped requirement applies once the work order has REACHED that
stage (`requirement order <= work order order`); unscoped applies always.
Only `required = true` requirements count.
## Aggregating in SQL without duplicating the rules
`src/app/repositories/projects/dashboardRepository.ts` counts in the
database — one grouped query over (workstream, contractor, stage) returns
a few hundred rows regardless of the work-order count. The rules are not
restated in SQL:
- Terminal-ness is never asked of SQL. The aggregate groups by stage id;
`buildStageLadders` classifies those few dozen ids in JS.
- Overdue is split at its `AND`: SQL counts rows past their forecast day
(`pastForecast`), and `overdueOf` drops terminal groups.
- Evidence applicability is decided by `requiredEvidenceRequirementIds`
and handed to SQL as a `(stage_id, requirement_id)` VALUES list — it
crosses into SQL as data, never as a re-implemented predicate.
`src/lib/projects/dashboardSummary.ts` folds those grouped rows into the
four bands, so the KPI totals and the tables beneath them are derived from
the same aggregate and cannot disagree.
## Notes
- "Unassigned contractor" is structurally unreachable in v1
(`work_order.project_workstream_contractor_id` is NOT NULL), so the
count reads 0. It is computed via a LEFT JOIN rather than hard-coded so
it becomes truthful on its own if that column is ever relaxed.
- Authorization is left on the existing `../authz` route seam, which the
per-project layout also applies. Replacing it with `@/lib/projects/authz`
is #409's work; this change adds no permission logic of its own.
- "Awaiting auth", "No access" and notifications are cut from the
wireframe, as the issue specifies — no schema support.
## Seeding: written, NOT run
`src/app/db/seed/seed-projects-dashboard.ts` seeds ~1,500 work orders so a
human can check the render at programme scale. **It was deliberately never
executed.** The database reachable from this environment is the shared
PRODUCTION database, so seeding from here would have written fabricated
delivery records into live data. Run it yourself against a non-production
database:
ALLOW_PROJECTS_DASHBOARD_SEED=i-am-not-production \
npx tsx src/app/db/seed/seed-projects-dashboard.ts <projectId>
It refuses to run without that variable, refuses under NODE_ENV=production,
and prints DB_HOST with a 5s pause before writing. Everything it inserts is
prefixed `SEED-418-` so it can be removed again.
Consequently the "renders acceptably with ~1,500 seeded orders" criterion
is UNVERIFIED — it needs that script run on a scratch database and the page
opened. The query shape is designed for it (grouped counts, bounded result
set), but designed-for is not measured.
## Tests
110 tests over the derivation logic, the fold and the query construction —
in-memory fixtures throughout. The repository test stubs `@/app/db/db` via
`vi.mock` so no `pg.Pool` is ever constructed and no connection can open.
Full suite: 618 passed. Typecheck and lint clean. Cypress not run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reorganises the authz lib along DDD lines by separating the persistence
boundary from the domain:
src/lib/projects/authz.ts domain — pure decisions
src/app/repositories/projects/authzRepository.ts persistence — Drizzle
The repository sits alongside src/app/db/ and mirrors the existing
db/schema/projects/ layout, so schema and repository stay symmetrical as
Ara Projects grows more of both.
Dependencies point one way: the repository imports the domain's fact types
and returns them, so the domain never learns a table shape and continues to
import no database client. The 34 guard tests still run with no connection.
No behaviour change — the queries and guards are untouched. Note this is a
new pattern for the codebase: the prevailing convention is a colocated
queries.ts/server.ts per lib folder, which 88 files still follow.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Code review: /onboarding is *user* onboarding, not Portfolio onboarding.
It captures the user's details and their required privacy-policy
acceptance (plus marketing opt-in) and touches no Portfolio at all —
confirmed by the form schema in src/app/onboarding/page.tsx, which has
no portfolio coupling and makes acceptedPrivacy mandatory.
The exemption added in e34f166f was therefore wrong: it let contractor
users into the app without the consent we are required to capture. A
contractor can complete onboarding perfectly well, so nobody needs the
bypass.
Removes the /projects exemption and isProjectsPath(). /projects stays in
the matcher, so the tree is still auth-guarded. Unit and Cypress tests
inverted to assert the corrected behaviour, plus a new case covering an
onboarded contractor reaching /projects.
Refs #406
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds migration 0274, a data-only custom SQL migration seeding the two
reference tables the setup wizard needs (#407):
- project_type: Retrofit, Planned Maintenance
- workstream: the canonical 8 from the UX wireframes, each with a
one-line description (the column is NOT NULL)
Note that #407 specifies the second project type as "New Build"; PR review
superseded that with "Planned Maintenance". The ticket text is stale, not
the code.
Idempotency uses INSERT ... SELECT ... WHERE NOT EXISTS rather than
ON CONFLICT DO NOTHING, because neither table has a unique constraint on
`name` — an ON CONFLICT (name) form would fail at runtime with "no unique
or exclusion constraint matching the ON CONFLICT specification". Re-running
is a no-op and leaves existing rows, including edited descriptions, alone.
The 0274 snapshot is a copy of 0273 with chained id/prevId, matching how
the previous data-only migration (0227) was recorded.
No UI for creating workstreams — deferred to #428.
UNVERIFIED AGAINST A LIVE DATABASE. This environment's DB credentials point
at production, so the migration was deliberately not executed. It was
verified statically only: both statements parse against the PostgreSQL
grammar (libpg_query), the target tables and columns were cross-checked
against the 0273 schema snapshot, every NOT NULL column without a default
is supplied, and the seeded values match the intended set exactly. A human
needs to run it against a non-production database and confirm both the
fresh-DB and re-run cases before it ships.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Ara Projects gets its own top-level route tree rather than living under
the /portfolio/[slug] chrome: Projects are organisation-scoped
(project.organisation_id) and contractor users from external orgs will
work in these screens without ever having a Portfolio.
- src/app/projects/layout.tsx: shell + top-level nav
- src/app/projects/page.tsx: list stub (real list lands with #408's
visibility rule)
- src/app/projects/[projectId]/{layout,page}.tsx: per-project nav
(Dashboard, Work orders, Import, Settings) + dashboard shell
- ProjectsNav: the Toolbar.tsx shadcn navigation-menu pattern, taking
its items as a prop so both tiers share one implementation
Middleware: adds /projects/:path* to the matcher and exempts the tree
from the portfolio onboarding redirect. A contractor is never
`onboarded`, so without the exemption they would be bounced to
/onboarding on every request with no way out. The decision is split into
a pure routeAuthenticatedRequest() so the rule is unit-testable without
minting a JWT.
Authorization is a thin seam in src/app/projects/authz.ts with TODOs
referencing #408 — the shell deliberately does not inline permission
logic, and user->organisation resolution does not exist yet either.
Tests: 8 unit tests over the middleware routing rules; Cypress smoke
test for the unauthenticated redirect, the authenticated render, the
contractor path, and the per-project shell.
Refs #406
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds src/lib/projects/authz.ts as the single source of permission truth
for every Ara Projects route handler — no route re-implements this inline.
Ara Projects are organisation-scoped, so none of the portfolio-scoped
team_portfolio_permissions machinery applies. Membership resolves through
team_members -> team -> org_id instead.
Role resolution is most-privileged-first: internal (@domna.homes) >
client (member of the owning org) > contractor (member of an org assigned
to any of the project's workstreams) > no access. project.domna_admin_access
gates the internal role only — with it off, a Domna user falls through to
whatever their own memberships earn them rather than losing access outright.
Client outranks contractor when one org is both owner and deliverer.
The two work-order guards check the permission flag on the *assignment*
rather than on the organisation: a contractor needs membership of that
work order's own assigned org plus the relevant flag, so holding a
permissive assignment on a sibling workstream grants nothing.
Structure follows the repo's model/queries split — authz.ts is pure and
imports no database client, so all 34 guard tests run on in-memory
fixtures with no connection. DB-backed fact loaders (getUserOrganisations
and friends) live in authzQueries.ts.
Note the guard signatures take the project facts explicitly —
canUpdateStage(user, project, workOrder) rather than the (user, workOrder)
in the ticket — since the role can only be resolved against the project.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Records that Ara Projects supersedes the HubSpot Live projects feature
long-term (coexisting for now, no v1 migration) and reserves the top-level
/projects namespace. Adds the canonical Ara Projects glossary to CONTEXT.md,
matching src/app/db/schema/projects/schema.md, and qualifies the ambiguous
'Project' references (HubSpot Project vs Ara Project).
Closes#405
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
majorConditionDeals was computed in computeLiveTrackerData and threaded through
LiveTrackerProps → LiveTracker → AnalyticsView, but never rendered — the
intended "Awaab's Law card" was never built. With the rendered "Flagged at
Survey" count now keyed on the same hasMajorConditionIssue signal, this set is
redundant. Removed the const, the LiveTrackerProps field, the prop plumbing in
LiveTracker + AnalyticsView, and its two tests. hasMajorConditionIssue stays
(used by the survey-flag count).
Typecheck + lint clean; live-tracker tests (64) + full suite (500) green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
24 Charlesworth Street (deal 507693469922) has a major condition issue recorded
(major_condition_issue_description = "Damp in bathroom") but didn't appear in the
live tracker's condition-issue surfaces. Two causes, both now keyed on the issue
itself via a shared hasMajorConditionIssue(deal) predicate:
- majorConditionDeals filtered on dealstage === "3061261536" — a HubSpot stage
that was retired, so the set was always empty. (This set is also not yet
rendered in any card — noted for follow-up.)
- The rendered "Damp, Mould & Other Condition Issues → Flagged at Survey" count
(computeDampMouldRisk.surveyFlagDeals) keyed on major_condition_issue_evidence_s3_url,
which is null for this deal (its photos are in the raw HubSpot field). This is
the count the property was actually missing from.
There is no yes/no column; the description is the "Yes" signal (only filled when
an issue exists). Removed the now-dead MAJOR_CONDITION_STAGE_ID. TDD:
red→green→refactor; 66 live-tracker tests, full suite (502) + typecheck + lint green.
Also gitignore .sandcastle/ (untracked agent worktree scratch).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Static Stitch export previously only available locally; referenced by the
Ara Projects issues (#405-#429). README explains which parts are spec vs
placeholder.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Post-PR structural changes to the Projects module schema (migration
regenerated; not yet applied):
- Delete the contractor table. Contractors are organisations:
project_workstream_contractor.contractor_id -> organisation_id
(uuid -> organisation).
- Rename project.client_id -> organisation_id (uuid -> organisation).
- uploaded_files: drop project_workstream_id and
project_workstream_contractor_id; keep
project_workstream_evidence_requirement_id; add nullable work_order_id
(evidence is per work order, not per project workstream).
- Rewire relations accordingly and regenerate migration 0273.
- Reconcile schema.md with all of the above.
Note: deleting contractor drops its phone_number / email_address, which
have no equivalent on organisation.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Record that, because evidence requirements reference the file_type enum
rather than a document_type table, adding a new required document kind is
a schema change (enum value) rather than a data insert.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Evidence requirements now reference the existing uploaded_files file_type
enum instead of a separate document_type reference table, so the required
and submitted document kinds share one taxonomy.
- Remove the document_type table and its relations.
- project_workstream_evidence_requirement.document_type_id ->
file_type (file_type enum, NOT NULL).
- Extract fileType/fileSource enums into uploaded_files_enums.ts so the
Projects module can reference fileType without an import cycle
(uploaded_files.ts imports the Projects tables for its FK columns).
uploaded_files.ts re-exports the enums for backwards compatibility.
- Regenerate migration 0273 (now 10 Projects tables, not 11).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
drizzle-kit recurses the schema directory and tries to compile every
file as JS/TS. The new projects/schema.md broke generation with a
SyntaxError. Narrow the glob to *.ts so docs can live alongside schema.
Generate migration 0273 for the Projects module: 11 new tables, the
three nullable uploaded_files FK columns, and the new "projects"
file_source enum value.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
landlord_property_id values like "07510027001" were being read via SheetJS
without raw:false, so CSV parsing coerced numeric-looking cells to numbers
and dropped leading zeros. Also wires up drag-and-drop on the bulk tag
upload dropzone, which was styled as one but only supported click-to-browse.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
AnalyticsView.tsx imported and rendered QueriesReviewPanel, but that
component file was never added in any commit -- the import has been
dangling since it was introduced, breaking the production build with
"Module not found: Can't resolve './QueriesReviewPanel'".
Removes the import and its render block. The Queries/Review-with-Landlord
row can be reintroduced once the component actually exists.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Postcode searches can return ~100 addresses. Add a client-side filter
above the results that narrows the list live by address substring, so a
user can jump to a house number or street name without scrolling.
- Filter box only appears when a postcode returns more than 8 addresses
- Header count reflects the filtered subset (e.g. "12 of 97 addresses")
- Empty state when nothing matches; clear button and new searches reset it
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Update the proposal doc to match what was actually built:
- client_id is uuid -> organisation (Client maps to organisation).
- property_id FKs are bigint (Property has a bigint PK).
- status/priority columns documented as text (values TBC).
- Drop the free-text uploaded_by column; note the existing bigint
FK -> user is reused.
- Document the new minimal project_type table.
- Record that a "projects" file_source value was added.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduce the Projects domain model as Drizzle table definitions
(schema-only; no migration generated/run yet).
New tables in src/app/db/schema/projects/projects.ts: project,
project_type, project_property, contractor, workstream,
project_workstream, project_workstream_stage,
project_workstream_contractor, document_type,
project_workstream_evidence_requirement, work_order.
Also:
- uploaded_files: add "projects" file_source value and three nullable
bigint FK columns linking evidence to the Projects module.
- relations.ts: wire the Projects relations.
- db.ts: register the new schema module.
Deviations from schema.md, decided during implementation:
- client_id is uuid -> organisation (Client maps to organisation).
- property_id FKs are bigint (Property has a bigint PK, not integer).
- TBC status/priority columns modelled as text for now.
- No new uploaded_by column; reuse the existing bigint FK -> user.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Postcode searches can return ~100 addresses. Add a client-side filter
above the results that narrows the list live by address substring, so a
user can jump to a house number or street name without scrolling.
- Filter box only appears when a postcode returns more than 8 addresses
- Header count reflects the filtered subset (e.g. "12 of 97 addresses")
- Empty state when nothing matches; clear button and new searches reset it
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The card rendered a blank value and "NaN%": Postgres returns column names
verbatim, so `AS without_epc` came back as `without_epc` while the code read
`withoutEpc` — undefined, and `undefined / total * 100` is NaN.
`db.execute<T>()` asserts the row shape rather than checking it, so tsc, lint and
the full suite all passed on the broken query. The previous field names (`estimated`,
`actual`) were single words and happened to survive the round trip; renaming to
camelCase did not. Quoted aliases are the existing convention (see utils.ts).
Verified against portfolio 839: keys come back as [withoutEpc, withEpc], and the
card renders 29 / "0.5% have no EPC on record".
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Impeccable critique (P1): the escalation was a bare blue text-link on the bar's
pale-blue tint, no persistent underline — invisible next to the navy primary,
and it's the exact action users hunt for. Promote it to a secondary button
(white fill + brand-navy border + check icon, fills on hover) so it stands out
without competing with "Choose a tag"; demote "Clear selection" to a legible
neutral underlined link (was blue-on-blue).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
"Homes Without an EPC" and "Expired EPCs" were both derived from estimatedSql,
which asks "was this picture gap-filled?". That conflates two different questions:
provenance (was it modelled?) and coverage (does a certificate exist?).
They used to agree, because a gap-fill implied no certificate. Model ADR-0054 ends
that: the gov EPC API only serves certs registered since 2012, so a pre-2012
dwelling is gap-filled even though a real, stale certificate exists — and gets the
new source='expired'. On portfolio 824, 375 of the 403 "without an EPC" homes have
exactly that: a real certificate, just an out-of-date one. A different, and far more
actionable, problem than never having been certified.
Two defects fell out of the same conflation:
- Every query matched source='predicted' literally, and `source` is plain text
with no enum or CHECK. An 'expired' row matches neither the lodged nor the
predicted alias, so the home would have read as having a VALID CURRENT cert —
silently gone from both cards, no Estimated badge, nothing to catch it.
- isExpiredSql read epl.registration_date alone, but 714 of 824's 4,873 lodged
certs carry only inspection_date. All 714 counted as current; 170 are in fact
>10 years old. lodgementDateSql already coalesced the two; expiry didn't.
The cards now partition on coverage: withoutEpcSql = no lodged AND no historical
record; isExpiredSql = a cert of either kind, out of date. estimatedSql keeps its
own meaning and spans the whole predicted slot, so an expired home is estimated AND
has an EPC — both true. The `AND estimated = false` guard is gone from the Expired
card: an expired home IS estimated, so it excluded the very homes being counted.
Legacy portfolios are untouched (632: 8/1662 before and after). 824 moves to
913 → 1,083 expired immediately (the inspection_date fix), then to 25 / 1,461 once
Model writes expired rows. Safe to deploy BEFORE Model — the predicates just find
none. The reverse was not safe, which is why this goes first.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
With a 7-row page size, the header "select all" (toggleAllPageRowsSelected)
only selected the visible 7 — so it never reached "all loaded", the condition
that reveals the "Select all N matching" escalation. It now selects every loaded
row (toggleAllRowsSelected):
- when everything matching is already loaded, that IS the whole set (tag via ids);
- when more match than are loaded (e.g. 250 loaded of 10,000), the escalation
appears and routes through mode:"filter" — the server resolves and tags the
entire matching set (getFilteredPropertyIds, chunked), so memory is never the
ceiling and no huge id list is sent from the client.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The header checkbox only selects loaded rows, so bulk-tagging a portfolio with
more properties than are loaded (250-row window) silently tagged just the page.
Bulk mode is now explicit (Gmail pattern): nothing selected = no target (Choose
a tag is disabled), so a tag can't be applied to everything by accident. Tick
rows to target them; once every loaded row is ticked and more match, a "Select
all N matching" escalation switches the target to the whole matching set,
resolved server-side via the existing assignments filter mode (no backend
change). Any manual checkbox change drops the escalation back to the explicit
selection; changing filters / leaving the mode clears it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Surface Design Constraints, Planning Comments, Planning Status, and
Planning Suggested Approach in the Live Reporting properties table,
following the same read-path wiring as the previous planning columns
(query mapping, HubspotDeal type, column defs, hidden-by-default toggle,
CSV export, test fixtures). DB columns already exist via migration 0267.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Surface the four planning columns from hubspot_deal_data — Planning
Authority, Designated Area, Article 4 PD Rights, Listed Building — as
optional columns in the Live Reporting properties table.
The DB columns already exist (migration 0267); this wires them through
the read path only: query mapping, HubspotDeal type, column defs, the
column-visibility toggle (hidden by default), and CSV export. Rendered
as plain text since they are text columns in HubSpot.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The not-modelled banner and the toolbar both rendered a navy "Run modelling"
CTA whenever a portfolio had properties but no plans — two identical buttons.
The banner owns the action in that state, so the toolbar's Run modelling is now
hidden while the banner is up (it returns once everything is modelled or the
banner is dismissed). One primary CTA at a time, as intended.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The selection checkbox column was persistently visible. Now it appears only
when a bulk assign/remove mode is active (launched from the Tags menu) via a
select-less column set, and the selection clears on entering and leaving the
mode. The default table has no checkbox column; per-property tagging still uses
the inline Tags-cell popover, and the bulk bar's target still follows selection
→ filter → whole portfolio while the mode is open.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
