Commit graph

1320 commits

Author SHA1 Message Date
Daniel Roth
5fb1f92d08 migration files 2026-07-06 15:46:41 +00:00
Daniel Roth
d49042eb7e add client booking reference to hubspot deal data 2026-07-06 15:46:18 +00:00
KhalimCK
c861d4a0b0
Merge pull request #353 from Hestia-Homes/feature/portfolio-scenarios
feat(scenarios): portfolio Scenarios tab — gallery, detail, and creation journey
2026-07-06 16:21:21 +01:00
Khalim Conn-Kowlessar
158d5ded92 fix(postcode-search): PR #355 review feedback
- Order address candidates naturally so Flat 2 sorts before Flat 10
  (TDD'd in the domain core; numeric locale compare).
- Disable spell check / autocomplete / autocorrect on the postcode input.
- Long result lists and the tray's postcode chips now scroll inside
  their containers instead of growing the page.
- gitignore docs/wip/** and untrack the postcode-search handover doc.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 15:13:49 +00:00
Khalim Conn-Kowlessar
8a0ad0c0dc feat(postcode-search): routes, add-properties page, PropertyTable entry point
Completes the add-properties-by-postcode journey (ADR-0007) on top of the
TDD'd domain core:

- GET /api/portfolio/[id]/properties/search?postcode= — postcodes.io for
  validation + geography (admin_district, parliamentary_constituency) every
  time; OS Places through the 30-day postcode_search read-through cache
  (refresh=1 bypasses it). Returns candidates with alreadyInPortfolio flags
  from one grouped uprn query. Cache reads accept legacy space-less keys;
  writes use the canonical spaced form.
- POST /api/portfolio/[id]/properties — re-derives facts server-side from
  classification codes (client propertyType never trusted), then one
  transaction: property rows via uq_property_portfolio_uprn ON CONFLICT DO
  NOTHING, vocabulary upserts with os_places provenance (DO NOTHING preserves
  user rows; snapshots mirror the resolved value), property_overrides
  snapshots (building_part 0, original description = OS code), and
  property_details_spatial per-property coordinates. Returns {added, skipped}.
- OS Places fetches now request output_srs=EPSG:4326 so records carry
  LAT/LNG (no mapper read the BNG X/Y coordinates).
- /portfolio/[slug]/properties/add built to the approved mockup: search card
  with cache-age line, per-postcode result cards, cross-postcode selection
  tray, done state. TanStack Query v4, no useEffect/useMemo.
- "Add properties" dropdown beside Export in PropertyTable (search by
  postcode + bulk excel import "Soon"); toolbar AddNew dropdown retired.
- New display/cache helpers (describeCandidate, postcodeCacheKeys,
  cacheAgeDays) TDD'd in the domain core.

Verified live against portfolio 814: search (cache + refresh), submit
(added 2, all four write layers checked in DB), dedup re-submit (skipped 2),
alreadyInPortfolio flags, page + dropdown render.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 14:33:22 +00:00
Khalim Conn-Kowlessar
463e17debc Merge remote-tracking branch 'origin/main' into feature/add-properties-by-postcode 2026-07-06 14:04:30 +00:00
Khalim Conn-Kowlessar
c8c00b87b5 feat(db): add os_places to override_source enum
New provenance value for the deterministic OS Places classification-code
mapping written by the postcode-search journey (ADR-0007, on
feature/add-properties-by-postcode). Enum-only change so it can be
applied to the database ahead of the feature merging.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 13:46:33 +00:00
Khalim Conn-Kowlessar
5b224677dc feat(postcode-search): pure domain core — normalisation, OS-code mapping, write plan
TDD'd (9 tests): postcode normalisation to canonical form; the ADR-0007
deterministic OS classification mapping (RD02/03/04/06/07; other
residential codes selectable with no facts, Unknown never written;
RD10 + non-RD unselectable); 30-day cache freshness for postcode_search;
DPA-preferred UPRN-deduped address candidates with postcode-stripped
title-case addresses; buildWritePlan producing the property row
(incl. local_authority/constituency from postcodes.io — closing the
no-source gap) and 0-2 property_overrides facts keyed to the OS code.

Adds 'os_places' to OverrideSourceValues — enum migration still needs
generating via generate_migration.sh before deploy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 13:32:18 +00:00
Khalim Conn-Kowlessar
a12215b839 feat(scenarios): sign-post disruptive measures in the creation journey
Internal wall and floor insulation (and room-in-roof) involve major
internal works, and landlords often exclude them. Encode that domain
knowledge once as DISRUPTIVE_MEASURES in the scenario model (with
plain-language reasons; guarded by a key-validity test) and surface it
three ways:

- measures step: allowed disruptive chips carry an amber "disruptive"
  tag + amber border, with the reason in the tooltip; a legend explains
  the marking
- "Low disruption" quick-start now excludes exactly this set, so the
  preset and the badges cannot drift apart
- review step: an amber nudge lists any disruptive measures still
  allowed before saving

New CONTEXT.md term: Disruptive measure.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 09:34:51 +00:00
Khalim Conn-Kowlessar
34dcb0a54f fix(scenarios): band picker order, letter contrast, quieter excluded chips
Design feedback:
- Target-band picker reads A→G left-to-right (reading order; the
  gallery/detail meters stay G→A as they mirror the building passport's
  efficiency gauge, which fills low→high).
- Unselected picker chips are now light band-colour tints with one
  consistent dark ink — no ink has legal contrast across all seven band
  colours at full saturation, so the old full-colour+opacity fade made
  letter colours look inconsistent and washed out. Only the selected
  chip goes full colour, with luminance-correct ink (shared bandInk()).
- Excluded measure chips recede (gray, struck through) instead of
  filling dark navy — "Exclude all" no longer paints a wall of blue;
  the measures still in play are the visually dominant ones. State is
  never colour-only: ✓/✕ icon + strikethrough carry it too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 08:56:52 +00:00
Khalim Conn-Kowlessar
fdae966461 feat(scenarios): type-to-confirm delete modal, replacing browser confirm()
Swap the native confirm() popup for the app's shadcn Dialog pattern
(as in Settings → Danger Zone): the destructive button stays disabled
until the user types "delete" (case-insensitive), with Enter-to-submit
and a proper explanation of what's removed. On a delete error the
dialog closes so the inline error is visible.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 08:51:24 +00:00
Khalim Conn-Kowlessar
639cee6c4c feat(scenarios): portfolio Scenarios tab — gallery, detail, creation journey
Users could only create scenarios buried inside the CSV-upload modal, at
trigger time. This adds a first-class Scenarios tab: a gallery of the
portfolio's scenarios, a read-only detail page, and a 3-step creation
journey (goal → measures → review) that saves app-authored scenario rows
per ADR-0003.

Domain rules (TDD'd in src/lib/scenarios/model.test.ts, 19 tests):
- goal semantics: "Increasing EPC" requires a target band; all other
  goals are maximise-within-budget with goal_value forced null
- exclusions-only measure model: normalised (dedupe+sort), unknown keys
  rejected, all-excluded rejected, empty set valid; serialised to the
  backend's brace text format (parses the legacy space-padded variant)
- insert mapping: multi_plan=true, is_default=false, budget per property
- exact-config duplicate detection ignoring name (review-step warning)
- status derived from plan existence — no stored status column

Queries follow the ADR's discipline: gallery status = one grouped
COUNT(DISTINCT property_id) per portfolio; detail = one scoped count
(index-only, powers the blocked-delete message); delete = single atomic
DELETE guarded on NOT EXISTS(plans). Rename is the only other mutation —
configuration is immutable.

Verified live on portfolio 814: create (400 on missing band, 201 on CO₂
draft), derived Awaiting/Modelled in the gallery, rename, delete of the
draft (200) and blocked delete of the modelled scenario (409, "338
properties…" matching the portfolio's true count).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 08:34:47 +00:00
Daniel Roth
cd8a43c798 migration files 2026-07-03 10:38:25 +00:00
Daniel Roth
a6c642966e add number of attempts to hubspot deals table 2026-07-03 10:38:09 +00:00
Jun-te Kim
f788c858c6
Merge pull request #350 from Hestia-Homes/fix/plan-recommendations-rec-id-index
Add index on plan_recommendations.recommendation_id to Drizzle schema
2026-07-02 17:37:49 +01:00
Jun-te Kim
8e39c00ec8 Add journal-tracked migration 0258 for the recommendation_id index
Generated with drizzle-kit generate, then hand-edited to IF NOT EXISTS:
the index already exists on the live DB (created CONCURRENTLY on
2026-07-02), so the migration must no-op there while still building the
index in fresh environments. Snapshot + journal are updated so the next
generated migration doesn't re-emit this index.

Applying it (drizzle-kit migrate) remains a human action.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 16:30:42 +00:00
KhalimCK
96245a05af
Merge pull request #349 from Hestia-Homes/fix/reporting-carbon-effective-baseline
fix(reporting): negative scenario carbon impact — move carbon fragments to effective baseline
2026-07-02 17:29:08 +01:00
Jun-te Kim
b49fbc4265 Add index on plan_recommendations.recommendation_id to Drizzle schema
Plan deletes cascade to recommendation rows, and each cascaded delete
fires an FK check against plan_recommendations.recommendation_id. With
~26M rows and no index on that column the check seq-scans per row,
making any plan deletion path time out.

The index already exists on the live DB (created CONCURRENTLY on
2026-07-02 as idx_plan_recommendations_recommendation_id, 74s build);
this brings the Drizzle schema in line so it stops drifting. No
migration is included — generating/applying it is left as a human
action, and it must reuse the existing index name (IF NOT EXISTS) so
it no-ops against prod.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 16:24:57 +00:00
Khalim Conn-Kowlessar
1cd48bc78e fix(reporting): negative scenario carbon impact — move carbon fragments to effective baseline
"Total carbon saved" on the reporting scenario overlay went negative
(portfolio 796: -10,673 t/yr, £/tCO2 -8,195). Cause: the baseline side
(getTotals → carbonSql) summed bp.lodged_co2_emissions_t_per_yr, which
is NULL for predicted-only properties (12,064 of 796's 31,919 — no
mirrored lodged estimate is written), so they contributed nothing to
the baseline. The scenario side counted their plans' post_co2_emissions
(21,046 t) in full, dragging saved below zero. The same subtraction fed
£-per-tonne-CO2 and average-per-unit.

carbonSql and energyConsumptionSql were left on lodged_* when ADR-0002
(effective performance is the canonical current, superseding "reporting
stays on lodged") migrated the SAP fragments — the same reporting cards
were already mixing effective SAP with lodged carbon. Move both to
effective_*, which is populated for every property with a baseline row
and is the figure the plans' post_* values were scored against. All
call sites (reporting averages/totals, both scenario metrics routes,
portfolio table CO2 column/filter) are current-state surfaces, so none
wanted lodged. Legacy branch untouched.

Verified live on 796: baseline and scenario totals computed in one
snapshot with the shared fragment give saved = +4,059 t/yr (a plan
regeneration was writing 796 mid-diagnosis, so values drift run-to-run;
pre-fix the same snapshot maths gives ~ -13k and worsening as plans
land). Legacy portfolio 433 totals unchanged.

No regression-test seam exists for cross-query consistency (no DB test
harness); consistency is structural — both sides compose the same
carbonSql fragment.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 14:52:21 +00:00
Daniel Roth
8ebb0d7fec amend migration files 2026-07-02 13:48:46 +00:00
Daniel Roth
03ba8400ff Build the new indexes CONCURRENTLY (matches 0141)
The DB is under load; a plain CREATE INDEX blocks writes for the
duration of each build.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 13:41:12 +00:00
Daniel Roth
7eb2705817 migration files 2026-07-02 13:39:05 +00:00
Daniel Roth
4956a95574 add db indexes 2026-07-02 13:38:42 +00:00
KhalimCK
ccd32fc6bb
Merge pull request #322 from Hestia-Homes/feature/update-ui-for-new-modelling
Feature/update UI for new modelling
2026-07-02 12:50:18 +01:00
Khalim Conn-Kowlessar
051927dda4 fix(reporting): age-band and property-type breakdowns for new-approach properties
getCountByAgeBand and getCountByPropertyType read year_built /
property_type straight off the property row, which is NULL for
new-approach properties — every one fell into the Unknown bucket on
/portfolio/[slug]/reporting. Branch them like their already-migrated
siblings (getCountByEpcBand, getEstimatedCounts):

- propertyTypeSql: epc_property (lodged, else predicted) with RdSAP
  code→label mapping and dwelling_type fallback, mirroring
  resolvePropertyDescriptors; legacy stays on property.property_type.
- constructionYearSql: the main building part's construction age band
  mapped to a representative start year (correlated subquery, since
  extension parts would multiply a plain JOIN), so both paths share the
  same year→band bucketing; legacy stays on numeric year_built.
- The RdSAP label maps move into the shared-fragments section as the
  single source of truth; AGE_BAND_YEARS becomes CONSTRUCTION_AGE_BANDS
  carrying label + representative year. Band I (1996–2002) straddles the
  1976–1999 / 2000+ boundary and buckets by its start year.

Verified live against portfolio 814 (all 338 properties resolve, buckets
match the epc_building_part distribution exactly) and legacy portfolios
419/433 (output identical to the pre-fix SQL).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 11:43:32 +00:00
Jun-te Kim
4ec648fb1a Add 'Secondary glazing' to the glazing pgEnum
Hyde 796 landlord descriptions include '100% secondary glazing (sap 9.94)'
(39 properties), currently flattened to 'Single glazing' (U = 4.8 instead of
secondary's 2.9). The calculator already supports secondary glazing (SAP10
code 5) — only this FE-owned enum and the model repo's GlazingType lacked a
member. Additive ALTER TYPE ... ADD VALUE only, generated via drizzle-kit.

Pairs with the model-repo GlazingType.SECONDARY change (Model#1416).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 10:55:36 +00:00
Khalim Conn-Kowlessar
9bf36ad4b6 perf(backfill): faster, resumable recommendation denormalization + ops tooling
Overhaul the recommendation plan_id/material_* backfill for the 21M-row prod
table (see docs/runbooks/recommendation-denormalization-backfill.md):

- Single combined pass: set plan_id + all four material_* columns in one write
  per row (was two full passes, rewriting every row twice). Guard + COALESCE keep
  it idempotent and resumable — done rows are never rewritten.
- Session tuning: synchronous_commit=off, tunable work_mem / maintenance_work_mem.
- Range + phase controls (BACKFILL_START_ID/END_ID/SKIP_CARDINALITY/ONLY_FINALIZE)
  to run disjoint id-range workers in parallel when the disk isn't the bottleneck.
- Percent-complete + ETA in the progress log.
- Finalize builds the two new indexes plain by default (faster in a quiet
  window) with a CONCURRENTLY toggle.

The real speed-up in prod was dropping the 3 secondary indexes for the backfill
then rebuilding — the table is packed (fillfactor 100) so updates can't go HOT
and otherwise maintain every index per row. Index drop/rebuild + vacuum are kept
as standalone SQL so ops controls exactly when the app loses/regains them:

- sql/drop-recommendation-backfill-indexes.sql
- sql/rebuild-recommendation-backfill-indexes.sql (1GB maintenance_work_mem — safe
  on the 4GB instance)
- sql/vacuum-recommendation.sql

run-sql.ts runs .sql files statement-by-statement, autocommitted (so VACUUM /
CREATE INDEX CONCURRENTLY work), for environments without the psql CLI. Wired up
as npm run db:run-sql and backfill:recommendation-restore.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 10:00:46 +00:00
Khalim Conn-Kowlessar
1dfb1e1534 fix(building-passport): portal tooltip content so it isn't trapped by stacking context
The shadcn TooltipContent rendered inline, missing the Portal that upstream
shadcn ships. On the Current Efficiency card the info tooltip lives inside a
`relative z-10` div while the Lodged EPC badge is a sibling at `z-20`, so the
popup's own `z-50` only competed within the z-10 subtree and the badge painted
over it (also latently clipped by the card's `overflow-hidden`). Wrapping
TooltipContent in TooltipPrimitive.Portal renders it at <body>, escaping both
the stacking context and the overflow clip — fixes all 9 tooltips.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 09:56:59 +00:00
Daniel Roth
dd63f46b76 migration files 2026-07-02 08:47:01 +00:00
Daniel Roth
bd6af9b74f add a new main fuel value 2026-07-02 08:46:18 +00:00
Daniel Roth
543cb69414 3 more main_heating_system values 2026-07-02 08:44:25 +00:00
Daniel Roth
4a6a0aabba 5 new main_heating_system values 2026-07-02 08:43:43 +00:00
Khalim Conn-Kowlessar
4941e75683 fix(epc): follow energyMainsGas rename in new-approach resolver
The merge of main renamed epc_property.energyMainsGas ->
energyGasConnectionAvailable (migration 0252), but the new-approach
resolver read the old property name, which main never touched so it
survived the merge with no conflict and broke the build.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 15:56:16 +00:00
Khalim Conn-Kowlessar
59297c3495 Merge remote-tracking branch 'origin/main' into feature/update-ui-for-new-modelling 2026-07-01 15:34:29 +00:00
Daniel Roth
34d8301f24 migration files 2026-07-01 14:44:05 +00:00
Daniel Roth
d59aa7f079 new main heating system value 2026-07-01 14:43:39 +00:00
Daniel Roth
003913a90e migration files 2026-07-01 14:22:16 +00:00
Daniel Roth
ca781771f2 add new water heating types 2026-07-01 14:21:58 +00:00
Daniel Roth
eda1079e65 migration files 2026-07-01 14:08:01 +00:00
Daniel Roth
0a67aa5e4f rename energyMainsGas to energyGasConnectionAvailable 2026-07-01 14:07:26 +00:00
Daniel Roth
7a0d2a4d64 migration files 2026-07-01 12:09:14 +00:00
Daniel Roth
57d91cae1e Add Mixed glazing to GlazingValues 2026-07-01 12:08:57 +00:00
Daniel Roth
6833bee645 migration files 2026-07-01 11:00:30 +00:00
Daniel Roth
c7ce0860c4 add flat room insulation values to RoofTypeValues 2026-07-01 10:59:53 +00:00
Daniel Roth
2ba084f858
Merge pull request #334 from Hestia-Homes/feature/dedupe-documents
Dedupe documents by only fetching the most recent for each S3 key
2026-07-01 09:27:18 +01:00
Daniel Roth
782ad7b1d0 migration files 2026-07-01 07:42:14 +00:00
Daniel Roth
e36fb5398d lodged_ columns on property_baseline_performance are nullable 2026-07-01 07:41:55 +00:00
Daniel Roth
813311f535 migration files 2026-07-01 07:24:19 +00:00
Daniel Roth
8789c9bab0 extend enum 2026-07-01 07:23:36 +00:00
Daniel Roth
54cf807f2b correct comment following previous commit 2026-06-30 11:10:19 +00:00