Commit graph

7885 commits

Author SHA1 Message Date
Khalim Conn-Kowlessar
b89b9fe944 Name the packaged file by Document Type with the original extension 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:07:25 +00:00
Khalim Conn-Kowlessar
63933db529 Name the packaged file by Document Type with the original extension 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:06:51 +00:00
Khalim Conn-Kowlessar
bd928828e3 Skip and report a document with no Document Type 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:06:26 +00:00
Khalim Conn-Kowlessar
34c48d4adc Skip and report a document with no Document Type 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:05:44 +00:00
Khalim Conn-Kowlessar
810e910961 Keep only the latest file of a Document Type in each property's folder 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:04:58 +00:00
Khalim Conn-Kowlessar
28b7b43372 Keep only the latest file of a Document Type in each property's folder 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:04:19 +00:00
Khalim Conn-Kowlessar
6d001d650e ADR-0059/0060: Bulk Document Download design — SES-SMTP email sender + capped best-effort Download Package 🟪
Grilled 2026-07-08. Locks: app-owned-task + attach-mode lane (ADR-0055);
FastAPI route resolves the requesting user (dbId -> UserModel.email) and pins
it + the resolved property_id set into tasks.inputs; one capped Download
Package = one sub_task; landlord_property_id matching (property_id a future
gap); address-named folders (hubspot-enriched), latest per Document Type,
null-type skipped+reported; best-effort (fail only on infra / wholly-empty);
URL to sub_task.outputs + email; DATA_BUCKET/bulk-downloads/ ~7-day lifecycle.
Backend-sent email via a repositories/email port + infrastructure/email SES-SMTP
adapter. Glossary: Bulk Document Download, Download Package, Document Type.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 08:54:43 +00:00
KhalimCK
bd01f403c9
Merge pull request #1490 from Hestia-Homes/fix/first-plan-is-default
A property's first plan becomes its default whatever the scenario says
2026-07-07 17:59:36 +01:00
Khalim Conn-Kowlessar
8dd125f199 Review nitpicks: pids_with_default naming, empty-batch guard, truthful fake 🟪
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 16:44:23 +00:00
Khalim Conn-Kowlessar
5c982beba4 A new default plan demotes the prior default across scenarios 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 16:42:47 +00:00
Khalim Conn-Kowlessar
aa94e76d3d A new default plan demotes the prior default across scenarios 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 16:41:48 +00:00
Khalim Conn-Kowlessar
0eaf6fa99c A property's first plan becomes its default whatever the scenario says 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 16:13:47 +00:00
Khalim Conn-Kowlessar
101c1f1f21 A property's first plan becomes its default whatever the scenario says 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 16:12:34 +00:00
Jun-te Kim
c9b5fc095b
Merge pull request #1487 from Hestia-Homes/feature/uprn-confirmation-before-finalise
Confirm UPRNs before finalise: withhold ambiguous address2uprn matches (ADR-0057)
2026-07-07 17:12:14 +01:00
Khalim Conn-Kowlessar
371240c885 The plan repository reports which properties already have a default plan 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 16:11:32 +00:00
Khalim Conn-Kowlessar
015db4275b The plan repository reports which properties already have a default plan 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 16:08:30 +00:00
Jun-te Kim
dbcdf29bd9 refactor(address2uprn): name the match/decision return types; rename helper
Address PR review (dancafc):
- introduce UprnMatch NamedTuple (datatypes/address_match.py) for the
  (uprn, address, lexiscore, certificate_number) return, replacing the bare
  4-tuple in get_uprn_from_epc_df / get_uprn_from_historic_epc /
  HistoricEpcResolver.resolve_uprn. Tuple-compatible, so unpacking is unchanged.
- rename get_uprn_with_epc_df -> get_uprn_from_epc_df (+ callers).
- type resolve_group_ambiguity via a GroupDecision NamedTuple and trim its
  docstring.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 16:00:00 +00:00
Daniel Roth
39beb303ef
Merge pull request #1488 from Hestia-Homes/deployment-fix
Abri API deployment correction
2026-07-07 16:57:08 +01:00
Daniel Roth
2e0cc7432c terraform correction 2026-07-07 15:23:44 +00:00
Jun-te Kim
269aade481 test(address2uprn): cover ambiguity withholding + override dedup; add ADR-0057
- unit tests for resolve_group_ambiguity (distinct addresses withheld,
  same-address re-listing kept, order preserved)
- Postgres integration tests for upsert_all's backstop dedup
- ADR-0057 recording the "confirm UPRNs before finalise" decision

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:21:04 +00:00
Jun-te Kim
4e0134dd27 Merge branch 'feat/address2uprn-certificate-number' into feature/uprn-confirmation-before-finalise
# Conflicts:
#	backend/address2UPRN/main.py
2026-07-07 15:17:47 +00:00
Jun-te Kim
daa1cd7967 feat(address2uprn): withhold ambiguous cross-row UPRN matches (ADR-0057)
Phase 1 of confirming UPRNs before finalise. address2uprn matched each
row independently, so one UPRN could be the best match for two distinct
addresses (a coarse EPC record absorbing several real addresses, e.g.
flats in a block). Those distinct addresses were then silently merged by
the property identity insert, and collided in property_overrides.

resolve_group_ambiguity() withholds a UPRN claimed by >=2 distinct
normalised addresses within a postcode group (keeps genuine same-address
re-listings), and the handler now emits an address2uprn_status column
(matched | ambiguous_duplicate | unmatched | invalid_postcode | error).
Withheld rows drop to a null UPRN but keep their lexiscore for triage on
the (upcoming) confirmation page.

Also adds the ADR-0057 backstop dedup in property_overrides upsert_all so
the ON CONFLICT statement can never double-touch a row.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:15:59 +00:00
Daniel Roth
66b2df99e9
Merge pull request #1482 from Hestia-Homes/feature/abri-api-integration
Abri API Integration: lambda handler and plumb everything together
2026-07-07 15:46:50 +01:00
Daniel Roth
c0e444fccb abri_api not abri 2026-07-07 14:40:33 +00:00
Jun-te Kim
7abc71ed95 Drop property-table wiring for certificate_number
On reflection the certificate number only needs to travel through the
address2uprn pipeline (result CSV, S3 output) as an internal value —
not persisted to property. Reverts the PropertyIdentityInsert /
property_table.py / property_postgres_repository.py changes; keeps
certificate_number flowing through get_epc_data_with_postcode,
get_uprn_with_epc_df, get_uprn_from_historic_epc, and the
address2uprn_certificate_number result column.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-07 14:16:55 +00:00
Daniel Roth
1085384f21 Merge branch 'main' into feature/abri-api-integration 2026-07-07 14:08:52 +00:00
Daniel Roth
f5059c0589 tf comment 2026-07-07 14:04:34 +00:00
KhalimCK
9ff4de0563
Merge pull request #1481 from Hestia-Homes/feature/modelling-trigger-run
Modelling Run Distributor: POST /v1/modelling/trigger-run
2026-07-07 14:36:45 +01:00
Jun-te Kim
3718743801 Carry the EPC certificate number through address2uprn to property
EpcClientService.search_by_postcode already returns the matched
certificate number alongside the UPRN, but it was dropped before
persistence. Thread it through get_epc_data_with_postcode ->
get_uprn_with_epc_df / get_uprn_from_historic_epc (using the historic
dataset's lmk_key) -> the address2uprn_certificate_number result
column -> PropertyIdentityInsert -> the property table's new
certificate_number column (assessment-model PR #362).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-07 13:26:49 +00:00
Khalim Conn-Kowlessar
cc4bf4394e Both postcode batchers share one group-preserving packing core 🟪
Review feedback (#1481): the address batcher and the Modelling Run batcher
implemented the same greedy packing; the core moves to
utilities/grouped_batching.py and both become thin wrappers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:24:02 +00:00
Daniel Roth
04b32b1515 minor refactor 2026-07-07 13:21:01 +00:00
Daniel Roth
7b6c7e1168 remove bad import 2026-07-07 13:19:03 +00:00
Daniel Roth
252a146cc3 remove silly comment 2026-07-07 13:17:24 +00:00
Daniel Roth
3934e2d359 move trigger check to HubspotDealDiffer 2026-07-07 13:15:40 +00:00
Daniel Roth
d9b441962f Name the Abri trigger builder for its None-or-message contract 🟪
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:02:25 +00:00
Daniel Roth
c016c8b754 Extract _trigger_abri_lambda for consistency with the other scraper triggers 🟪
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 12:58:48 +00:00
Khalim Conn-Kowlessar
07190fc332 Legacy property columns no longer resolve type or built form 🟩
ADR-0056 amended: override and EPC own type/form facts; a property with
neither is Unknown whatever the legacy columns say. Mirror columns removed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 12:52:11 +00:00
Khalim Conn-Kowlessar
ddd64a9297 Legacy property columns no longer resolve type or built form 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 12:49:19 +00:00
Daniel Roth
7ad0dc0595 Merge branch 'main' into feature/abri-api-integration 2026-07-07 12:46:17 +00:00
Daniel Roth
a3985389ab The deal row names the job number client_booking_reference, matching HubSpot and the schema 🟪
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 12:43:40 +00:00
Daniel Roth
5286e2a2a2 The job number is stored in the client_booking_reference column 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 12:31:07 +00:00
Khalim Conn-Kowlessar
60c448e66d FastAPI Lambda can send to the modelling_e2e queue
Remote state + MODELLING_E2E_SQS_URL env var + send policy ARN, and
fast_api_lambda deploys after modelling_e2e_lambda so the remote state
exists before it applies.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 12:18:48 +00:00
Khalim Conn-Kowlessar
bf8647c9b5 The 409 guard and batch creation read as methods on ModellingRunTasks 🟪
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 12:16:42 +00:00
Khalim Conn-Kowlessar
b62901a9de The distributor coexists with both table stacks and mounts at /v1/modelling 🟪
DB access via parameterised SQL, not the infrastructure SQLModel mirrors:
importing those alongside the legacy backend.app.db.models mirrors of the
same tables double-registers them in the shared metadata and crashes the
app at import. Contained to the modelling package until the DDD cut-over.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 12:15:31 +00:00
Khalim Conn-Kowlessar
b22b9cbca0 Scenarios outside the portfolio refuse the run 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 12:06:42 +00:00
Khalim Conn-Kowlessar
ca799cc5e5 Scenarios outside the portfolio refuse the run 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 12:06:02 +00:00
Khalim Conn-Kowlessar
e9eb068be9 Filters that match no properties refuse the run 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 12:05:39 +00:00
Khalim Conn-Kowlessar
e9b39df322 Filters that match no properties refuse the run 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 12:05:03 +00:00
Khalim Conn-Kowlessar
cfedb8e9cf A task that already has sub_tasks refuses re-distribution 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 12:04:34 +00:00
Khalim Conn-Kowlessar
dade7a3a5f A task that already has sub_tasks refuses re-distribution 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 12:04:05 +00:00