Commit graph

7991 commits

Author SHA1 Message Date
Khalim Conn-Kowlessar
b6746cc24a Terraform + CI for bulk_document_download: exports bucket, 10GB /tmp, SES SMTP, wiring 🟩
New lambda_with_sqs consumer (timeout 900, memory 3008, ephemeral_storage 10240
for multi-GB ZIPs); dedicated retrofit-document-exports bucket (no lifecycle on
DATA_BUCKET); IAM to read source buckets + write/presign-read the exports
bucket; SES SMTP creds baked from Secrets Manager (no ses:* on the role). Adds
ephemeral_storage_size knob to the shared lambda modules (default 512, backward
compatible). Wires the queue url+arn into fast-api and orders the CI jobs
(ADR-0055).

NOTE: terraform is drafted, not validated (no AWS/terraform in the dev env).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 10:03:50 +00:00
Khalim Conn-Kowlessar
3c00e0ef00 FastAPI trigger route: POST /v1/documents/bulk-download 🟩
Resolves the authenticated requester's email (ADR-0059), resolves + caps the
property selection, pins the recipe (landlord_property_ids, recipient_email,
package_name) onto one pre-created sub_task (raw SQL, dodging the mirror
double-registration), and enqueues one message to the worker. Refuses
double-submits (409) and oversized selections (400).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:59:02 +00:00
Daniel Roth
4a1fdd87a8 Merge branch 'main' into feature/abri-api-integration 2026-07-08 09:53:45 +00:00
Daniel Roth
8b7d32433e
Merge pull request #1492 from Hestia-Homes/feature/abri-api-integration
Abri API integration: abandon job
2026-07-08 10:53:21 +01:00
Khalim Conn-Kowlessar
6db6bbd98f ADR-0060: dedicated exports bucket + multi-GB streaming (drop DATA_BUCKET lifecycle) 🟪
Per review: packages go to a separate DOCUMENT_EXPORTS_BUCKET (not DATA_BUCKET),
no 7-day lifecycle imposed on the shared data bucket; packages can be several GB
so they stream via /tmp + multipart upload with raised ephemeral storage.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:52:43 +00:00
Khalim Conn-Kowlessar
8c0b015923 bulk_document_download Lambda: attach-mode handler, trigger body, packaging 🟩
Reads the recipe from sub_task.inputs, assembles the Download Package via the
orchestrator (source docs from each row's bucket -> ZIP -> the dedicated
DOCUMENT_EXPORTS_BUCKET), returns the presigned URL + skip summary for
sub_task.outputs. Dockerfile/requirements mirror bulk_upload_finaliser.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:51:51 +00:00
Khalim Conn-Kowlessar
a36aa49c2b Stream the Download Package ZIP via /tmp and multipart upload 🟪
Multi-GB packages are no longer held in memory: each source file is read,
written into the on-disk ZIP, and released; the archive is streamed up from
disk. Behaviour unchanged (moto tests green).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:51:26 +00:00
Khalim Conn-Kowlessar
879581bb32 Stream a local file to S3 with managed multipart upload 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:50:34 +00:00
Khalim Conn-Kowlessar
79c8890d07 Stream a local file to S3 with managed multipart upload 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:50:10 +00:00
Khalim Conn-Kowlessar
0387f4a897 Resolve a property's display address from its landlord_property_id 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:45:13 +00:00
Khalim Conn-Kowlessar
456d4dffe7 Resolve a property's display address from its landlord_property_id 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:44:36 +00:00
Khalim Conn-Kowlessar
79fa46d97c Read document bytes from an arbitrary source bucket 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:42:37 +00:00
Khalim Conn-Kowlessar
0a09b56425 Read document bytes from an arbitrary source bucket 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:42:05 +00:00
Khalim Conn-Kowlessar
77e7b0fda9 Package the good documents and report the skipped ones 🟩
Pins the best-effort contract (ADR-0060) delivered with the tracer — no red
phase; discriminating: a strict impl would fail the run or omit the report.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:33:58 +00:00
Khalim Conn-Kowlessar
59ef8a12ab Treat a selection with no documents as a recorded failure 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:33:25 +00:00
Khalim Conn-Kowlessar
73b21e8d64 Treat a selection with no documents as a recorded failure 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:32:37 +00:00
Khalim Conn-Kowlessar
e7181b7d1f Build, upload and email a Download Package for a property set 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:31:49 +00:00
Khalim Conn-Kowlessar
115f506353 Build, upload and email a Download Package for a property set 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:30:38 +00:00
Khalim Conn-Kowlessar
5151fea9b2 STARTTLS and authenticate before sending the email 🟩
Pins the auth handshake delivered with the tracer green (no red phase);
discriminating: an unauthenticated/cleartext send would be rejected by SES.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:27:31 +00:00
Khalim Conn-Kowlessar
74732c6e5a Send a document-download email to the requesting user over SES SMTP 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:27:09 +00:00
Khalim Conn-Kowlessar
7466b78e54 Send a document-download email to the requesting user over SES SMTP 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:25:56 +00:00
Khalim Conn-Kowlessar
135dd9abda Fetch all uploaded files for a set of properties by landlord_property_id 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:23:47 +00:00
Khalim Conn-Kowlessar
910f7cc06c Fetch all uploaded files for a set of properties by landlord_property_id 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:23:12 +00:00
Khalim Conn-Kowlessar
da3e0cd4b4 Generate a presigned GET URL for an S3 object 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:18:54 +00:00
Khalim Conn-Kowlessar
ba13300ee1 Generate a presigned GET URL for an S3 object 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:18:10 +00:00
Khalim Conn-Kowlessar
e906c8fc98 Scope latest-per-Document-Type per property, each in its own folder 🟩
Pins behavior delivered with the tracer slice (the group key is
(landlord_property_id, document_type)) — no red phase; discriminating: a
global-by-type dedupe would drop one property's file.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:15:44 +00:00
Khalim Conn-Kowlessar
b89b9fe944 Name the packaged file by Document Type with the original extension 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:07:25 +00:00
Khalim Conn-Kowlessar
63933db529 Name the packaged file by Document Type with the original extension 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:06:51 +00:00
Khalim Conn-Kowlessar
bd928828e3 Skip and report a document with no Document Type 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:06:26 +00:00
Khalim Conn-Kowlessar
34c48d4adc Skip and report a document with no Document Type 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:05:44 +00:00
Khalim Conn-Kowlessar
810e910961 Keep only the latest file of a Document Type in each property's folder 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:04:58 +00:00
Khalim Conn-Kowlessar
28b7b43372 Keep only the latest file of a Document Type in each property's folder 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 09:04:19 +00:00
Khalim Conn-Kowlessar
6d001d650e ADR-0059/0060: Bulk Document Download design — SES-SMTP email sender + capped best-effort Download Package 🟪
Grilled 2026-07-08. Locks: app-owned-task + attach-mode lane (ADR-0055);
FastAPI route resolves the requesting user (dbId -> UserModel.email) and pins
it + the resolved property_id set into tasks.inputs; one capped Download
Package = one sub_task; landlord_property_id matching (property_id a future
gap); address-named folders (hubspot-enriched), latest per Document Type,
null-type skipped+reported; best-effort (fail only on infra / wholly-empty);
URL to sub_task.outputs + email; DATA_BUCKET/bulk-downloads/ ~7-day lifecycle.
Backend-sent email via a repositories/email port + infrastructure/email SES-SMTP
adapter. Glossary: Bulk Document Download, Download Package, Document Type.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 08:54:43 +00:00
Daniel Roth
84d0e1e0d5 Date an abandonment to the deal's own dates, not now() 🟩
date_abandoned now resolves to the third failed attempt's confirmed
survey date, falling back to its last submission date, rather than the
processing date. This dates the OpenHousing cancellation to when the
job actually lapsed even if the trigger message lags or redelivers.

- domain: abandonment_date() encodes the confirmed-survey -> last-
  submission fallback
- DealAbandonment carries both dates; abandon_job raises
  AbandonmentDateUnknownError when a deal has neither
- last_submission_date now flows through the trigger message and request
- the injected clock is dropped: nothing reads now() any more

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 08:51:39 +00:00
Khalim Conn-Kowlessar
fbedf4541b A small roof derives Sub-Ladder rungs below Google's smallest config 🟪
North-drop now runs once: the selection computes the usable (north-dropped)
ladder, feasibility filters it, and Sub-Ladder derivation consumes it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 22:10:37 +00:00
Khalim Conn-Kowlessar
5d0a7751e6 A small-roof house gets Sub-Ladder solar Options instead of nothing 🟩
End-to-end pin at the generator seam (recommend_solar) — passed on first
run since option construction is unchanged by design (ADR-0058).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 22:09:18 +00:00
Khalim Conn-Kowlessar
024a01b128 The Dwelling-Roof Cap bounds Sub-Ladder rungs too 🟩
Pins that the unchanged cap regime (min of 0.7×Google and the ADR-0038
budget) flows into ADR-0058 derivation — no red phase; the tracer
implementation passed the already-min'd cap through.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 22:08:28 +00:00
Khalim Conn-Kowlessar
1756976d7e A roof capped below two panels is still offered no PV 🟩
Pins the ADR-0058 install floor delivered with the tracer slice (rung_max
< 2 returns empty) — no red phase.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 22:07:39 +00:00
Khalim Conn-Kowlessar
5a382948d5 Sub-Ladder rungs never take from north planes 🟩
Pins behavior delivered with the tracer slice (derivation starts from the
north-dropped remainder) — no red phase; the mutation is discriminating
(a kept north panel would win fill-by-generation and shift the yields).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 22:07:18 +00:00
Khalim Conn-Kowlessar
c1a8c27bc1 Sub-Ladder rungs fill from the highest-yield segment first 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 22:06:30 +00:00
Khalim Conn-Kowlessar
0b33f7dc65 Sub-Ladder rungs fill from the highest-yield segment first 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 22:06:10 +00:00
Khalim Conn-Kowlessar
1a34e788d1 A small roof derives Sub-Ladder rungs below Google's smallest config 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 22:05:37 +00:00
Khalim Conn-Kowlessar
79d7743bb1 A small roof derives Sub-Ladder rungs below Google's smallest config 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 22:03:43 +00:00
Khalim Conn-Kowlessar
3ef432c8b2 ADR-0058: Sub-Ladder PV Configurations give small roofs the array their caps allow 🟪
Grilled decisions (issue #1496): 2-panel floor, every rung 2..floor(cap),
pro-rata per-segment yield (conservative), fill-by-generation, same cap
regime as today (no new roof gate), symmetric battery pairing. Glossary
gains "Sub-Ladder Configuration" ("trimmed"/"synthesized" deliberately
avoided — both collide with existing vocabulary).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 21:57:53 +00:00
KhalimCK
bd01f403c9
Merge pull request #1490 from Hestia-Homes/fix/first-plan-is-default
A property's first plan becomes its default whatever the scenario says
2026-07-07 17:59:36 +01:00
Khalim Conn-Kowlessar
8dd125f199 Review nitpicks: pids_with_default naming, empty-batch guard, truthful fake 🟪
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 16:44:23 +00:00
Daniel Roth
9ec629daf1 correct missed lambda renames 2026-07-07 16:43:25 +00:00
Khalim Conn-Kowlessar
5c982beba4 A new default plan demotes the prior default across scenarios 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 16:42:47 +00:00
Khalim Conn-Kowlessar
aa94e76d3d A new default plan demotes the prior default across scenarios 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 16:41:48 +00:00
Daniel Roth
4c79fb3c8e update comment 2026-07-07 16:30:36 +00:00