ADR-0059/0060: Bulk Document Download design — SES-SMTP email sender + capped best-effort Download Package 🟪

Grilled 2026-07-08. Locks: app-owned-task + attach-mode lane (ADR-0055);
FastAPI route resolves the requesting user (dbId -> UserModel.email) and pins
it + the resolved property_id set into tasks.inputs; one capped Download
Package = one sub_task; landlord_property_id matching (property_id a future
gap); address-named folders (hubspot-enriched), latest per Document Type,
null-type skipped+reported; best-effort (fail only on infra / wholly-empty);
URL to sub_task.outputs + email; DATA_BUCKET/bulk-downloads/ ~7-day lifecycle.
Backend-sent email via a repositories/email port + infrastructure/email SES-SMTP
adapter. Glossary: Bulk Document Download, Download Package, Document Type.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

CONTEXT.md

@@ -99,6 +99,18 @@ _Avoid_: ventilation report, audit report, PAS ventilation (that is the external
 An externally-uploaded ventilation survey document produced by a human assessor and ingested from an external source (e.g. Coordination Hub). Recorded in `uploaded_files` with `file_type = PAS_2023_VENTILATION`. Distinct from a **Ventilation Audit**, which is machine-generated from MagicPlan floor plan data.
 _Avoid_: ventilation audit (that is the generated output)
 
+**Document Type**:
+The category of an uploaded document — the `uploaded_files.file_type` enum (photo_pack, site_note, the pas_2023_* / ecmk_* families, magic_plan_json, mcs_compliance_certificate, ventilation_audit, other, …). Nullable on the row. In a **Download Package** exactly one file per Document Type is included per property — the newest by `s3_upload_timestamp`; a row with a null Document Type is skipped and reported (ADR-0060).
+_Avoid_: document category, file kind, doc class
+
+**Download Package**:
+The single ZIP archive a **Bulk Document Download** produces: one folder per property (named by human-readable address, enriched from the hubspot deals data, with `landlord_property_id` appended for uniqueness), each folder holding the latest file of each **Document Type** held for that property. Built best-effort — properties/documents that don't resolve are skipped and listed in the run's `sub_task.outputs`, not failed (ADR-0060). Stored under `DATA_BUCKET/bulk-downloads/` with a ~7-day lifecycle; delivered as a 60-minute presigned URL.
+_Avoid_: export (that is the plan/scenario XLSX export), bundle, archive (ambiguous), zip (the format, not the concept)
+
+**Bulk Document Download**:
+The feature/job that assembles a **Download Package** for a chosen set of properties (a hand-picked list or a whole portfolio) and emails the requester a link. Runs on the app-owned-task + attach-mode lane (ADR-0055): the FastAPI route pins the resolved `property_id` set and the recipient email into `tasks.inputs`, and the `applications/bulk_document_download` Lambda builds the package, writes the URL to `sub_task.outputs`, and emails it (ADR-0059/0060). Files are matched by `landlord_property_id` (the missing `property_id` on `uploaded_files` is a known future gap). Selection is capped by property count at the trigger.
+_Avoid_: document export, bulk export, file dump
+
 ### Source data
 
 **Site Notes**:

docs/adr/0059-outbound-email-via-ses-smtp-adapter.md

@@ -0,0 +1,79 @@
+---
+status: accepted
+---
+
+# Outbound email is sent from the backend via an SES-SMTP adapter, with the recipient threaded from the trigger route
+
+The Bulk Document Download feature (ADR-0060) must email a completed package's
+link to the user who requested it. This is the **first outbound email in the
+Python backend** — there is no `smtplib`/SES/SendGrid code anywhere today. Two
+facts force a decision rather than a copy:
+
+1. **Where the email is sent from.** The front end can send email, but the
+   package is produced **asynchronously in a Lambda**, long after the trigger
+   request has returned `202`. The completion event exists only on the backend.
+2. **Who to email.** Auth today only *gates* the FastAPI routes
+   (`validate_token` returns the raw token, not the user); neither the handler
+   nor the `Task`/`SubTask` model captures a user. The requesting identity is
+   provable (the NextAuth JWT carries `dbId` → `UserModel.email`) but is not
+   currently threaded anywhere.
+
+## Decision
+
+**The backend sends the email, through a DDD port + SES-SMTP adapter, to a
+recipient the trigger route resolves and pins into `tasks.inputs`.**
+
+- **Send from the backend, on completion.** The job that finishes the work
+  owns the notification — the orchestrator calls an injected email port after
+  the package is uploaded and the URL minted. The front end is not asked to
+  poll-then-send.
+- **Port + adapter (hexagonal).** A `repositories/email/` port
+  (`EmailSender` — `send(to, subject, body/html)`) with an
+  `infrastructure/email/ses_smtp_email_sender.py` adapter over `smtplib`, using
+  the **existing SES SMTP** setup (terraform `modules/ses`, domain
+  `domna.homes`, credentials already in Secrets Manager
+  `${stage}/ses/smtp_credentials`). SMTP, not the SES SendEmail API, because
+  that is what the provisioned infrastructure exposes. The port is injected into
+  the orchestrator so the send is testable behind an interface and the transport
+  is swappable.
+- **Recipient threaded at the route, pinned in `inputs`.** The FastAPI trigger
+  route resolves the authenticated user (`dbId` → `UserModel.email`, via the
+  existing `get_user`) and writes the email into the `tasks.inputs` JSON at task
+  creation. The Lambda reads it from `inputs` and never touches auth. This
+  graduates the route from *gating* the token to *injecting* the user — a
+  deliberate, contained change to `backend/app/dependencies.py` usage at the one
+  new route.
+- **Config follows the Lambda convention.** SMTP host/username/password reach
+  the Lambda as environment variables (sourced from the Secrets Manager entry in
+  terraform), read via `os.environ`/`PostgresConfig`-style config — not
+  `backend/app/config.py get_settings()`, which is the legacy app boundary.
+
+## Considered options
+
+- **Front end sends the email.** Rejected: the completion moment lives in the
+  Lambda; making the FE responsible means it must poll task status to
+  completion and then send — more moving parts, and the notification would lag
+  or be missed if the user closes the tab. Backend-sends keeps the trigger and
+  the notification on the same side of the queue.
+- **SES SendEmail API (boto3 `ses`) instead of SMTP.** Rejected for now: the
+  provisioned infrastructure issues SMTP credentials; using the API would need
+  new IAM + identity wiring. Revisit if a templated/bulk API becomes worthwhile.
+- **Resolve the recipient in the Lambda from `portfolio_id` → `PortfolioUsers`.**
+  Rejected: a portfolio has many users, so the join cannot identify *the
+  requester* — real risk of emailing the wrong person or several. The requester
+  is only unambiguous at the authenticated route.
+- **FE passes the recipient email in the trigger body.** Rejected: trusts the
+  client for an identity the JWT already proves, and can drift from the
+  logged-in user.
+
+## Consequences
+
+- A reusable `EmailSender` capability now exists for future backend
+  notifications (not just this feature).
+- The one new FastAPI route must inject the user (not merely gate) and persist
+  the email in `inputs`; `Task`/`SubTask` still gain no user column — the
+  recipient lives in the request payload, scoped to the job.
+- Email content is plain for v1 (a link, a 60-minute expiry note, and the
+  skipped-document summary from `sub_task.outputs`); templating can follow.
+- A new terraform env-var surface (SMTP creds from the existing Secrets Manager
+  entry) is added to the Lambda; no new SES infrastructure is required.

docs/adr/0060-bulk-document-download-package-model.md

@@ -0,0 +1,92 @@
+---
+status: accepted (builds on ADR-0055, ADR-0059)
+---
+
+# Bulk Document Download builds one capped, best-effort Download Package per request
+
+Users need to pull the documents held in `uploaded_files` for many properties
+at once — per property, the latest file of each **Document Type** — as a single
+archive, without clicking through the UI file by file. The request is initiated
+in the front end, can span a hand-picked set of properties or a whole
+portfolio, and finishes with a link emailed to the requester (ADR-0059).
+
+`uploaded_files` links to a property by `uprn` **or** `landlord_property_id`
+(both nullable) and has **no `property_id`** column; `file_type` (the Document
+Type) is itself nullable; and files live across arbitrary buckets
+(`s3_file_bucket` per row).
+
+## Decision
+
+**A request produces exactly one Download Package: a ZIP of one folder per
+property, each holding the latest file of each Document Type — built
+best-effort, size-capped at the trigger, on the app-owned-task + attach-mode
+lane (ADR-0055).**
+
+- **Trigger & lifecycle (ADR-0055).** The FastAPI route creates the `tasks`
+  row, resolves the selection to a concrete `property_id` set (expanding "all
+  properties" of a portfolio), **pins that set plus the recipient email into
+  `tasks.inputs`** (TEXT/JSON — a large id list is why this lives in `inputs`,
+  not the 256 KB SQS body), pre-creates **one** `sub_task`, and drops one SQS
+  message (`task_id`, `sub_task_id`). The new `applications/bulk_document_download`
+  Lambda runs in **attach mode**; `TaskOrchestrator` owns status + roll-up.
+- **One package = one sub_task.** No fan-out: the job streams each document out
+  of its own `s3_file_bucket` into a single ZIP in `/tmp` and multipart-uploads
+  it. One artifact, one URL, one email.
+- **Selection cap at the route.** Reject `> N` properties synchronously with a
+  legible "narrow your selection" error (property count, not a document/byte
+  query — cheap, no S3 on the hot path). A `/tmp` size budget inside the Lambda
+  is a backstop that fails the sub_task with a clear reason if a pathological
+  selection still overflows. `N` starts at a conservative, tunable value.
+- **Matching on `landlord_property_id`.** Files are gathered by
+  `landlord_property_id`; the missing `property_id` on `uploaded_files` is a
+  **known future gap**, out of scope here. Property display info (the folder
+  name) is enriched from the hubspot deals data
+  (`repositories/hubspot_deals/`).
+- **Layout.** One folder per property named by human-readable **address**
+  (hubspot-enriched), with `landlord_property_id` appended for uniqueness;
+  inside, one file per Document Type = the newest by `s3_upload_timestamp`.
+  Rows with a **null Document Type are skipped** and listed in the run output.
+- **Best-effort failure contract.** Build from whatever resolves. Skipped
+  properties (no documents) and skipped null-type files are recorded in
+  `sub_task.outputs`. The run **fails only** on an infrastructure error
+  (S3/DB/zip/upload/email) or when the whole selection yields **zero
+  documents**. The email carries an "N properties, M documents, X skipped"
+  summary.
+- **Delivery — both channels.** The presigned URL (60-minute expiry) and the
+  skip-summary are written to `sub_task.outputs` (the FE already polls task
+  status and can show the link) **and** emailed (ADR-0059).
+- **Retention.** Packages are written to `DATA_BUCKET` under a
+  `bulk-downloads/` prefix with an S3 lifecycle rule expiring objects after
+  ~7 days — longer than the 60-minute URL (so a link can be re-issued without
+  rebuilding) but short enough that bundles never accumulate.
+
+## Considered options
+
+- **Fan out into per-batch sub_tasks / multiple ZIPs** (like the modelling
+  run). Rejected: the user would receive N links / N emails and "the download"
+  would stop being one file; the capped-single-package model keeps the artifact
+  and the notification singular.
+- **Cap on resolved document count / bytes.** Rejected for v1: it adds
+  synchronous DB + S3 HEAD work to the trigger and couples the route to the
+  packaging logic; a property-count cap is cheap and predictable.
+- **Strict "all-or-nothing" packaging.** Rejected: one unreadable row or a
+  property with no documents would deny the entire package; best-effort +
+  reporting matches "include documents and properties where they exist".
+- **Match on `uprn`.** Deferred: `landlord_property_id` is the agreed key and
+  hubspot enrichment is keyed to the deal/property; revisit if/when the
+  `property_id` gap on `uploaded_files` is closed.
+
+## Consequences
+
+- New DDD pieces: `applications/bulk_document_download/` (thin handler +
+  trigger body), `orchestration/bulk_document_download_orchestrator.py`,
+  packaging rules in `domain/`, a `landlord_property_id`-keyed
+  "latest-per-Document-Type" query on the uploaded-file repository, a
+  `generate_presigned_url` + multipart upload on `S3Client`, and the email
+  port/adapter from ADR-0059. The only `backend/` touch is the trigger route.
+- The stored, pinned `property_id` set makes a run reproducible even if the
+  portfolio changes between trigger and execution.
+- `N`, the `/tmp` size budget, and the 7-day lifecycle are tunable knobs, not
+  load-bearing invariants.
+- Closing the `uploaded_files.property_id` gap later would let matching move off
+  `landlord_property_id` without changing the package model.