The differ and the HubSpot fetch already named notes_for_surveyor, but the
SQLModel had no such field, so check_for_db_update_trigger raised
AttributeError on every deal and the scrape could not diff at all.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A second export selection alongside the per-Scenario one, chosen by
`plan_selection` on the export request:
- "scenario" (default, unchanged): one sheet per Scenario, freshest Plan per
(property, scenario); scenario_ids required.
- "default" (new): a single "Default Plans" sheet of each home's is_default
Plan (one-per-property across all Scenarios, ADR-0012/0017) — the portfolio's
current state, one row per home; scenario_ids ignored.
Repository gains `default_rows_for(portfolio_id, property_ids)`: same read-model
and Effective-EPC join as `rows_for`, with the plan-selection WHERE swapped from
`scenario_id = :scenario_id` to `is_default = TRUE`. Orchestrator branches on
plan_selection; the router validates it and only requires scenario_ids for the
scenario selection. A recipe without plan_selection defaults to "scenario", so
the change is backward-compatible.
Both selections now surface the chosen Plan's `name` as a `plan_name` column.
Tests: repository default_rows_for (scenario-independent selection, plan_name,
non-default excluded); orchestrator default path (one "Default Plans" sheet;
empty selection is a recorded failure). 62 passed, pyright --strict clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fetch the property from the HubSpot deals API, persist it on
hubspot_deal_data (column added by the frontend repo's drizzle
migration), and include it in the differ's FIELD_MAP so a change
to the field triggers a DB reload.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Wires POST /v1/exports/scenario -> tasks.inputs recipe -> pinned sub_task ->
ARA_EXPORT_SQS_URL -> ara_export Lambda -> orchestrator. Route resolution and the
trigger body are covered by tests; the Lambda handler mirrors bulk_document_download.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Behaviour-preserving lift-and-shift of the condition module out of legacy
backend/condition/ into domain/condition, infrastructure/condition,
infrastructure/postgres/condition_tables.py, repositories/condition,
applications/condition, and tests/condition. Imports rewritten to the DDD
paths; ConditionPostgres and the ORM models keep the legacy backend.app.db
Base/db_session bridges so the existing suite proves behaviour is unchanged
(SQLModel + session-DI conversion tracked as a follow-up in ADR-0064).
16 condition tests pass at the new location.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sync Planning Authority, Designated Area, Article 4 PD Rights, and
Listed Building from HubSpot into hubspot_deal_data so downstream
consumers of the existing HubSpot ETL process can read them.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
uploaded_files has no property_id — matching is on landlord_property_id — so the
property_ids path was pointless indirection (look property.id up in property just
to translate back to landlord_property_id). The FE holds landlord_property_id per
row anyway.
task.inputs hand-pick key is now landlord_property_ids: str[] (was property_ids:
int[]). resolve_selection unions two landlord_property_id sources — project_codes
expanded via hubspot_deal_data, and the hand-picked ids taken as given — and no
longer queries the property table at all. Trigger-only change; the domain plan,
orchestrator and matching already work in landlord_property_id.
ADR-0060, CONTEXT.md and the request schema updated to match.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 'all properties' selection resolved against property.portfolio_id, but a
portfolio spans multiple HubSpot projects — so 'all' pulled the whole portfolio,
not the project the user was looking at. The project↔property grain lives on
hubspot_deal_data, not property.
task.inputs is now {project_codes?: str[], property_ids?: int[], portfolio_id?}:
the route resolves the distinct landlord_property_id set as the union of every
property in the named project_codes (from hubspot_deal_data) and the hand-picked
property_ids; portfolio_id is optional and only names the package. Drops the
portfolio-scoped select_all. Cap now applies to the resolved set size.
ADR-0060, CONTEXT.md and the request schema updated to match. New router tests
cover project-code resolution, the union+dedup, null landlord_property_id drop,
and the empty-project rejection.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses PR #1498 review:
- A per-file read failure (missing/deleted object, or a bucket the role can't
reach) is now a SkippedDocument(reason=unreadable), not a whole-run abort;
an all-unreadable selection still fails (no empty package). Best-effort on
the read path (ADR-0060).
- Each ZIP member is streamed to a temp file (S3DocumentDownloader.download,
boto download_file) and added from disk, so a single multi-GB member never
hits the heap — the 'never held whole in memory' claim is now true.
- The 409 double-submit guard is DB-arbitrated: the sub_task insert is
conditional on the task having none, so a race creates only one.
- Local env gets a placeholder recipient email so the route is exercisable.
- PackageEntry carries landlord_property_id so a read failure can be reported.
- TODO noting the UploadedFile.s3_upload_timestamp typing fix.
Two new tests: per-file read failure skips-and-reports; address fallback flows
to the folder name.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Per PR review: the FE writes the selection config
({portfolio_id, property_ids?, select_all?}) into the FE-owned task.inputs and
passes only task_id, so a large hand-picked selection never travels in an HTTP
body. The route reads task.inputs, resolves to landlord_property_ids, caps, and
pins the recipe onto sub_task.inputs as before. Declares the FE-owned inputs
column on the TaskRow mirror so the backend can read it and the test schema
builds it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Resolves the authenticated requester's email (ADR-0059), resolves + caps the
property selection, pins the recipe (landlord_property_ids, recipient_email,
package_name) onto one pre-created sub_task (raw SQL, dodging the mirror
double-registration), and enqueues one message to the worker. Refuses
double-submits (409) and oversized selections (400).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review feedback (#1481): the address batcher and the Modelling Run batcher
implemented the same greedy packing; the core moves to
utilities/grouped_batching.py and both become thin wrappers.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ADR-0056 amended: override and EPC own type/form facts; a property with
neither is Unknown whatever the legacy columns say. Mirror columns removed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
DB access via parameterised SQL, not the infrastructure SQLModel mirrors:
importing those alongside the legacy backend.app.db.models mirrors of the
same tables double-registers them in the shared metadata and crashes the
app at import. Contained to the modelling package until the DDD cut-over.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
/health already returned GITHUB_SHA in its response but never logged
it, and the fastapi lambda's Terraform environment never actually set
GITHUB_SHA — so every health check response contained "unknown" in
production. Wire var.github_sha through the fastapi lambda module
(default "unknown" for local/other invocations) via a new
TF_VAR_github_sha env var set from `github.sha` in
_deploy_lambda.yml's Terraform Plan step, and log it on every /health
call so a request in CloudWatch can be tied back to the deploy that
served it.
Also fix the zip-size gate added for PR #1469: putting it in
tests/test_lambda_zip_size.py (run via the Docker-based ddd_tests.yml
suite) broke CI, because Dockerfile.test's build context excludes
deployment/* (.dockerignore), so check_lambda_zip_size.py couldn't
find variables.tf to read zip_excludes from inside that container.
Move the check to its own lightweight workflow,
check_lambda_zip_size.yml, triggered on pull_request into main — a
plain checkout (no Docker build) has the full repo, so the check
works, runs fast, and still gates merges to main before a regression
can roll into the dev deploy.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Relocate scripts/check_lambda_zip_size.py to
backend/app/requirements/, right beside the requirements.txt it's
built to measure, since it's specific to the fastapi lambda rather
than a generic repo script.
While moving it, also make the excludes list self-updating: the
script now reads zip_excludes straight out of
deployment/terraform/modules/lambda_with_api_gateway/variables.tf
by default instead of duplicating it as CLI flags, so the CI check
and the actual Terraform packaging can't drift apart. requirements.txt
defaults to the sibling file too, so both the CI step and local runs
now collapse to a bare `check_lambda_zip_size.py` invocation.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
