Sync Planning Authority, Designated Area, Article 4 PD Rights, and
Listed Building from HubSpot into hubspot_deal_data so downstream
consumers of the existing HubSpot ETL process can read them.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
uploaded_files has no property_id — matching is on landlord_property_id — so the
property_ids path was pointless indirection (look property.id up in property just
to translate back to landlord_property_id). The FE holds landlord_property_id per
row anyway.
task.inputs hand-pick key is now landlord_property_ids: str[] (was property_ids:
int[]). resolve_selection unions two landlord_property_id sources — project_codes
expanded via hubspot_deal_data, and the hand-picked ids taken as given — and no
longer queries the property table at all. Trigger-only change; the domain plan,
orchestrator and matching already work in landlord_property_id.
ADR-0060, CONTEXT.md and the request schema updated to match.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 'all properties' selection resolved against property.portfolio_id, but a
portfolio spans multiple HubSpot projects — so 'all' pulled the whole portfolio,
not the project the user was looking at. The project↔property grain lives on
hubspot_deal_data, not property.
task.inputs is now {project_codes?: str[], property_ids?: int[], portfolio_id?}:
the route resolves the distinct landlord_property_id set as the union of every
property in the named project_codes (from hubspot_deal_data) and the hand-picked
property_ids; portfolio_id is optional and only names the package. Drops the
portfolio-scoped select_all. Cap now applies to the resolved set size.
ADR-0060, CONTEXT.md and the request schema updated to match. New router tests
cover project-code resolution, the union+dedup, null landlord_property_id drop,
and the empty-project rejection.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses PR #1498 review:
- A per-file read failure (missing/deleted object, or a bucket the role can't
reach) is now a SkippedDocument(reason=unreadable), not a whole-run abort;
an all-unreadable selection still fails (no empty package). Best-effort on
the read path (ADR-0060).
- Each ZIP member is streamed to a temp file (S3DocumentDownloader.download,
boto download_file) and added from disk, so a single multi-GB member never
hits the heap — the 'never held whole in memory' claim is now true.
- The 409 double-submit guard is DB-arbitrated: the sub_task insert is
conditional on the task having none, so a race creates only one.
- Local env gets a placeholder recipient email so the route is exercisable.
- PackageEntry carries landlord_property_id so a read failure can be reported.
- TODO noting the UploadedFile.s3_upload_timestamp typing fix.
Two new tests: per-file read failure skips-and-reports; address fallback flows
to the folder name.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Per PR review: the FE writes the selection config
({portfolio_id, property_ids?, select_all?}) into the FE-owned task.inputs and
passes only task_id, so a large hand-picked selection never travels in an HTTP
body. The route reads task.inputs, resolves to landlord_property_ids, caps, and
pins the recipe onto sub_task.inputs as before. Declares the FE-owned inputs
column on the TaskRow mirror so the backend can read it and the test schema
builds it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Resolves the authenticated requester's email (ADR-0059), resolves + caps the
property selection, pins the recipe (landlord_property_ids, recipient_email,
package_name) onto one pre-created sub_task (raw SQL, dodging the mirror
double-registration), and enqueues one message to the worker. Refuses
double-submits (409) and oversized selections (400).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review feedback (#1481): the address batcher and the Modelling Run batcher
implemented the same greedy packing; the core moves to
utilities/grouped_batching.py and both become thin wrappers.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ADR-0056 amended: override and EPC own type/form facts; a property with
neither is Unknown whatever the legacy columns say. Mirror columns removed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
DB access via parameterised SQL, not the infrastructure SQLModel mirrors:
importing those alongside the legacy backend.app.db.models mirrors of the
same tables double-registers them in the shared metadata and crashes the
app at import. Contained to the modelling package until the DDD cut-over.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
/health already returned GITHUB_SHA in its response but never logged
it, and the fastapi lambda's Terraform environment never actually set
GITHUB_SHA — so every health check response contained "unknown" in
production. Wire var.github_sha through the fastapi lambda module
(default "unknown" for local/other invocations) via a new
TF_VAR_github_sha env var set from `github.sha` in
_deploy_lambda.yml's Terraform Plan step, and log it on every /health
call so a request in CloudWatch can be tied back to the deploy that
served it.
Also fix the zip-size gate added for PR #1469: putting it in
tests/test_lambda_zip_size.py (run via the Docker-based ddd_tests.yml
suite) broke CI, because Dockerfile.test's build context excludes
deployment/* (.dockerignore), so check_lambda_zip_size.py couldn't
find variables.tf to read zip_excludes from inside that container.
Move the check to its own lightweight workflow,
check_lambda_zip_size.yml, triggered on pull_request into main — a
plain checkout (no Docker build) has the full repo, so the check
works, runs fast, and still gates merges to main before a regression
can roll into the dev deploy.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Relocate scripts/check_lambda_zip_size.py to
backend/app/requirements/, right beside the requirements.txt it's
built to measure, since it's specific to the fastapi lambda rather
than a generic repo script.
While moving it, also make the excludes list self-updating: the
script now reads zip_excludes straight out of
deployment/terraform/modules/lambda_with_api_gateway/variables.tf
by default instead of duplicating it as CLI flags, so the CI check
and the actual Terraform packaging can't drift apart. requirements.txt
defaults to the sibling file too, so both the CI step and local runs
now collapse to a bare `check_lambda_zip_size.py` invocation.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Pulls in 42 commits of calculator/mapper accuracy fixes from the per-cert
mapper-validation and floor/roof/heating fronts.
Conflict resolutions:
- mapper `_is_elmhurst_roof_window`: main dropped the branch's "wall location →
vertical" guard (it broke cert 000516's rooflight), but that re-broke cert
001431's two External-wall U>3.0 windows (which must stay vertical). The two
certs lodge a BYTE-IDENTICAL §11 row, so neither location nor U separates
them — the real discriminator is the room-in-roof context. Replaced the
unconditional U>3.0 backstop with one gated on the BP having a room-in-roof
(`_elmhurst_bp_has_room_in_roof`): 000516's Main BP has a "Room in roof type
1" (→ rooflight), 001431's does not (→ vertical). Validated against BOTH —
full Elmhurst worksheet suite 1038 pass + the 001431 window-extraction pin.
- property_postgres_repository: kept main's `ids_by_uprn` method + the branch's
`_restrictions_of` helper.
- sap_fuel.py: the branch relocated it to domain/billing/ (already carrying
main's to_table_32_code normalization), so kept the old path deleted.
Fallout from main's fabric fixes (validated by the boiler-3 real-cert pin which
still reproduces at delta 0):
- re-pinned the boiler-1 + boiler-instant-hw ASHP snapshot scores;
- main's §14.2 gas-boiler main-fuel derivation resolved the BGB/102 baseline
gap, so `test_gas_boiler_instant_hw_before_baselines` is now a passing test
(was an xfail tripwire).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
