Commit graph

43 commits

Author SHA1 Message Date
Daniel Roth
8e9c1c257d Pin the AWS provider to 6.56.0 across every Terraform stack
v6.57.0 (released 29 July 2026) breaks ordinary reads across unrelated AWS
services — SSM GetParameter returns SerializationException, IAM GetPolicy
returns a 302, ECR returns InvalidSignatureException — with no config
change. Reported upstream as hashicorp/terraform-provider-aws#49170, open
with no root cause identified; 6.56.0 is confirmed good.

Every stack declared `>= 5.0` with no upper bound and no lock files are
committed, so each CI run silently resolved whatever HashiCorp had shipped
most recently. That is how a provider released today broke a pipeline
nobody had touched. Pinning exactly makes the deployed version a reviewed
decision rather than a discovery.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 10:46:45 +00:00
Daniel Roth
9870005230 Ship the Renamer image and infrastructure with database access 🟩
The task lane pulls the task domain types, both task repositories and four
Postgres modules into the image, plus SQLAlchemy, SQLModel and a driver.
Terraform gains the DB credentials block and five Postgres env vars; the
deploy job gains the three DB secrets the shared workflow already declares.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 09:11:41 +00:00
KhalimCK
49a36581f0
Merge pull request #1546 from Hestia-Homes/feat/ara-scenario-export
ARA Scenario Export: branded sheet-per-scenario XLSX export (ADR-0065)
2026-07-23 09:34:51 +01:00
Daniel Roth
edd4ec00de Make the OpenHousing resource mandatory, dropping the default fallback
The deal's third_party_surveyor_identifier is now required by the log
and amend flows: the trigger contract rejects messages without it
(blank treated as missing, since a blank resource makes OpenHousing
silently drop the appointment), and LogJobRequest/AmendJobRequest carry
it as a required field. The ABRI_RELAY_DEFAULT_RESOURCE fallback and
its config/terraform/workflow wiring are removed as dead code.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 13:13:28 +00:00
Jun-te Kim
f0f9c8d8d7 feat(terraform): point modelling_e2e at the historic-EPC backup (ADR-0054)
Sets HISTORIC_EPC_S3_ROOT, the env var the handler reads to build the historic-EPC
reader. With it set, an EPC-less Property whose expired pre-2012 certificate is in
the backup has its prediction conditioned on that certificate and persisted as
source="expired" rather than "predicted" — which is what lets reporting tell "no EPC
at all" apart from "only an expired one".

The bucket comes from the shared remote state rather than a rebuilt string:
`retrofit_sap_data_bucket_name` is — despite the name — the retrofit-data-<stage>
bucket (shared/main.tf:167), and engine and fast-api already read it from that output.
modelling_e2e was the outlier hardcoding "retrofit-data-${var.stage}".

No IAM change: modelling_e2e_s3_read already grants GetObject + ListBucket across the
whole retrofit-data-<stage> bucket, which covers historical_epc/.

Note this makes 'expired' rows start appearing once deployed. assessment-model's
epcSources.ts joins only 'lodged' and 'predicted', so until it addresses the predicted
slot — source IN ('predicted','expired') — an 'expired' row matches neither and the
home drops out of both the "Homes Without an EPC" and "Expired EPCs" cards.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 12:08:29 +00:00
Khalim Conn-Kowlessar
30c7133b9a Add ara_export Lambda + SQS terraform module 🟩
Self-contained root module mirroring bulk_document_download (4GB memory per
ADR-0065, exports-bucket-only IAM). Cross-module wiring (shared ECR/state,
fast-api remote_state + ARA_EXPORT_SQS_URL env, CI image build) still needs a
terraform plan review — unverifiable in this environment.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 09:38:07 +00:00
Daniel Roth
dd54c33c0e
Merge pull request #1508 from Hestia-Homes/feature/abri-api-integration
Correct reference to abri api tf state bucket in hubspot etl lambda
2026-07-08 14:16:09 +01:00
Daniel Roth
358fa1b01d correct reference to abri api tf state bucket in hubspot etl lambda 2026-07-08 13:14:07 +00:00
Daniel Roth
312e5e3339
Merge pull request #1505 from Hestia-Homes/feature/abri-api-integration
Terraform fix for hubspot etl
2026-07-08 13:51:00 +01:00
Daniel Roth
399e00606f tf correction for hubspot etl 2026-07-08 12:48:10 +00:00
KhalimCK
bd24a789b9
Merge pull request #1498 from Hestia-Homes/feature/bulk-document-download
Bulk Document Download: emailed ZIP of a property set's documents (ADR-0059/0060)
2026-07-08 12:49:46 +01:00
Daniel Roth
8e880eea6f Expose the Abri test project-code override to the scraper lambda 🟩
Wires TEST_ABRI_PROJECT_CODE_OVERRIDE from an optional deploy-time tfvar
through to the hubspot_deal_etl lambda env; unset keeps production behaviour.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 10:28:48 +00:00
Khalim Conn-Kowlessar
b6746cc24a Terraform + CI for bulk_document_download: exports bucket, 10GB /tmp, SES SMTP, wiring 🟩
New lambda_with_sqs consumer (timeout 900, memory 3008, ephemeral_storage 10240
for multi-GB ZIPs); dedicated retrofit-document-exports bucket (no lifecycle on
DATA_BUCKET); IAM to read source buckets + write/presign-read the exports
bucket; SES SMTP creds baked from Secrets Manager (no ses:* on the role). Adds
ephemeral_storage_size knob to the shared lambda modules (default 512, backward
compatible). Wires the queue url+arn into fast-api and orders the CI jobs
(ADR-0055).

NOTE: terraform is drafted, not validated (no AWS/terraform in the dev env).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 10:03:50 +00:00
Daniel Roth
c0e444fccb abri_api not abri 2026-07-07 14:40:33 +00:00
Daniel Roth
1085384f21 Merge branch 'main' into feature/abri-api-integration 2026-07-07 14:08:52 +00:00
Khalim Conn-Kowlessar
60c448e66d FastAPI Lambda can send to the modelling_e2e queue
Remote state + MODELLING_E2E_SQS_URL env var + send policy ARN, and
fast_api_lambda deploys after modelling_e2e_lambda so the remote state
exists before it applies.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 12:18:48 +00:00
Daniel Roth
7b2fe80688 The abri lambda ships with its queue, DLQ and UAT-pointed staging config 🟩
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 10:29:01 +00:00
Jun-te Kim
e0380123b6 fix(terraform): wire landlord-overrides SQS queue into FastAPI Lambda
The /v1/bulk-uploads/trigger-landlord-overrides endpoint reads
settings.LANDLORD_OVERRIDES_SQS_URL, but the fast-api Lambda's terraform
never set it, so it fell back to the config default "changeme" and
SendMessage failed with AWS.SimpleQueueService.NonExistentQueue.

The landlord-overrides queue (and its finaliser sibling) were added to the
router/config but only the finaliser was wired here. This adds the missing:
  - terraform_remote_state data source for the queue's state
  - LANDLORD_OVERRIDES_SQS_URL env var on the FastAPI Lambda
  - queue ARN in the fastapi-sqs-send IAM policy (else AccessDenied)

Mirrors the existing postcode-splitter / combiner / finaliser wiring.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 15:36:58 +00:00
Jun-te Kim
adb1bf4b56 Log GITHUB_SHA on /health; move zip-size gate to its own PR workflow
/health already returned GITHUB_SHA in its response but never logged
it, and the fastapi lambda's Terraform environment never actually set
GITHUB_SHA — so every health check response contained "unknown" in
production. Wire var.github_sha through the fastapi lambda module
(default "unknown" for local/other invocations) via a new
TF_VAR_github_sha env var set from `github.sha` in
_deploy_lambda.yml's Terraform Plan step, and log it on every /health
call so a request in CloudWatch can be tied back to the deploy that
served it.

Also fix the zip-size gate added for PR #1469: putting it in
tests/test_lambda_zip_size.py (run via the Docker-based ddd_tests.yml
suite) broke CI, because Dockerfile.test's build context excludes
deployment/* (.dockerignore), so check_lambda_zip_size.py couldn't
find variables.tf to read zip_excludes from inside that container.
Move the check to its own lightweight workflow,
check_lambda_zip_size.yml, triggered on pull_request into main — a
plain checkout (no Docker build) has the full repo, so the check
works, runs fast, and still gates merges to main before a regression
can roll into the dev deploy.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-06 13:43:36 +00:00
Daniel Roth
71b304fa47 decrease lambda concurrency from 32 to 12 2026-06-29 13:32:53 +00:00
Khalim Conn-Kowlessar
ecb31bf114 Inject Solar throttle gap derived from queue maximum_concurrency
Terraform computes SOLAR_MIN_REQUEST_INTERVAL_SECONDS = N / (0.8 * 600 / 60)
from var.maximum_concurrency (=4.0s at N=32) so the fleet width has one source
of truth and the per-container Solar pacing tracks it automatically.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 16:05:57 +00:00
Jun-te Kim
a7d3e00178 32 2026-06-23 15:42:00 +00:00
Jun-te Kim
c12bfc7413 test for 16 2026-06-23 14:11:30 +00:00
Jun-te Kim
00af7b5a54 data types 2026-06-23 12:42:53 +00:00
Daniel Roth
6b8eb88a38 increase lambda timeout to maximum 2026-06-22 16:17:30 +00:00
Daniel Roth
2156109e99 concurrency = 2 2026-06-22 14:54:23 +00:00
Daniel Roth
233dace245 2 concurrent executions 2026-06-22 14:49:28 +00:00
Daniel Roth
d05e5bd1f3 new application to trigger e2e for a single property and scenario 2026-06-22 12:54:53 +00:00
Jun-te Kim
360e069386 hubspot etl 20 in parrell 2026-06-19 11:07:30 +00:00
Daniel Roth
8b27a5fda2 correct lambda name 2026-06-15 14:08:40 +00:00
Daniel Roth
963b7d70fe fix terraform error and pass handler bool for dry runs 2026-06-15 14:06:54 +00:00
Daniel Roth
8cb0e986e6 Deploy SharePoint renamer as Lambda with SQS trigger 🟩 2026-06-15 10:52:52 +00:00
Daniel Roth
c1dda41857 add missing variable to lambda variables 2026-06-09 15:33:35 +00:00
Daniel Roth
163703285f deployment 2026-06-09 14:15:52 +00:00
Daniel Roth
11fd88bcb0 rename database environment variables 2026-06-05 16:04:33 +00:00
Jun-te Kim
384e3da92a
Merge pull request #1172 from Hestia-Homes/feature/landlord_data
Feature/landlord data
2026-06-04 16:14:39 +01:00
Daniel Roth
ab465bf010 increase pashub fetcher timeout to 5 mins 2026-06-04 15:12:29 +00:00
Jun-te Kim
dfd05ba28b tests files 2026-06-04 11:47:42 +00:00
Daniel Roth
eb72b0223f add ECO_SHAREPOINT_ID env var 2026-06-04 09:11:08 +00:00
Jun-te Kim
47dfe34ec0 added landlord description overrides 2026-05-29 12:12:54 +00:00
Jun-te Kim
8610a0c875 actually deploy postcode splitter 2026-05-20 15:17:55 +00:00
Jun-te Kim
914a8ed51e postcode splliter working e2e 2026-05-20 11:07:40 +00:00
Jun-te Kim
54a674b5c8 added postcode splitter rewrite to ddd 2026-05-19 16:35:09 +00:00