Commit graph

3 commits

Author SHA1 Message Date
Khalim Conn-Kowlessar
b6746cc24a Terraform + CI for bulk_document_download: exports bucket, 10GB /tmp, SES SMTP, wiring 🟩
New lambda_with_sqs consumer (timeout 900, memory 3008, ephemeral_storage 10240
for multi-GB ZIPs); dedicated retrofit-document-exports bucket (no lifecycle on
DATA_BUCKET); IAM to read source buckets + write/presign-read the exports
bucket; SES SMTP creds baked from Secrets Manager (no ses:* on the role). Adds
ephemeral_storage_size knob to the shared lambda modules (default 512, backward
compatible). Wires the queue url+arn into fast-api and orders the CI jobs
(ADR-0055).

NOTE: terraform is drafted, not validated (no AWS/terraform in the dev env).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 10:03:50 +00:00
Jun-te Kim
409934a730 Fix fastapi lambda exceeding AWS's 250MB unzipped size limit
dev deploys have failed since PR #1231 (2026-06-15) with
InvalidParameterValueException: Unzipped size must be smaller than
262144000 bytes. The lambda zip is built from the whole repo root
(deployment/terraform/modules/lambda_with_api_gateway) and only
excluded tests/deployment/pycache — so .git (206MB packed), the
per-schema corpus.jsonl fixtures under backend/epc_api/json_samples/
(~50MB), and the sap10_calculator PDF specs (~13MB) were all being
zipped up even though none of them are read at runtime.

Also add scripts/check_lambda_zip_size.py, which mirrors the
terraform module's pip install + zip-with-excludes behaviour to
report the projected unzipped size without mutating the repo, and
wire it into deploy_terraform.yml as a fast_api_lambda_zip_size_check
job that gates the real deploy so this fails fast with a clear
message instead of surfacing as an opaque Terraform/AWS error.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-06 12:54:03 +00:00
Jun-te Kim
54a674b5c8 added postcode splitter rewrite to ddd 2026-05-19 16:35:09 +00:00