Addresses PR #1498 review:
- A per-file read failure (missing/deleted object, or a bucket the role can't
reach) is now a SkippedDocument(reason=unreadable), not a whole-run abort;
an all-unreadable selection still fails (no empty package). Best-effort on
the read path (ADR-0060).
- Each ZIP member is streamed to a temp file (S3DocumentDownloader.download,
boto download_file) and added from disk, so a single multi-GB member never
hits the heap — the 'never held whole in memory' claim is now true.
- The 409 double-submit guard is DB-arbitrated: the sub_task insert is
conditional on the task having none, so a race creates only one.
- Local env gets a placeholder recipient email so the route is exercisable.
- PackageEntry carries landlord_property_id so a read failure can be reported.
- TODO noting the UploadedFile.s3_upload_timestamp typing fix.
Two new tests: per-file read failure skips-and-reports; address fallback flows
to the folder name.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The row→domain mapper now names all 93 constructor arguments explicitly
instead of splatting a lowercased dict, takes a plain Mapping (a
DataFrame.to_dict("records") row) instead of a pandas Series, and ignores
columns the domain type doesn't know. A missing/renamed CSV column fails
loudly as a KeyError at the row. Both iterrows() call sites move to
to_dict("records") — pandas-stubs types iterrows' Series unparameterized,
which strict mode rejects. pandas-stubs + boto3-stubs[s3] make the stack
check clean: pyright strict is now 0 errors across the PR's files.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
HistoricEpcS3Repository reached into utils/s3.py (read_csv_gz_from_s3 +
parse_s3_uri), the legacy utility that self-constructs boto3 inside free
functions. The other S3 repositories deliberately depend on the
infrastructure/s3 layer instead (UnstandardisedAddressListCsvS3Repository
injects a CsvS3Client). Bring historic EPC into line.
- Add GzipCsvS3Client(S3Client) in infrastructure/s3: read_csv_gz(key) ->
DataFrame (get_object + gzip decode).
- Inject it into HistoricEpcS3Repository; the bucket lives in the client and
the repo only builds the per-postcode key + maps rows (no S3/HTTP code).
Add with_default_s3_client(s3_root) for composition roots.
- Update main.py and the match_addresses_for_postcode seam to the factory.
- Repo tests inject a real GzipCsvS3Client over a controlled boto stub
(exact key assertions + AccessDenied); add a moto-based client test and a
factory test covering s3_root -> bucket+key.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MQE5TsSuQTeNSCSz9A9GQf
