mirror of
https://github.com/Hestia-Homes/Model.git
synced 2026-07-19 17:03:02 +00:00
7aea692521
HistoricEpcS3Repository reached into utils/s3.py (read_csv_gz_from_s3 + parse_s3_uri), the legacy utility that self-constructs boto3 inside free functions. The other S3 repositories deliberately depend on the infrastructure/s3 layer instead (UnstandardisedAddressListCsvS3Repository injects a CsvS3Client). Bring historic EPC into line. - Add GzipCsvS3Client(S3Client) in infrastructure/s3: read_csv_gz(key) -> DataFrame (get_object + gzip decode). - Inject it into HistoricEpcS3Repository; the bucket lives in the client and the repo only builds the per-postcode key + maps rows (no S3/HTTP code). Add with_default_s3_client(s3_root) for composition roots. - Update main.py and the match_addresses_for_postcode seam to the factory. - Repo tests inject a real GzipCsvS3Client over a controlled boto stub (exact key assertions + AccessDenied); add a moto-based client test and a factory test covering s3_root -> bucket+key. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MQE5TsSuQTeNSCSz9A9GQf
22 lines
807 B
Python
22 lines
807 B
Python
from __future__ import annotations
|
|
|
|
from io import BytesIO
|
|
|
|
import pandas as pd
|
|
|
|
from infrastructure.s3.s3_client import S3Client
|
|
|
|
|
|
class GzipCsvS3Client(S3Client):
|
|
"""Reads a gzipped-CSV S3 object into a pandas DataFrame.
|
|
|
|
The S3-facing half of the historic-EPC read: an :class:`S3Client` (injected
|
|
boto client + bucket) plus the gzip/CSV decode, so a Repo can depend on this
|
|
instead of the ``utils.s3`` free functions. ``low_memory=False`` so the wide,
|
|
mixed-type historic-EPC columns infer a dtype from the whole column rather
|
|
than per-chunk (which would otherwise split one column across object/float).
|
|
"""
|
|
|
|
def read_csv_gz(self, key: str) -> pd.DataFrame:
|
|
raw = self.get_object(key)
|
|
return pd.read_csv(BytesIO(raw), compression="gzip", low_memory=False)
|