Commit graph

1 commit

Author SHA1 Message Date
Khalim Conn-Kowlessar
cf8e5b9ec6 feat(modelling): read the gov EPC bulk export via HTTP range requests
The bulk endpoint 302-redirects to a 15.7 GB S3 ZIP with one NDJSON member per
year; each line wraps the per-cert payload in a stringified 'document' that
parses to the same RdSAP-Schema-21.0.1 shape from_api_response already handles.
parse_bulk_line unwraps a record; is_sap_version filters to SAP 10.2; RangeFile
exposes the S3 object as a seekable file so zipfile streams a single year's
member (and a sampler stops early) without downloading the whole archive.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 12:16:18 +00:00