Model/services/ml_training_data
Khalim Conn-Kowlessar a1f89b6033 slice 15c: stream build_features so 500k+ cert runs fit memory
Previously kept the full list of EpcPropertyData in memory before calling
EpcMlTransform.to_rows. For the 25k slice that's ~30 MB; for the 580k
full-2026 corpus it OOM-killed the process silently. Now: parse cert ->
to_row -> append dict -> drop EpcPropertyData reference, so memory is
O(row-dict * n) instead of O(EpcPropertyData * n). Same end-of-frame
post-processing (categorical casts, column-order pin).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 00:36:53 +00:00
..
src/ml_training_data slice 15c: stream build_features so 500k+ cert runs fit memory 2026-05-17 00:36:53 +00:00
tests slice 14h: handle real bulk-JSON shape (NDJSON wrappers + document payload) 2026-05-16 19:45:52 +00:00
pyproject.toml slice 14g: remote_bulk_fetcher extracts ZIP entries via HTTP Range (no full download) 2026-05-16 19:16:52 +00:00