Model/applications/modelling_e2e
Jun-te Kim 17b9ae08eb Hold one DB connection per modelling_e2e invocation
The modelling_e2e Lambda held up to ~4 concurrent Postgres connections per
invocation: the read Session stayed open across the write loop (the catalogue
was queried live and overrides were read per-Property), each per-Property Unit
of Work opened a second, and the TaskOrchestrator ran on its own NullPool
engine — so the pool needed pool_size=2 + max_overflow=1 just for the modelling
work. Under 32 concurrent containers that approached RDS max_connections.

Restructure the handler to read everything up front — overrides, Scenario, an
in-memory catalogue snapshot, and stored Solar — through one short-lived read
Session, close it, then write each Property in a sequential Unit of Work. The
read and write Sessions no longer overlap, so the engine drops to pool_size=1,
max_overflow=0. Fold the orchestrator onto the same pooled engine: its repos
commit on every save, releasing the connection between bookkeeping calls, so it
holds none during the work. One invocation now uses one connection at a time.

The catalogue becomes a per-invocation snapshot (MaterialSnapshotRepository),
mirroring ProductPostgresRepository.get exactly — same drift mapping, lowest-id
pick, and errors — but priced after the Session closes. Transaction isolation
is preserved: per-Property writes and orchestrator bookkeeping keep their own
independent transactions, just drawn sequentially from a single connection.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 16:58:21 +00:00
..
local_handler remove smoke tests from gh workflows 2026-06-24 11:35:30 +00:00
Dockerfile move handler stuff to applications directory 2026-06-22 13:35:23 +00:00
errors.py Fail unmodellable properties with a specific, debuggable error 🟥 2026-06-24 14:10:55 +00:00
handler.py Hold one DB connection per modelling_e2e invocation 2026-06-24 16:58:21 +00:00
modelling_e2e_trigger_body.py process multiple properties in one message 2026-06-22 15:46:18 +00:00
requirements.txt various fixes 2026-06-22 14:21:52 +00:00