Model/docs/adr/0055-modelling-run-batches-attach-to-the-app-owned-task.md

A Modelling Run's batch sub_tasks are pre-created by the distributor and attach to the app-owned task; their status is a record, not retry logic

Status

accepted

Context

The Ara app is gaining a filter-scoped modelling trigger: POST /v1/modelling/trigger-run accepts a portfolio, a set of scenario ids, and property-group filters, and fans one plan-generation job per (Scenario, Property) out to the existing modelling_e2e workers. The app creates the task before calling (service modelling_run, status in progress, the full request JSON in the new tasks.inputs column) and renders progress as count(completed)/count(all) over the task's sub_tasks.

Two prior patterns conflicted:

• The legacy /v1/plan/trigger route creates the task and its per-chunk sub_tasks in the API, passing task_id/subtask_id in each SQS message.
• The modelling_e2e worker's @task_handler creates its own Task per SQS message and fans per-property child sub_tasks under it (TaskOrchestrator.run_subtasks); nothing can attach to a caller's task.

The parent-status roll-up (Task.recalculate_from_subtasks) had three properties that break shared-parent fan-out: a complete+waiting mix rolls up to waiting (a status the app does not render); SubTask.start() refuses to start from failed (breaking any redelivery/re-run of a failed batch); and TaskOrchestrator._cascade short-circuits on a FAILED parent, making failure permanently sticky.

Decisions

1. The distributor pre-creates one sub_task per SQS message at accept time

One task spans the whole run. The distributor resolves the filters, batches the properties (50 per message, postcode-grouped so the workers' prediction cohort cache keeps paying, one message per (scenario, batch)), bulk-creates one waiting sub_task per message under the app's task, and sends each message carrying task_id, subtask_id, and the batch payload. Each sub_task's inputs is the exact message payload — the run's precise what-ran record, and a failed batch's re-run recipe (re-send its own inputs).

The progress denominator is therefore fixed and correct the moment the endpoint returns 202, and it counts batches, not properties.

2. The worker gains an attach mode; the script path is untouched

When a message carries task_id + subtask_id, task_handler creates no Task and the handler runs the whole batch under the supplied sub_task — no per-property child sub_tasks. Without those fields, current behaviour (own task, per-property children) is unchanged, so scripts/trigger_modelling_e2e_sqs.py keeps working.

3. Sub_task status is a database record, not retry fuel

Per-property failures inside a batch (unresolvable type, degenerate prediction, …) stay isolated as today — the surviving properties model and persist. But if any property failed, the batch sub_task is marked failed with outputs = {succeeded: n, failed: [{property_id, error}]}, and the parent task rolls up failed. The lambda still succeeds to SQS: a failed sub_task never triggers redelivery. Recovery is deliberate — fix the cause, re-send the sub_task's own inputs, the sub_task completes, and the parent task un-fails to complete. Batch-level crashes (OOM, timeout, unhandled infra errors) keep normal SQS redelivery semantics; that is infrastructure retry, not status-driven retry.

4. The roll-up rules change to make failure recoverable and progress honest

• A complete/failed + waiting mix rolls up in progress, not waiting.
• SubTask.start() may start from failed (redelivery and re-runs).
• _cascade no longer short-circuits on a FAILED parent: the parent status is always recomputed from the children, so completing a re-run batch flips the task from failed to complete.

5. Contract guards

• 409 if the task already has sub_tasks (double-submit / blind retry protection — the app checks progress instead of re-POSTing).
• 400 if the filters resolve to zero properties (a zero-sub_task task could never complete; the app's preview shows the same count first).
• 4xx if any scenario does not belong to the portfolio. Scenarios are immutable after creation (rename-only; deletable only while no Plans reference them), so workers read the live Scenario row safely.
• Pipeline flags are pinned by the endpoint, not the caller: refetch_epc=True, repredict_epc=True, refetch_solar=True, dry_run=False. Note refetch_solar is a misnomer: it only gates the Google fetch for UPRNs with no stored row; stored Solar is never re-fetched.

Consequences

• The app's progress bar denominator is batch count (≈ ceil(N/50) × scenarios), not the preview's property count. Property-level failure detail is summed from sub_task outputs.
• A 10k-property run whose single bad batch is fixed and re-run ends complete — no permanently red runs over recoverable data issues.
• The roll-up changes are global to the task machinery; other task_handler lambdas keep their semantics (their per-message tasks have no shared parent), but FAILED-parent recomputation now costs one children read.
• FastAPI needs the modelling_e2e queue wired: MODELLING_E2E_SQS_URL from the modelling_e2e remote state (outputs already exported), its ARN in the fastapi-sqs-send policy, and deploy_terraform.yml's fast_api_lambda job ordered after modelling_e2e_lambda.