feat(modelling-runs): run config on tasks.inputs; backend contract updates

- The run's config now lives on the task row's new `inputs` column
  (migration PR #361) instead of a config subtask — sub_tasks belong
  entirely to the distributor's execution work.
- Distributor path corrected to /v1/modelling/trigger-run.
- Sub-task granularity is batches, not properties: history shows progress
  as a percentage under the status chip; the Plans column always shows the
  plan count (batch counts are never comparable to property/plan numbers).
- ADR-0008 amended accordingly.

Live verification of the task write waits for PR #361 to be applied.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Khalim Conn-Kowlessar 2026-07-07 10:46:29 +00:00
parent 10888e3fff
commit 83117e90c5
6 changed files with 84 additions and 70 deletions

View file

@ -26,17 +26,22 @@ explicit marker when trigger wiring lands. This is that wiring.
- **A Modelling run is recorded as an app-created task** — the BulkUpload
trigger convention (`triggerAddressMatching`): Next.js inserts the `tasks`
row (service `modelling_run`, `source_id` = portfolio, status `waiting`)
plus one config `sub_task` whose `inputs` JSON carries
`{scenario_ids, filters, previewed_property_count, triggered_by}`, then
passes `task_id` to the distributor, which attaches its execution subtasks
to the same task. **No new table, no migration.** Run history and the
per-scenario in-flight badge read the portfolio's own `modelling_run` tasks
(portfolio-scoped, a handful of rows) and parse the app-authored inputs
JSON; execution progress ("61 of 104 ready") falls out of the existing
subtask-count summary pattern.
- **The distributor receives `{task_id, portfolio_id, scenario_ids, filters}`**
— filters, never property-id lists. The payload stays tiny at any portfolio
size; the config subtask is the durable copy of the same request.
whose **`inputs` column** (added for this — mirrors `sub_task.inputs`)
carries `{portfolio_id, scenario_ids, filters, previewed_property_count,
triggered_by}` as JSON, then passes `task_id` to the distributor, which
attaches its execution sub-tasks to the same task. **One new column, no new
table.** Run history and the per-scenario in-flight badge read the
portfolio's own `modelling_run` tasks (portfolio-scoped, a handful of rows)
and parse the app-authored inputs JSON; execution progress falls out of the
existing subtask-count summary pattern. Sub-task granularity is the
distributor's **batches**, so progress is shown as a percentage — batch
counts are internally consistent but never comparable to property or plan
numbers.
- **The distributor (`POST /v1/modelling/trigger-run`) receives
`{task_id, portfolio_id, scenario_ids, filters}`** — filters, never
property-id lists. The payload stays tiny at any portfolio size; the task's
`inputs` is the durable copy of the same request (plus provenance the
distributor doesn't need).
- **Dispatch failure marks the task `failed`** (an improvement on the
bulk-upload flow, which leaves failed triggers at `waiting`), so history
never shows a ghost run as pending.
@ -63,7 +68,7 @@ explicit marker when trigger wiring lands. This is that wiring.
run and the execution share one record by construction. Scenario badges
gain a third derived state — "Modelling in progress" — and a per-portfolio
run history lists each run's filters, counts, initiator and derived status
with subtask-count progress.
with batch-based percentage progress.
- **Concurrent runs warn, never block**: triggering a scenario with a run in
flight shows who started it and when, but proceeds. Plans append and
latest-per-property wins on read, so overlap is safe; a hard lock could leak

View file

@ -18,6 +18,11 @@ export const tasks = pgTable("tasks", {
sourceId: text("source_id"), // identifier for the source (e.g., portfolio_id value)
// The task's request inputs, mirroring sub_task.inputs (JSON as text; could
// later change to JSONB). For a modelling run this is the run's config —
// portfolio id, scenario ids, filters (ADR-0008).
inputs: text("inputs"),
updatedAt: timestamp("updated_at", { precision: 6, withTimezone: true })
.defaultNow()
.$onUpdate(() => new Date())

View file

@ -434,21 +434,7 @@ export default function RunModellingClient({
{r.previewedPropertyCount.toLocaleString()}
</td>
<td className="border-b border-gray-100 px-3 py-2.5 align-top tabular-nums">
{r.status.state === "in_progress" ? (
<>
<span className="whitespace-nowrap text-[12px] text-gray-500">
{r.status.ready.toLocaleString()} of {r.status.total.toLocaleString()} ready
</span>
<span className="mt-1 block h-1 w-24 overflow-hidden rounded-full bg-gray-100">
<span
className="block h-full rounded-full bg-brandmidblue"
style={{ width: `${Math.round((100 * r.status.ready) / Math.max(1, r.status.total))}%` }}
/>
</span>
</>
) : (
r.planCount.toLocaleString()
)}
{r.planCount.toLocaleString()}
</td>
<td className="border-b border-gray-100 px-3 py-2.5 align-top">
<span
@ -462,6 +448,21 @@ export default function RunModellingClient({
>
{STATUS_LABELS[r.status.state]}
</span>
{/* Work is chunked into batches, so show a percentage
the counts never line up with property/plan numbers */}
{r.status.state === "in_progress" && (
<>
<span className="mt-1 block text-[11px] tabular-nums text-gray-400">
{Math.round((100 * r.status.ready) / Math.max(1, r.status.total))}% done
</span>
<span className="mt-0.5 block h-1 w-20 overflow-hidden rounded-full bg-gray-100">
<span
className="block h-full rounded-full bg-brandmidblue"
style={{ width: `${Math.round((100 * r.status.ready) / Math.max(1, r.status.total))}%` }}
/>
</span>
</>
)}
</td>
</tr>
))}

View file

@ -44,7 +44,7 @@ describe("normaliseRunFilters", () => {
});
describe("buildRunRecord", () => {
it("produces the app-authored task row and config-subtask inputs (ADR-0008)", () => {
it("produces the app-authored task row carrying the run config as its inputs (ADR-0008)", () => {
const record = buildRunRecord({
portfolioId: "814",
scenarioIds: ["12", "34"],
@ -58,27 +58,33 @@ describe("buildRunRecord", () => {
source: "portfolio_id",
sourceId: "814",
status: "waiting",
inputs: JSON.stringify({
portfolio_id: 814,
scenario_ids: [12, 34],
filters: { postcodes: ["B93 8SU"], property_types: ["House"] },
previewed_property_count: 214,
triggered_by: "khalim@domna.homes",
}),
});
expect(record.configInputs).toEqual({
// What the distributor receives — provenance fields stay on the task only
expect(record.dispatchPayload).toEqual({
portfolio_id: 814,
scenario_ids: [12, 34],
filters: { postcodes: ["B93 8SU"], property_types: ["House"] },
previewed_property_count: 214,
triggered_by: "khalim@domna.homes",
});
});
});
describe("parseRunConfig", () => {
it("round-trips a stored run record back to the app's shape", () => {
const { configInputs } = buildRunRecord({
const { task } = buildRunRecord({
portfolioId: "814",
scenarioIds: ["12"],
filters: { builtForms: ["Detached", "Unknown"] },
previewedPropertyCount: 57,
triggeredBy: "priya@domna.homes",
});
expect(parseRunConfig(JSON.stringify(configInputs))).toEqual({
expect(parseRunConfig(task.inputs)).toEqual({
scenarioIds: ["12"],
filters: { builtForms: ["Detached", "Unknown"] },
previewedPropertyCount: 57,

View file

@ -168,9 +168,9 @@ export interface RunConfigInputs {
}
/**
* The rows a Modelling run persists (ADR-0008): an app-authored task (the
* BulkUpload trigger convention) plus the config-subtask inputs that make the
* run's request durable and history-readable.
* The row a Modelling run persists (ADR-0008): an app-authored task whose
* `inputs` carries the run's config the durable, history-readable record.
* Sub-tasks are the distributor's: per-property execution work.
*/
export function buildRunRecord(req: RunRequest): {
task: {
@ -179,13 +179,22 @@ export function buildRunRecord(req: RunRequest): {
source: "portfolio_id";
sourceId: string;
status: "waiting";
inputs: string;
};
configInputs: RunConfigInputs;
/** What the distributor receives — provenance fields stay on the task. */
dispatchPayload: Omit<RunConfigInputs, "previewed_property_count" | "triggered_by">;
} {
const filters: RunConfigInputs["filters"] = {};
if (req.filters.postcodes) filters.postcodes = req.filters.postcodes;
if (req.filters.propertyTypes) filters.property_types = req.filters.propertyTypes;
if (req.filters.builtForms) filters.built_forms = req.filters.builtForms;
const configInputs: RunConfigInputs = {
portfolio_id: Number(req.portfolioId),
scenario_ids: req.scenarioIds.map(Number),
filters,
previewed_property_count: req.previewedPropertyCount,
triggered_by: req.triggeredBy,
};
return {
task: {
taskSource: `Modelling run ${req.scenarioIds.length} scenario${req.scenarioIds.length === 1 ? "" : "s"}`,
@ -193,13 +202,12 @@ export function buildRunRecord(req: RunRequest): {
source: "portfolio_id",
sourceId: req.portfolioId,
status: "waiting",
inputs: JSON.stringify(configInputs),
},
configInputs: {
portfolio_id: Number(req.portfolioId),
scenario_ids: req.scenarioIds.map(Number),
dispatchPayload: {
portfolio_id: configInputs.portfolio_id,
scenario_ids: configInputs.scenario_ids,
filters,
previewed_property_count: req.previewedPropertyCount,
triggered_by: req.triggeredBy,
},
};
}

View file

@ -13,13 +13,11 @@ import {
} from "./model";
import { builtFormTypeSql, propertyTypeSql } from "@/lib/services/epcSources";
// The config subtask that makes a run's request durable (ADR-0008); execution
// subtasks attached by the distributor carry other service values.
export const RUN_CONFIG_SERVICE = "modelling_run_config";
// Distributor entry point on the Model backend. It fans the run out to
// workers, attaching execution subtasks to the task_id we pass (ADR-0008).
const DISTRIBUTOR_ENDPOINT = "/v1/plan/trigger-run";
// workers, attaching execution sub-tasks to the task_id we pass (ADR-0008).
// NOTE: sub-task granularity is BATCHES, not properties — progress counts are
// internally consistent but never comparable to the previewed property count.
const DISTRIBUTOR_ENDPOINT = "/v1/modelling/trigger-run";
type BackendResult<T> = { ok: true; data: T } | { ok: false; status: number; message: string };
@ -62,8 +60,9 @@ export type TriggerRunOutcome =
| { kind: "dispatch_failed"; status: number; message: string };
/**
* Record + dispatch a Modelling run: app-authored task and config subtask
* (BulkUpload trigger convention), then hand task_id to the distributor. A
* Record + dispatch a Modelling run: one app-authored task whose `inputs`
* carries the run's config (BulkUpload trigger convention), then hand
* task_id to the distributor sub_tasks are its, one per property. A
* failed dispatch marks the task failed so history never shows a ghost run
* as pending (ADR-0008).
*/
@ -94,23 +93,12 @@ export async function triggerModellingRun(args: {
.insert(tasks)
.values({ ...record.task, jobStarted: now })
.returning();
await db.insert(subTasks).values({
taskId: task.id,
status: "waiting",
service: RUN_CONFIG_SERVICE,
inputs: JSON.stringify(record.configInputs),
});
// Minimal dispatch payload — previewed count and triggered_by are app
// provenance, already durable on the config subtask keyed by this task_id.
// provenance, already durable on the task's inputs.
const dispatch = await callBackend(
DISTRIBUTOR_ENDPOINT,
{
task_id: task.id,
portfolio_id: record.configInputs.portfolio_id,
scenario_ids: record.configInputs.scenario_ids,
filters: record.configInputs.filters,
},
{ task_id: task.id, ...record.dispatchPayload },
args.sessionToken,
);
if (!dispatch.ok) {
@ -210,9 +198,10 @@ export interface RunHistoryRow {
}
/**
* Run history for a portfolio: its modelling_run tasks with app-authored
* config parsed back out and progress from execution-subtask counts. One
* grouped query portfolio-scoped, never per-run probes.
* Run history for a portfolio: its modelling_run tasks with the app-authored
* config parsed from task inputs and progress from sub-task counts (one
* sub-task per property, distributor-attached). One grouped query
* portfolio-scoped, never per-run probes.
*/
export async function listModellingRuns(
portfolioId: string,
@ -224,10 +213,10 @@ export async function listModellingRuns(
jobStarted: tasks.jobStarted,
jobCompleted: tasks.jobCompleted,
status: tasks.status,
configInputs: sql<string | null>`max(${subTasks.inputs}) filter (where ${subTasks.service} = ${RUN_CONFIG_SERVICE})`,
total: sql<number>`count(${subTasks.id}) filter (where ${subTasks.service} is distinct from ${RUN_CONFIG_SERVICE})::int`,
completed: sql<number>`count(case when lower(${subTasks.status}) in ('completed','complete') and ${subTasks.service} is distinct from ${RUN_CONFIG_SERVICE} then 1 end)::int`,
failed: sql<number>`count(case when lower(${subTasks.status}) in ('failed','failure','error') and ${subTasks.service} is distinct from ${RUN_CONFIG_SERVICE} then 1 end)::int`,
configInputs: tasks.inputs,
total: sql<number>`count(${subTasks.id})::int`,
completed: sql<number>`count(case when lower(${subTasks.status}) in ('completed','complete') then 1 end)::int`,
failed: sql<number>`count(case when lower(${subTasks.status}) in ('failed','failure','error') then 1 end)::int`,
})
.from(tasks)
.leftJoin(subTasks, eq(subTasks.taskId, tasks.id))