Model/docs/adr/0065-modelling-asserts-baseline-coherence-rather-than-anchoring.md
Khalim Conn-Kowlessar 735a3a1c8e Record the banding and baseline-coherence decisions 🟪
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 16:04:49 +00:00

4.3 KiB
Raw Blame History

status
accepted (extends ADR-0002, ADR-0004; composes with ADR-0011)

Modelling asserts baseline coherence rather than anchoring to the persisted Effective

The Baseline stage persists Effective Performance — what the app reports as a Property's current state, and what a Plan's post_* figures are read against (ADR-0002 / ADR-0004). The Modelling stage does not read that row: it re-scores the Effective EPC itself, because it needs a continuous score and a SapResult to price bills from (ADR-0011 forbids an in-memory hand-off between stages).

Both stages hydrate the same aggregate through the same repo and call the same calculator, so they normally agree — measured across portfolios 796 and 824, mean divergence is 0.008 SAP over 25,838 measure-free plans. But nothing enforces that, and when they drift the failure is silent and user-visible: the app subtracts one stage's baseline from the other's post score and reports a gain nobody modelled. Property 749719 displayed "+9.65 SAP" on a plan whose own CO2, bill and consumption savings were all exactly 0.0 — the plan was empty and correct; the +9.65 was post_sap (72.65, modelling's baseline) minus effective_sap (63, the Baseline stage's). Two baselines, subtracted.

#1655 originally proposed anchoring: read the persisted Effective and use its effective_sap_score as the Optimiser's baseline. We rejected that.

Decided with Khalim, 2026-07-21, while working #1652 / #1655.

Decision

Modelling keeps its own re-score, and checks it against the persisted Effective Performance. A divergence beyond rounding is logged as an error; it is never silently reconciled.

  • _check_baseline_coherence(property_id, modelled_baseline_sap, persisted) runs once per Property, off the Plan's already-computed baseline Score — the baseline picture is scenario-independent, and re-scoring a dwelling per Property purely to check it would not pay for itself across a 31k batch.
  • Tolerance is 1.0 SAP. The persisted figure is the rounded integer, so up to half a point is rounding alone (ADR-0064); firing at 0.5 would be noise.
  • A missing baseline row warns and continues, modelling against the re-score. A data gap should degrade one Property, not abort a portfolio; ADR-0012 reserves the abort for a load-bearing calculator raise.

Why not anchor

  • It would break a different coherence to fix this one. A Score carries CO2 and primary energy beside SAP, and Bills are derived from the same SapResult. Overwriting only the SAP with the persisted integer leaves a Plan whose SAP disagrees with its own carbon, energy and bill figures.
  • It hides the defect instead of surfacing it. A divergence means the two stages scored different pictures — a real fault upstream. Anchoring makes the numbers agree without making them right, and removes the evidence.
  • The figure being anchored to may itself be wrong. #1656 documents SAP 10.2 certs where the persisted Effective sits 610 points below the accredited lodged rating. Hard-wiring Modelling to that number would propagate a known error into every Plan's post_*.

Consequences

  • Plan post_* figures are unchanged. This adds a signal, not an arithmetic change — no re-modelled figure moves because of this ADR.
  • A future drift surfaces as an error naming the Property and both figures, instead of as a phantom gain in a board pack.
  • The guarantee is detection, not prevention: a divergent Property still gets a Plan, and the app will still show the mismatched gain until the underlying cause is fixed. Making it fatal is a live option if the error proves rare enough in practice.
  • Paired with a HIGH costed-plan-without-real-gain anomaly check, closing the auditor's coverage hole: plan-score-below-baseline fired only on a drop beyond 0.5 and zero-works-post-differs only on £0 plans, so a costed plan landing flat against its baseline — the 112 "no real improvement" homes in 796 — was caught by neither.
  • The divergence this was built for was not reproducible at the time of writing: the properties that showed it now agree with the persisted baseline, most likely resolved by intervening calculator fixes. This is a guard against recurrence, not a fix for live breakage.