From 54acc0388bfdd996d32e043b5ccfeb5e24a6f3b8 Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Mon, 29 Jun 2026 16:08:17 +0000 Subject: [PATCH] =?UTF-8?q?docs(audit):=20record=20stale-default-plan=20ex?= =?UTF-8?q?pectation=20+=20default-plan-only=20rule=20=F0=9F=9F=AA?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Portfolio-796/scenario-1268 audit found the bulk of MEDIUM/HIGH anomalies (already-meets-goal-with-works, zero-works-post-differs, plan-score-below- baseline, and the non-fuel-switch slice of negative-bill-savings) trace to one root cause: the persisted default plan is stale relative to the live model, so it resolves on re-model rather than a code change. Confirmed via run_modelling_e2e on three samples (stored vs live plans differ wholesale). No new check added — the staleness is already surfaced by the existing checks and addressed by the override-aware-rebaseline / persistence-fidelity work, so a new check would only re-flag known divergence. Instead record the expectation and the "audit the default plan only" rule in the skill Notes so the next reviewer starts ahead. References kept to durable docs/adr (no PR numbers). Co-Authored-By: Claude Opus 4.8 (1M context) --- .claude/skills/audit-ara-portfolio/SKILL.md | 24 ++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.claude/skills/audit-ara-portfolio/SKILL.md b/.claude/skills/audit-ara-portfolio/SKILL.md index 6621a9f3..6886a93f 100644 --- a/.claude/skills/audit-ara-portfolio/SKILL.md +++ b/.claude/skills/audit-ara-portfolio/SKILL.md @@ -114,9 +114,27 @@ durable, compounding output of every audit. ## Notes - Read-only on the DB. `run_modelling_e2e` is a dry run. +- **Audit the default plan only** — every check, and every characterisation + query, must filter `plan.is_default = TRUE` (the FE-shown plan). The runner's + `_QUERY`/`_ROLLUP_QUERY` already do; keep ad-hoc SQL consistent or the counts + mix superseded plans into the picture. +- **The stored default plan can be STALE vs the live model** — the persisted plan + is the output of an *earlier* modelling run; `run_modelling_e2e` re-models + against current logic + live EPC/solar. A stored-vs-live gap is the single + biggest driver of MEDIUM/HIGH anomalies on a not-yet-re-modelled portfolio, and + it spans more checks than the one below names. The fix is operational + (**re-model the portfolio, then re-audit**), not a code change — confirm a + sample with `run_modelling_e2e` before debugging the calculator. - **Expected, not bugs** (until the override-aware-rebaseline + persistence-fidelity - PR deploys and the portfolio is re-modelled): much of `zero-works-post-differs` - and `plan-score-below-baseline` is the pre-fix baseline-vs-plan divergence and - should shrink after re-model — note it, don't re-debug it. + work — see `docs/adr/` — is re-modelled into the portfolio): much of + `zero-works-post-differs`, `plan-score-below-baseline`, `already-meets-goal-with-works`, + and the non-fuel-switch slice of `negative-bill-savings` is the same stale + stored-plan-vs-live divergence and should shrink after re-model — note it, + don't re-debug it. Confirm a sample's stored plan differs from `run_modelling_e2e` + (stored `air_source_heat_pump` for tens of £k where the live plan is a cheap + profitable `solar_pv`; stored works on a property the live model leaves at £0 + because it already meets goal). The `negative-bill` certs that DO carry an + `air_source_heat_pump` are a genuine gas→electric fuel-switch (bills rise at + current price ratios) — expected by design. - Adding a check is one decorated `(PropertyAudit) -> Optional[str]` function in `scripts/audit/anomalies.py`; see its module docstring.