A focused sibling to audit-ara-portfolio: that skill audits baselines/plans/SAP;
this one audits the *recommendations themselves* — why a measure was or wasn't
offered. Motivated by the portfolio-814 review (Khalim's HHRSH-on-community-
heating, missing-HHRSH, missing-secondary-heating-removal, and a neighbour split).
Adds:
- .claude/skills/find-weird-recommendations/SKILL.md — scan -> neighbour scan ->
live re-model deep-dive -> root-cause -> codify, with a seeded known-bug
catalogue and the query-safety rules inherited from audit-ara-portfolio.
- scripts/audit/anomalies.py: new `plan-stops-short-of-goal` HIGH check — the
default plan ends below the goal band on an unlimited-budget scenario (the
deterministic worklist for "why didn't this get recommended X"). Adds
scenario_budget to the bundle/query so budget-capped scenarios are excluded.
- scripts/audit/neighbour_divergence.py: groups a portfolio by (postcode,
property_type, built_form) and flags effective-SAP outliers vs the cohort
median. Never touches the 26m-row recommendation table, so it is safe
portfolio-wide.
- Tests for both (12 passing).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The `recommendation` table (~26m rows) has no index on `plan_id`, so any query
reaching it via `plan_id` — including the audit's own rollup — seq-scans the
whole table and saturates the shared DB (it blocked the DB during the
portfolio-796 audit). Make the audit safe-by-default:
- statement_timeout (120s) on the audit connection — a hard ceiling so a bad
plan aborts instead of hammering the DB.
- The recommendation rollup (the two solar checks) is now opt-in via
--with-recommendations, and EXPLAIN-gated: it refuses to run (raising
RecommendationScanError) when the plan contains a Seq Scan on recommendation,
which it does on any large portfolio until idx_recommendation_plan_id exists.
- SKILL.md documents the plan_id-no-index trap, the reach-via-property_id /
EXPLAIN-first / confirm-with-user rules, and the index as the real fix.
Verified on 796/1268: default run is bounded and completes (2,952 anomalies over
31,919 properties); --with-recommendations aborts pre-scan portfolio-wide but is
allowed for a single property.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add Phase 6 (self-improve) to audit-ara-portfolio: when a run confirms a
novel systematic problem, codify it as a check — gated on systematic (>=5
props, root-caused), not-already-covered, and /grill-me-pressure-tested.
Each check records provenance (motivating cause + example properties) so the
registry stays sharp and compounds every run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Script takes an optional --scenario to restrict to one scenario's plans. New
skill drives the full loop: run the deterministic scan, review groups,
deep-dive samples via run_modelling_e2e, characterise sub-classes, and
cross-reference open PRs/ADRs — then proposes new checks to codify.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
