From 735a3a1c8e236d2ed9deba728b2e5c09ac109f9b Mon Sep 17 00:00:00 2001 From: Khalim Conn-Kowlessar Date: Tue, 21 Jul 2026 16:04:49 +0000 Subject: [PATCH] =?UTF-8?q?Record=20the=20banding=20and=20baseline-coheren?= =?UTF-8?q?ce=20decisions=20=F0=9F=9F=AA?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 4.8 (1M context) --- CONTEXT.md | 8 +- ...rship-is-judged-on-the-published-rating.md | 70 +++++++++++++++++ ...aseline-coherence-rather-than-anchoring.md | 76 +++++++++++++++++++ 3 files changed, 150 insertions(+), 4 deletions(-) create mode 100644 docs/adr/0064-band-membership-is-judged-on-the-published-rating.md create mode 100644 docs/adr/0065-modelling-asserts-baseline-coherence-rather-than-anchoring.md diff --git a/CONTEXT.md b/CONTEXT.md index 047087e97..c6b900858 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -28,8 +28,8 @@ The date an EPC was lodged with the government register; used to identify the mo _Avoid_: assessment date, submission date **EPC Band**: -A single letter A–G representing a property's current or potential energy efficiency rating. -_Avoid_: energy rating, EPC grade, EPC score +A single letter A–G representing a property's current or potential energy efficiency rating. A dwelling is in a band by its **published rating** — the continuous SAP rounded **half up** to the integer printed on the certificate (SAP 10.2 §13; Python's built-in `round` is half-to-*even* and must not be used for this). Anything holding a continuous score therefore judges band membership via `Epc.from_sap_continuous`, and any "has it reached band X?" test compares against `Epc.sap_lower_bound_continuous()` — the integer floor **less half a point** (C → 68.5), which is where the published rating enters the band. Comparing a continuous score against the raw integer floor makes a dwelling that already publishes in the band read as a fraction short (ADR-0064). +_Avoid_: energy rating, EPC grade, EPC score; comparing a continuous SAP against `sap_lower_bound()` **Schema Type**: The versioned RdSAP or SAP schema that describes the structure of an EPC's raw data (e.g. `RdSAP-Schema-21.0.1`). @@ -156,7 +156,7 @@ The SAP / EPC Band / carbon emissions / Primary Energy Intensity recorded on the _Avoid_: original performance, raw EPC values, recorded baseline **Effective Performance**: -The SAP / EPC Band / carbon emissions / Primary Energy Intensity the modelling pipeline actually scored against — equal to Lodged Performance when no Rebaselining trigger fires, replaced by **SAP10 Calculation** output (the deterministic `Sap10Calculator`, which superseded the old ML-API rebaseliner; an ML residual head over the calculator is future — ADR-0009/0013) when triggered. The half of Baseline Performance that says "what we modelled". +The SAP / EPC Band / carbon emissions / Primary Energy Intensity the modelling pipeline actually scored against — equal to Lodged Performance when no Rebaselining trigger fires, replaced by **SAP10 Calculation** output (the deterministic `Sap10Calculator`, which superseded the old ML-API rebaseliner; an ML residual head over the calculator is future — ADR-0009/0013) when triggered. The half of Baseline Performance that says "what we modelled". **Modelling does not read it**: it re-scores the Effective EPC itself (it needs a continuous score and a `SapResult` to bill from, and ADR-0011 forbids an in-memory hand-off between stages), then **asserts** its re-score agrees with this persisted figure and logs an error when it does not. The two are deliberately *not* reconciled — a divergence means the stages scored different pictures, and anchoring the Plan's SAP to this integer would leave it disagreeing with its own carbon, energy and bill figures (ADR-0065). Because a Plan's `post_*` are read against this figure, an unchecked divergence surfaces to the user as a gain nobody modelled. _Avoid_: modelled performance, rebaselined performance (only correct when rebaselining ran), scored values **Calculated SAP10 Performance**: @@ -309,7 +309,7 @@ A "selecting A requires B" edge between **Recommendations**, for couplings that _Avoid_: best-practice measure (legacy term), forced measure **Optimised Package**: -The subset of a Property's Recommendations selected by the Optimiser Service for installation. For an **Increasing EPC** goal the objective is **least-cost-to-target**: the cheapest package that reaches the goal band — so it **stops at the target and does not overshoot** into a higher band, leaving surplus budget unspent. When the target is **unreachable within budget**, it falls back to the **maximum improvement the budget buys** (best effort, below target). With **no budget** it is simply the cheapest package that reaches the target. Reaching the target is judged on the **true whole-package re-score** (ADR-0016), not on summed per-measure scores. (Other goals — Energy Savings, Reducing CO₂ — don't yet set a target and currently maximise improvement within budget; future work.) +The subset of a Property's Recommendations selected by the Optimiser Service for installation. For an **Increasing EPC** goal the objective is **least-cost-to-target**: the cheapest package that reaches the goal band — so it **stops at the target and does not overshoot** into a higher band, leaving surplus budget unspent. When the target is **unreachable within budget**, it falls back to the **maximum improvement the budget buys** (best effort, below target). With **no budget** it is simply the cheapest package that reaches the target. Reaching the target is judged on the **true whole-package re-score** (ADR-0016), not on summed per-measure scores — and on the **published** band (see **EPC Band**): the target is the goal band's floor on the continuous scale, so a dwelling whose published baseline already meets the goal is **already at target** and gets an empty, £0 package rather than a marginal top-up (ADR-0064). (Other goals — Energy Savings, Reducing CO₂ — don't yet set a target and currently maximise improvement within budget; future work.) _Avoid_: selected measures, default measures, optimal solution, recommended bundle **Measure Type**: diff --git a/docs/adr/0064-band-membership-is-judged-on-the-published-rating.md b/docs/adr/0064-band-membership-is-judged-on-the-published-rating.md new file mode 100644 index 000000000..0615c76c7 --- /dev/null +++ b/docs/adr/0064-band-membership-is-judged-on-the-published-rating.md @@ -0,0 +1,70 @@ +--- +status: accepted (extends ADR-0016; constrains ADR-0062's Increasing EPC path) +--- + +# Band membership is judged on the published rating, not on a raw continuous score + +A dwelling is in an EPC band by the **integer rating printed on its +certificate** — the continuous SAP rounded to the nearest whole point. The +Optimiser, however, works in continuous SAP, and judged "has this dwelling +reached band X?" by comparing that continuous score against the band's +**integer** floor (`Epc.sap_lower_bound()`, band C → `69.0`). + +Those two rules disagree in the half-point window below every band floor. A +dwelling at a continuous 68.6 publishes as SAP 69 and **is** band C, but read as +0.4 short of an integer floor of 69 — so the Optimiser bought the cheapest +measure available to close a gap that did not exist. In portfolio 796 / scenario +1268 ("Reach EPC C") this sold **630 homes already published at band C** about +one SAP point of works each, £282k in total; portfolio 824 / scenario 1278 added +172 more at £56k. The measures were real and were installed against real +budgets; the gap they closed was an artefact of comparing two different scales. + +A second, smaller disagreement sat underneath it: `sap_rating_integer` published +the rating using Python's built-in `round`, which is half-to-**even**, so an +exact `x.5` fell to `x` for even `x` — at a band floor, publishing a whole band +low. SAP rounds half **up**, the convention the mapper already followed +(`_round_half_up_2dp`). + +Found and fixed while working #1652 / #1654 with Khalim, 2026-07-21. + +## Decision + +**One rule decides band membership everywhere: round the continuous SAP half up, +then band the integer. Anything comparing a continuous score to a band judges it +on that basis.** + +- **`sap_rating_integer` rounds half up** (`Decimal` + `ROUND_HALF_UP`), matching + SAP 10.2 §13 and the mapper's existing convention. A continuous 68.5 publishes + as SAP 69. +- **`Epc.sap_lower_bound_continuous()`** is the band floor expressed on the + continuous scale — the integer floor less half a point (C → 68.5), i.e. the + lowest un-rounded score whose *published* rating is in the band. +- **`_target_sap` returns the continuous floor.** The Optimiser's target must be + in the currency of everything it is compared against. This single change fixes + all four comparison sites (`target_gain`, the two reached-target checks, and + `_repair_to_target`'s loop) without touching optimiser control flow. +- **`Epc.from_sap_continuous()`** bands a continuous score as it would be + published, replacing the repeated `from_sap_score(round(x))` idiom at every + call site (`Plan.post_epc_rating`, `Plan.baseline_epc_rating`, the plan table + harness, the anomaly auditor). +- Scoped to the **SAP-goal path only**: goal-aligned CO2 / bill objectives pass + `target_sap=None` (ADR-0062) and are untouched. + +## Consequences + +- A dwelling whose **published** baseline already meets the goal band gets a + **£0, measure-free** plan. Dwellings genuinely below the band still get plans + that reach it — the target moved by half a point, not the ambition. +- Re-modelling 796/1268 should drop ~630 plans and £282,763 of `cost_of_works` + (642 measures), and 824/1278 ~172 plans and £56,226 (176 measures). Both are + unbudgeted, so no budget interaction confounds the change. The remaining spend + is an upper bound: a lower target may also let genuinely-below dwellings stop + one measure earlier. +- Half-up publishing can move a rating by one point only when the continuous + score is *exactly* `x.5`. No fixture in the accuracy corpus is, and the full + SAP suite (2,072 tests, including the Elmhurst worksheet pins) passed + unchanged — but the corpus should be re-checked if the calculator's arithmetic + ever shifts onto exact halves. +- `already-meets-goal-with-works` moves MEDIUM → HIGH in the anomaly auditor. + It was firing correctly all along and was bucketed as expected stale-plan + noise, which is how this survived: a check nobody believes is not a check. diff --git a/docs/adr/0065-modelling-asserts-baseline-coherence-rather-than-anchoring.md b/docs/adr/0065-modelling-asserts-baseline-coherence-rather-than-anchoring.md new file mode 100644 index 000000000..21f3f25a6 --- /dev/null +++ b/docs/adr/0065-modelling-asserts-baseline-coherence-rather-than-anchoring.md @@ -0,0 +1,76 @@ +--- +status: accepted (extends ADR-0002, ADR-0004; composes with ADR-0011) +--- + +# Modelling asserts baseline coherence rather than anchoring to the persisted Effective + +The Baseline stage persists **Effective Performance** — what the app reports as a +Property's current state, and what a Plan's `post_*` figures are read against +(ADR-0002 / ADR-0004). The Modelling stage does not read that row: it re-scores +the Effective EPC itself, because it needs a continuous score and a `SapResult` +to price bills from (ADR-0011 forbids an in-memory hand-off between stages). + +Both stages hydrate the same aggregate through the same repo and call the same +calculator, so they normally agree — measured across portfolios 796 and 824, +mean divergence is **0.008 SAP** over 25,838 measure-free plans. But nothing +*enforces* that, and when they drift the failure is silent and user-visible: the +app subtracts one stage's baseline from the other's post score and reports a gain +nobody modelled. Property 749719 displayed **"+9.65 SAP"** on a plan whose own +CO2, bill and consumption savings were all exactly `0.0` — the plan was empty and +correct; the +9.65 was `post_sap` (72.65, modelling's baseline) minus +`effective_sap` (63, the Baseline stage's). Two baselines, subtracted. + +#1655 originally proposed **anchoring**: read the persisted Effective and use its +`effective_sap_score` as the Optimiser's baseline. We rejected that. + +Decided with Khalim, 2026-07-21, while working #1652 / #1655. + +## Decision + +**Modelling keeps its own re-score, and checks it against the persisted Effective +Performance. A divergence beyond rounding is logged as an error; it is never +silently reconciled.** + +- **`_check_baseline_coherence(property_id, modelled_baseline_sap, persisted)`** + runs once per Property, off the Plan's already-computed `baseline` Score — the + baseline picture is scenario-independent, and re-scoring a dwelling per + Property purely to check it would not pay for itself across a 31k batch. +- **Tolerance is 1.0 SAP.** The persisted figure is the *rounded* integer, so up + to half a point is rounding alone (ADR-0064); firing at 0.5 would be noise. +- **A missing baseline row warns and continues**, modelling against the + re-score. A data gap should degrade one Property, not abort a portfolio; + ADR-0012 reserves the abort for a load-bearing calculator raise. + +### Why not anchor + +- **It would break a different coherence to fix this one.** A `Score` carries + CO2 and primary energy beside SAP, and Bills are derived from the same + `SapResult`. Overwriting only the SAP with the persisted integer leaves a Plan + whose SAP disagrees with its own carbon, energy and bill figures. +- **It hides the defect instead of surfacing it.** A divergence means the two + stages scored different pictures — a real fault upstream. Anchoring makes the + numbers *agree* without making them *right*, and removes the evidence. +- **The figure being anchored to may itself be wrong.** #1656 documents SAP 10.2 + certs where the persisted Effective sits 6–10 points below the accredited + lodged rating. Hard-wiring Modelling to that number would propagate a known + error into every Plan's `post_*`. + +## Consequences + +- Plan `post_*` figures are unchanged. This adds a signal, not an arithmetic + change — no re-modelled figure moves because of this ADR. +- A future drift surfaces as an error naming the Property and both figures, + instead of as a phantom gain in a board pack. +- The guarantee is **detection, not prevention**: a divergent Property still + gets a Plan, and the app will still show the mismatched gain until the + underlying cause is fixed. Making it fatal is a live option if the error + proves rare enough in practice. +- Paired with a HIGH `costed-plan-without-real-gain` anomaly check, closing the + auditor's coverage hole: `plan-score-below-baseline` fired only on a *drop* + beyond 0.5 and `zero-works-post-differs` only on £0 plans, so a **costed** plan + landing flat against its baseline — the 112 "no real improvement" homes in 796 + — was caught by neither. +- The divergence this was built for was **not reproducible at the time of + writing**: the properties that showed it now agree with the persisted baseline, + most likely resolved by intervening calculator fixes. This is a guard against + recurrence, not a fix for live breakage.