mirror of
https://github.com/Hestia-Homes/Model.git
synced 2026-06-08 11:17:27 +00:00
docs(adr): Bill Derivation (ADR-0014) + calculator goes load-bearing (ADR-0013 amend)
Pin the bills design from a /grill-with-docs session: - ADR-0014: whole-home annual bill from SAP10 Calculation's delivered kWh per end use, re-priced at real Fuel Rates (NOT the calculator's SAP-notional total_fuel_cost_gbp, which is RdSAP Table 32 standardised prices ~half real electricity). Fuel enum + FuelRates + FuelRatesRepository static snapshot; per-section + total flat columns; raise on unpriced fuel (house coal / heat network are the named gaps). - ADR-0013 amendment: the shadow stepping-stone is collapsed — the calculator is load-bearing now. effective=calculated for sap_version<10.2 (StubRebaseliner floor 10.0->10.2); >=10.2 keeps lodged + logs divergence; a strict-raise aborts the batch (load-bearing for bills regardless of version). - CONTEXT: EPC Energy Derivation -> Bill Derivation (no "service" suffix); Baseline Performance energy block = per-end-use kWh + per-section bill + total; Fuel Rates = committed static snapshot; Rebaselining trigger threshold 10.2. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
741993270e
commit
8c2bc08df3
3 changed files with 120 additions and 10 deletions
20
CONTEXT.md
20
CONTEXT.md
|
|
@ -82,11 +82,11 @@ The EpcPropertyData scored by the modelling pipeline for a single Property, deri
|
|||
_Avoid_: modelling EPC, working EPC, resolved EPC, derived EPC
|
||||
|
||||
**Rebaselining**:
|
||||
Re-predicting a Property's SAP score, CO2 emissions, Primary Energy Intensity, space heating kWh, and hot water kWh via **SAP10 Calculation** (the deterministic `Sap10Calculator`, which superseded the old ML-API rebaseliner; an ML residual head over the calculator is future — ADR-0009/0013) so the modelling pipeline scores it against the current SAP10 methodology. Triggered when either (a) the Effective EPC was lodged under a pre-SAP10 schema (`sap_version < 10.0`), so the recorded scores reflect a superseded methodology, or (b) Site Notes / Landlord Overrides changed the physical state of the Property (walls / heating / windows / etc.) so the lodged scores no longer reflect what's installed. Both triggers may fire together. Produces Effective Performance; Lodged Performance is preserved unchanged. kWh is included as ML targets per ADR-0007 — see [[epc-ml-transform]].
|
||||
Re-predicting a Property's SAP score, CO2 emissions, Primary Energy Intensity, space heating kWh, and hot water kWh via **SAP10 Calculation** (the deterministic `Sap10Calculator`, which superseded the old ML-API rebaseliner; an ML residual head over the calculator is future — ADR-0009/0013) so the modelling pipeline scores it against the current SAP10 methodology. Triggered when either (a) the Effective EPC was lodged under a methodology the calculator supersedes (`sap_version < 10.2`, the calculator's target spec), so the recorded scores reflect a superseded methodology, or (b) Site Notes / Landlord Overrides changed the physical state of the Property (walls / heating / windows / etc.) so the lodged scores no longer reflect what's installed. Both triggers may fire together. Produces Effective Performance; Lodged Performance is preserved unchanged. kWh is included as ML targets per ADR-0007 — see [[epc-ml-transform]].
|
||||
_Avoid_: re-scoring, re-prediction, performance recomputation, refresh (for cache-freshness)
|
||||
|
||||
**Baseline Performance**:
|
||||
A Property's current performance aggregate, holding both Lodged Performance and Effective Performance plus annual space heating kWh, hot water kWh, fuel split, and bills derived from the Effective EPC — kWh values come from the EPC's recorded fields for SAP10 baselines or from ML when Rebaselining fires; bills are derived deterministically from kWh × current Fuel Rates. Persisted as one row; surfaced as one block in the UI.
|
||||
A Property's current performance aggregate, holding both Lodged Performance and Effective Performance plus the energy block: delivered kWh **per end use** (heating, hot water, lighting, appliances, cooking, pumps/fans, …) and the **annual bill** composed into per-section costs plus a total, produced by **Bill Derivation** from SAP10 Calculation's per-end-use kWh × current Fuel Rates. Persisted as one row (flat typed columns, per-section kWh + cost + total); surfaced as one block in the UI.
|
||||
_Avoid_: baseline predictions, predicted baseline, rebaselined values
|
||||
|
||||
**Lodged Performance**:
|
||||
|
|
@ -98,7 +98,7 @@ The SAP / EPC Band / carbon emissions / Primary Energy Intensity the modelling p
|
|||
_Avoid_: modelled performance, rebaselined performance (only correct when rebaselining ran), scored values
|
||||
|
||||
**Calculated SAP10 Performance**:
|
||||
The SAP score, EPC Band, CO2 emissions, Primary Energy Intensity, space heating kWh, and hot water kWh produced by **SAP10 Calculation** from a Property's EpcPropertyData. It is **not** a separately-persisted third value-set beside Lodged and Effective: in every baselining scenario the calculator's output *is* the **Effective Performance** (real lodged SAP10 EPC with no overrides ⇒ Calculated = Lodged = Effective; overrides or an estimated / pre-SAP10 EPC ⇒ Calculated = Effective, there being no lodged SAP10 figure to compare against). The calculator is therefore the mechanism that produces Effective Performance, having superseded the old ML-API rebaseliner. While it is being hardened it runs in **shadow** for the first baselining slice — computed on every Property, compared to Lodged, and any divergence (SAP > 0.5, or PEUI / CO2 beyond tolerance) or strict-raise **logged, not persisted** — then is promoted to drive Effective Performance once overrides / estimation land (ADR-0013). The ≥1000-cert parity confirmation against the cert-reported SAP (see [[sap-spec-version]]) gates that promotion. ADR-0009 introduced the term, as amended by ADR-0010 and realized by ADR-0013.
|
||||
The SAP score, EPC Band, CO2 emissions, Primary Energy Intensity, space heating kWh, and hot water kWh produced by **SAP10 Calculation** from a Property's EpcPropertyData. It is **not** a separately-persisted third value-set beside Lodged and Effective: in every baselining scenario the calculator's output *is* the **Effective Performance** (real lodged SAP10 EPC with no overrides ⇒ Calculated = Lodged = Effective; overrides or an estimated / pre-SAP10 EPC ⇒ Calculated = Effective, there being no lodged SAP10 figure to compare against). The calculator is therefore the mechanism that produces Effective Performance, having superseded the old ML-API rebaseliner. The calculator is **load-bearing**: for `sap_version < 10.2` (lodged under a superseded methodology) its output *is* the Effective Performance; for `≥ 10.2` the API's lodged figures are kept and the calculator runs **alongside, logging any divergence** (SAP > 0.5, PEUI/CO2 beyond tolerance) as a validation signal (see [[sap-spec-version]]). It is load-bearing for **Bill Derivation regardless of version** (the EPC lodges no per-end-use kWh), so a calculator strict-raise **aborts the batch** and the un-mapped cert is fixed immediately. ADR-0009 introduced the term, amended by ADR-0010, realized by ADR-0013 (whose shadow stepping-stone is superseded) and ADR-0014.
|
||||
_Avoid_: calculator output, computed performance, worksheet performance, SAP10 output, calculated value-set (it is not a stored third set)
|
||||
|
||||
**SAP10 Calculation**:
|
||||
|
|
@ -117,9 +117,9 @@ _Avoid_: parity cohort, validation set, corpus sample
|
|||
The process that translates an Optimised Package into cert-field changes and produces the "ending state snapshot" EpcPropertyData that Plan Phase persists. Implemented by the `MeasureApplicator` service class in `domain/sap/` (or a sibling package). Each Measure Type's translation rules (e.g. `loft_insulation` → `roof_insulation_thickness_mm = 270mm`, `ashp` → `main_heating_details[0]` replacement) live here. Pure function — does not run SAP10 Calculation itself; the caller chains `MeasureApplicator.apply(epc, package) → Sap10Calculator.calculate(post_epc)`. ADR-0009.
|
||||
_Avoid_: measure overrides (rejected during ADR-0009 grill — phantom mid-layer), package applier, retrofit simulator
|
||||
|
||||
**EPC Energy Derivation**:
|
||||
The process that derives a Property's fuel split and annual bills from its space heating kWh and hot water kWh values plus the heating fuel deduced from SAP fields. kWh values themselves come from the EPC's recorded fields (`renewable_heat_incentive.space_heating_kwh` and `.water_heating_kwh`) for SAP10 baselines, or from ML prediction when Rebaselining fires or when scoring a post-measure state. Bills are computed deterministically from delivered kWh × current Fuel Rates + standing charges + SEG credits. The UCL Correction is no longer applied at runtime — it is folded into ML training labels (see [[epc-ml-transform]] and ADR-0007).
|
||||
_Avoid_: kWh prediction (kWh is now an ML target — see Rebaselining), baseline kWh, energy estimation
|
||||
**Bill Derivation**:
|
||||
The deterministic process that derives a Property's annual energy **bill**, composed into per-end-use sections (heating, hot water, lighting, appliances, cooking, pumps/fans, …) plus a **total**, by pricing **SAP10 Calculation**'s delivered kWh per end use at **current Fuel Rates** — each end use billed at its fuel's rate, rolled up per fuel for **standing charges** (metered fuels only — gas/electricity; oil/LPG/solid have none) minus **SEG** export credit on PV. Implemented by `BillDerivation` in `domain/property_baseline/` (deterministic, ADR-0006). Reads Fuel Rates from a committed static snapshot via `FuelRatesRepository` (no live ETL yet). **Distinct from the calculator's `total_fuel_cost_gbp`**, which is the SAP-rating notional cost at RdSAP Table 32 standardised prices (~half the real electricity price) — not what the household pays. Raises on a fuel it has no rate for (e.g. house coal, heat network). ADR-0014.
|
||||
_Avoid_: EPC Energy Derivation (renamed), EpcEnergyDerivationService (no "service" suffix), kWh prediction, baseline kWh, energy estimation
|
||||
|
||||
**UCL Correction**:
|
||||
The per-band linear correction (Few et al. 2023, _Energy & Buildings_ 288 113024) that aligns EPC-modelled Primary Energy Intensity with metered consumption. Folded into ML training labels at fit time (per ADR-0007) rather than applied at runtime — the trained model emits metered-equivalent PEUI directly, avoiding the discontinuities at EPC band boundaries that arose when the per-band linear correction was applied post-prediction. Calibrated against gas-heated, non-PV homes in England and Wales rated under SAP 2012; the current implementation extrapolates it to all properties (open question §15.14).
|
||||
|
|
@ -174,11 +174,11 @@ _Avoid_: code list, code dictionary, vocab
|
|||
### Reference data
|
||||
|
||||
**Fuel Rates**:
|
||||
The current per-fuel rate (pence/kWh) and standing charge used to compute a Property's bills; time-versioned and regional, refreshed from Ofgem's published caps via an ETL. The Smart Export Guarantee rate sits in the same set as `electricity_export`. Consumed by EPC Energy Derivation.
|
||||
The current per-fuel rate (pence/kWh) and standing charge used to compute a Property's bills; time-versioned and regional. Sourced for now from a **committed static snapshot** (national, Ofgem-cap period for gas/electricity + DESNZ/NEP for off-gas fuels), read via `FuelRatesRepository`; an Ofgem-cap ETL automating the refresh is future, not a prerequisite. The Smart Export Guarantee rate sits in the same set as `electricity_export`. Consumed by Bill Derivation.
|
||||
_Avoid_: fuel prices (commodity prices, different concept), tariff, energy cost
|
||||
|
||||
**Carbon Factors**:
|
||||
The per-fuel CO2 emission factor (kgCO2e/kWh) used to compute a Property's carbon emissions; time-versioned, refreshed from Defra's annual publication. Consumed by EPC Energy Derivation.
|
||||
The per-fuel CO2 emission factor (kgCO2e/kWh) used to compute a Property's carbon emissions; time-versioned, refreshed from Defra's annual publication. Consumed by Bill Derivation.
|
||||
_Avoid_: emission factors (ambiguous), CO2 rates
|
||||
|
||||
### Outputs
|
||||
|
|
@ -277,7 +277,7 @@ _Avoid_: API key, auth token, secret
|
|||
- When a **Property** has both **Site Notes** and a public **EPC**, the newer of the two derives the **Effective EPC**. **Landlord Overrides** apply only when the **EPC** is the source — never when **Site Notes** are.
|
||||
- A Property's **Baseline Performance** holds two halves: **Lodged Performance** (the gov register's SAP / band / carbon / heat) and **Effective Performance** (what the modelling pipeline scored against). The two are equal unless **Rebaselining** fires.
|
||||
- **Rebaselining** produces **Effective Performance** by ML re-prediction across SAP score, CO2 emissions, Primary Energy Intensity, space heating kWh, and hot water kWh, when either (a) the Effective EPC was lodged under a pre-SAP10 schema, or (b) the Effective EPC's physical state diverges from the lodged EPC. **Lodged Performance** is never overwritten.
|
||||
- **EPC Energy Derivation** derives **fuel split** and **bills** from kWh values (sourced from the EPC's `renewable_heat_incentive` fields for baseline SAP10 properties, or from ML when Rebaselining fires), reading current **Fuel Rates** and **Carbon Factors** from their respective repos.
|
||||
- **Bill Derivation** derives **fuel split** and **bills** from kWh values (sourced from the EPC's `renewable_heat_incentive` fields for baseline SAP10 properties, or from ML when Rebaselining fires), reading current **Fuel Rates** and **Carbon Factors** from their respective repos.
|
||||
- The **EPC Prediction Service** uses **Comparable Properties** for both gap-filling and producing **EPC Anomaly Flags**.
|
||||
- A **Scenario** carries one or more ordered **Scenario Phases**. Triggering the model against N Scenarios produces N **Plans** per Property; each Plan carries an ordered list of **Plan Phases** matching the Scenario's shape.
|
||||
- Each **Plan Phase** holds its **Optimised Package**, the ending state snapshot, and any **Rolled-over Options** that flow as candidates into the next Plan Phase. A single-phase Scenario is one Scenario Phase with all measure types allowed; the same machinery handles it.
|
||||
|
|
@ -289,7 +289,7 @@ _Avoid_: API key, auth token, secret
|
|||
|
||||
> **Dev:** "A landlord uploads a corrected boiler for one of their properties. What happens?"
|
||||
>
|
||||
> **Domain expert:** "That's a **Landlord Override** on the heating fields. Save it against the **Property**. The **Effective EPC** has changed, so **Rebaselining** runs to re-predict SAP / carbon / PEUI / space heating kWh / hot water kWh, and **EPC Energy Derivation** re-runs to update the fuel split and bills based on the new kWh values and fuel deduction. With fresh **Baseline Performance** we regenerate **Recommendations**."
|
||||
> **Domain expert:** "That's a **Landlord Override** on the heating fields. Save it against the **Property**. The **Effective EPC** has changed, so **Rebaselining** runs to re-predict SAP / carbon / PEUI / space heating kWh / hot water kWh, and **Bill Derivation** re-runs to update the fuel split and bills based on the new kWh values and fuel deduction. With fresh **Baseline Performance** we regenerate **Recommendations**."
|
||||
|
||||
> **Dev:** "What if the same Property also has Site Notes?"
|
||||
>
|
||||
|
|
|
|||
|
|
@ -86,3 +86,24 @@ property test cohort is about to flow through baselining. So this lands in two s
|
|||
wild); each caught exception is logged with its type and `property_id`.
|
||||
- This decision is short-lived in its shadow form by design; the durable half — "the calculator
|
||||
produces Effective Performance; there is no third value-set" — outlives it.
|
||||
|
||||
## Amendment (2026-06-02): shadow collapsed — the calculator is load-bearing now
|
||||
|
||||
The shadow stepping-stone was right in shape but wrong in duration: the calculator was ready, and
|
||||
wiring [Bill Derivation](0014-bill-derivation-from-real-fuel-rates.md) onto its delivered-kWh
|
||||
breakdown makes it load-bearing for *bills on every property* — so the "shadow until overrides /
|
||||
estimation land" timeline collapses to now. The durable decision stands (calculator produces
|
||||
Effective Performance; no third value-set); only the timing changes:
|
||||
|
||||
- **`sap_version < 10.2`** → effective performance **is** the calculator's output (the
|
||||
`StubRebaseliner` floor moves `10.0 → 10.2`; mechanism is the calculator, not ML).
|
||||
- **`sap_version ≥ 10.2`** → effective = the API's lodged figures; the calculator still runs
|
||||
**alongside, logging divergence** (the surviving half of the shadow runner) as a validation signal.
|
||||
- **Failure posture flips to abort:** the calculator is load-bearing for Bill Derivation regardless
|
||||
of version, so a strict-raise **aborts the batch** (ADR-0012) — the un-mapped cert is fixed
|
||||
immediately rather than skipped. The shadow's catch-and-log of raises is retired; divergence
|
||||
*warnings* on `≥ 10.2` certs remain.
|
||||
|
||||
The `≥1000-cert parity` gate from ADR-0009/0010 still governs whether the calculator's figures are
|
||||
*trusted as definitive* for the SAP-10.2 cohort, but it no longer gates *wiring* — pre-10.2 certs
|
||||
have no current-spec lodged figure to fall back to, so the calculator is the only source there.
|
||||
|
|
|
|||
89
docs/adr/0014-bill-derivation-from-real-fuel-rates.md
Normal file
89
docs/adr/0014-bill-derivation-from-real-fuel-rates.md
Normal file
|
|
@ -0,0 +1,89 @@
|
|||
---
|
||||
Status: accepted
|
||||
---
|
||||
|
||||
# Bill Derivation: whole-home annual bill from the calculator's delivered kWh × real Fuel Rates (not SAP prices)
|
||||
|
||||
Lifts the bills/fuel-split deferral in [ADR-0004](0004-baseline-performance-lodged-effective-pair.md)
|
||||
and its migration note, and builds on [ADR-0013](0013-calculator-produces-effective-performance-shadow-first.md)
|
||||
(the calculator is load-bearing). Decided in a `/grill-with-docs` session (2026-06-02).
|
||||
|
||||
## Context
|
||||
|
||||
ADR-0004's amendment deferred fuel split + bills "because bills require a current Fuel Rates
|
||||
source (Ofgem-cap ETL) that does not yet exist." A static snapshot lifts that blocker. The old
|
||||
`backend/ml_models/AnnualBillSavings.py` is the fragile reference (a blended `PRICE_FACTOR`, two
|
||||
disagreeing rate sources, a standing-charge precedence bug, a 10× unit slip) — we rewrite, not port.
|
||||
|
||||
## Decisions
|
||||
|
||||
### 1. The bill is whole-home, composed per end use, from the calculator's delivered kWh
|
||||
|
||||
`SAP10 Calculation` already emits delivered (post-efficiency, billable) kWh for every regulated end
|
||||
use — main/secondary heating, hot water, pumps/fans, lighting, cooling — and computes appliances +
|
||||
cooking electricity internally (Appendix L L13-L20). **`BillDerivation`** consumes that per-end-use
|
||||
breakdown and produces per-section costs + a total. The EPC lodges no per-end-use kWh, so the
|
||||
calculator is the only source — which is why it is **load-bearing for bills regardless of
|
||||
`sap_version`** (a raise aborts the batch, ADR-0013).
|
||||
|
||||
### 2. Bills use real Fuel Rates, not the calculator's `total_fuel_cost_gbp`
|
||||
|
||||
The calculator's fuel cost is the SAP-rating notional cost at **RdSAP Table 32 standardised
|
||||
prices** — deliberately frozen for rating comparability, and ~half the real electricity price
|
||||
(Table 32 elec ~13 p/kWh vs Ofgem Apr–Jun 2026 cap ~24.7 p/kWh). Billing on it would roughly halve
|
||||
an electric/heat-pump home's bill. So `BillDerivation` **re-prices** the delivered kWh at current
|
||||
**Fuel Rates**, and the calculator's `total_fuel_cost_gbp` is used only for the SAP rating.
|
||||
|
||||
### 3. Fuel Rates = committed static snapshot, read via `FuelRatesRepository`
|
||||
|
||||
A national snapshot (Ofgem-cap period for gas/electricity, DESNZ/NEP for off-gas fuels), keyed by a
|
||||
canonical **`Fuel`** enum (`MAINS_GAS, ELECTRICITY, ELECTRICITY_OFF_PEAK, OIL, LPG, SMOKELESS,
|
||||
WOOD_LOGS, WOOD_PELLETS, HEAT_NETWORK`), each entry carrying `unit_rate_p_per_kwh` +
|
||||
`standing_charge_p_per_day`, plus a top-level `seg_export_p_per_kwh`. The calculator's per-end-use
|
||||
SAP fuel codes map to this enum via the existing `is_gas_code` / `is_electric_fuel_code` /
|
||||
`is_liquid_fuel_code` helpers — so the snapshot and the calculator meet at one vocabulary, not raw
|
||||
SAP codes. Read through a `FuelRatesRepository` port (ADR-0011: a Repo reads stored reference data
|
||||
by key); an Ofgem-cap ETL automating the refresh is future, behind the same port — not a
|
||||
prerequisite. National now; the 14 cap regions are a later refinement behind the same port.
|
||||
|
||||
### 4. Bill arithmetic
|
||||
|
||||
Total = Σ (per-end-use delivered kWh × that end use's fuel unit rate) + per-meter **standing
|
||||
charges** (metered fuels only — gas/electricity; oil/LPG/solid have none) − **SEG** export credit on
|
||||
PV. Off-peak electricity splits day/night via the calculator's existing Table 12a high/low-rate
|
||||
fractions.
|
||||
|
||||
### 5. Strict-raise on an unpriced fuel
|
||||
|
||||
`BillDerivation` **raises** on a fuel it has no rate for — same discipline as the calculator. Two
|
||||
named gaps surface immediately rather than billing at a wrong default:
|
||||
- **House coal** — no standard domestic price (its domestic sale is illegal in England).
|
||||
- **Communal / heat network** — scheme-specific, no national tariff. The one common case (flats);
|
||||
a heat-network rate model is a named follow-up.
|
||||
|
||||
### 6. Persistence: flat per-section columns on `property_baseline_performance`
|
||||
|
||||
The energy block lands as **flat typed columns** on the existing row (ADR-0004's flat-column rule
|
||||
holds — the SAP end-uses are a *fixed enumerable set*, so there is no column explosion and no
|
||||
variable-shape JSON): per-section `*_kwh` + `*_cost_gbp` (heating, hot water, lighting, appliances,
|
||||
cooking, pumps/fans), `standing_charges_gbp`, `seg_credit_gbp`, and `total_annual_bill_gbp`. The
|
||||
production migration is FE-owned (Drizzle); `docs/migrations/` updated.
|
||||
|
||||
## Consequences
|
||||
|
||||
- `BillDerivation` is named for the operation, **no "Service" suffix** (user preference).
|
||||
- A `Fuel` enum + a SAP-code→`Fuel` mapping become first-class; `FuelRates` + `FuelRatesRepository`
|
||||
+ a committed snapshot file are new.
|
||||
- Carbon emissions are unaffected (they stay on Lodged/Effective Performance from the calculator's
|
||||
CO2 factors); this ADR is about £ bills only.
|
||||
- The snapshot goes stale on the Ofgem-cap cadence (quarterly); the file records its period, and the
|
||||
ETL that automates refresh is the deferred follow-up.
|
||||
|
||||
## Considered alternatives
|
||||
|
||||
- **Bill from `RenewableHeatIncentive` heating+HW kWh only** (CONTEXT's original scope) — rejected:
|
||||
the user wants the whole-home bill, and heating+HW omits lighting/appliances/cooking, which only
|
||||
the calculator supplies.
|
||||
- **Bill at SAP Table 32 prices** — rejected: standardised rating prices, ~half real electricity.
|
||||
- **JSON `bill_breakdown` block** — rejected: end-uses are fixed-cardinality, so flat columns are
|
||||
clean and stay queryable (ADR-0004).
|
||||
Loading…
Add table
Reference in a new issue