Model/docs/adr/0014-bill-derivation-from-real-fuel-rates.md

Status
accepted

Bill Derivation: whole-home annual bill from the calculator's delivered kWh × real Fuel Rates (not SAP prices)

Lifts the bills/fuel-split deferral in ADR-0004 and its migration note, and builds on ADR-0013 (the calculator is load-bearing). Decided in a /grill-with-docs session (2026-06-02).

Context

ADR-0004's amendment deferred fuel split + bills "because bills require a current Fuel Rates source (Ofgem-cap ETL) that does not yet exist." A static snapshot lifts that blocker. The old backend/ml_models/AnnualBillSavings.py is the fragile reference (a blended PRICE_FACTOR, two disagreeing rate sources, a standing-charge precedence bug, a 10× unit slip) — we rewrite, not port.

Decisions

1. The bill is whole-home, composed per end use, from the calculator's delivered kWh

SAP10 Calculation already emits delivered (post-efficiency, billable) kWh for every regulated end use — main/secondary heating, hot water, pumps/fans, lighting, cooling — and computes appliances + cooking electricity internally (Appendix L L13-L20). BillDerivation consumes that per-end-use breakdown and produces per-section costs + a total. The EPC lodges no per-end-use kWh, so the calculator is the only source — which is why it is load-bearing for bills regardless of sap_version (a raise aborts the batch, ADR-0013).

2. Bills use real Fuel Rates, not the calculator's total_fuel_cost_gbp

The calculator's fuel cost is the SAP-rating notional cost at RdSAP Table 32 standardised prices — deliberately frozen for rating comparability, and ~half the real electricity price (Table 32 elec ~13 p/kWh vs Ofgem Apr–Jun 2026 cap ~24.7 p/kWh). Billing on it would roughly halve an electric/heat-pump home's bill. So BillDerivation re-prices the delivered kWh at current Fuel Rates, and the calculator's total_fuel_cost_gbp is used only for the SAP rating.

3. Fuel Rates = committed static snapshot, read via FuelRatesRepository

A national snapshot (Ofgem-cap period for gas/electricity, DESNZ/NEP for off-gas fuels), keyed by a canonical Fuel enum (MAINS_GAS, ELECTRICITY, ELECTRICITY_OFF_PEAK, OIL, LPG, SMOKELESS, WOOD_LOGS, WOOD_PELLETS, HEAT_NETWORK), each entry carrying unit_rate_p_per_kwh + standing_charge_p_per_day, plus a top-level seg_export_p_per_kwh. The calculator's per-end-use SAP fuel codes map to this enum via the existing is_gas_code / is_electric_fuel_code / is_liquid_fuel_code helpers — so the snapshot and the calculator meet at one vocabulary, not raw SAP codes. Read through a FuelRatesRepository port (ADR-0011: a Repo reads stored reference data by key); an Ofgem-cap ETL automating the refresh is future, behind the same port — not a prerequisite. National now; the 14 cap regions are a later refinement behind the same port.

4. Bill arithmetic

Total = Σ (per-end-use delivered kWh × that end use's fuel unit rate) + per-meter standing charges (metered fuels only — gas/electricity; oil/LPG/solid have none) − SEG export credit on PV. Off-peak electricity splits day/night via the calculator's existing Table 12a high/low-rate fractions.

5. Strict-raise on an unpriced fuel

BillDerivation raises on a fuel it has no rate for — same discipline as the calculator. Two named gaps surface immediately rather than billing at a wrong default:

• House coal — no standard domestic price (its domestic sale is illegal in England).
• Communal / heat network — scheme-specific, no national tariff. The one common case (flats); a heat-network rate model is a named follow-up.

6. Persistence: flat per-section columns on property_baseline_performance

The energy block lands as flat typed columns on the existing row (ADR-0004's flat-column rule holds — the SAP end-uses are a fixed enumerable set, so there is no column explosion and no variable-shape JSON): per-section *_kwh + *_cost_gbp (heating, hot water, lighting, appliances, cooking, pumps/fans), standing_charges_gbp, seg_credit_gbp, and total_annual_bill_gbp. The production migration is FE-owned (Drizzle); docs/migrations/ updated.

Consequences

• BillDerivation is named for the operation, no "Service" suffix (user preference).
• A Fuel enum + a SAP-code→Fuel mapping become first-class; FuelRates + FuelRatesRepository
  • a committed snapshot file are new.
• Carbon emissions are unaffected (they stay on Lodged/Effective Performance from the calculator's CO2 factors); this ADR is about £ bills only.
• The snapshot goes stale on the Ofgem-cap cadence (quarterly); the file records its period, and the ETL that automates refresh is the deferred follow-up.

Deferred / TODO

• Appliances + cooking kWh are computed inside cert_to_inputs (Appendix L L13-L20) but not yet threaded onto SapResult. Until they are, the SapResult → EnergyBreakdown adapter stubs them at 0 kWh, so the bill total currently understates by the unregulated electricity load. Khalim is adding the fields to SapResult directly; the adapter wires the APPLIANCES/COOKING sections in as soon as they land.
• Off-peak (Economy 7) day/night split — resolved, see the 2026-06-24 amendment below.
• Heat-network rate model — heat-network certs raise UnpricedFuel for now (the one common gap).
• Regional rates + Ofgem-cap ETL — national snapshot now; both are later refinements behind the same FuelRatesRepository port.

Considered alternatives

• Bill from RenewableHeatIncentive heating+HW kWh only (CONTEXT's original scope) — rejected: the user wants the whole-home bill, and heating+HW omits lighting/appliances/cooking, which only the calculator supplies.
• Bill at SAP Table 32 prices — rejected: standardised rating prices, ~half real electricity.
• JSON bill_breakdown block — rejected: end-uses are fixed-cardinality, so flat columns are clean and stay queryable (ADR-0004).

Amendment (2026-06-02): fuel is a calculator output; §3's mapping helpers corrected

Wiring the SapResult → EnergyBreakdown adapter forced the question §3 left implicit: where does the fuel each end use burns come from? Resolved in a /grill-with-docs session.

• Decision: per-end-use fuel is calculator output. The calculator resolves the fuel for each billable end use (it already uses it to derive the delivered kWh and the rating cost), so it emits the per-end-use fuel codes on SapResult (main-1 / main-2 / secondary / hot water — the electric end uses are electricity by construction), alongside pv_exported_kwh for the SEG credit. These are the calculator's own fuel codes (which, per ADR-0015, may be raw API codes or already-Table-32 depending on the mapper), so sap_fuel.sap_code_to_fuel normalizes them through the calculator's own table_32.to_table_32_code (T32-first, then API-translate — the same normalization the calculator's pricing/classification uses) before the Table-32 → Fuel dispatch. BillDerivation's adapter is then a pure SapResult → EnergyBreakdown map and can never price the calculator's kWh at a fuel the calculator never used. Rejected: an adapter that re-reads raw EpcPropertyData fuel fields and re-normalizes them — that duplicates cert_to_inputs (_main_fuel_code, _water_heating_fuel_code, HW→main default, CHP blend, the MissingMainFuelType strict-raise) and reopens divergence between the bill and the rating.

• §3 correction. §3 says the per-end-use fuel codes map to Fuel "via the existing is_gas_code / is_electric_fuel_code / is_liquid_fuel_code helpers." That is not what shipped: mapping is domain/property_baseline/sap_fuel.py::sap_code_to_fuel, a bounded Table-32 fuel-code → Fuel dispatch that strict-raises UnmappedSapCode on an unmapped code. The "meet at one vocabulary, not raw SAP codes" intent stands; the named helpers do not.

• Interim, pending ADR-0015. Fuel resolution sits in the calculator because EpcPropertyData is not yet a strict normalized type. Once ADR-0015 lands (mappers normalize at the boundary), attribution can move upstream and the SapResult fuel-code fields may be retired.

• COOLING section added. §1 listed cooling as an end use but §6's flat columns omitted it. BillSection gains COOLING (kWh from SapResult.space_cooling_fuel_kwh_per_yr, electricity by construction), so §6's layout gains a cooling_kwh + cooling_cost_gbp column pair (FE-owned Drizzle migration).

Amendment (2026-06-03): Bill Derivation is cross-stage; the Modelling stage prices the post-package end-state

Bill Derivation is no longer Baseline-only — the Modelling stage now re-runs it on the Optimised Package to produce post-retrofit bills and savings. Decided in a /grill-with-docs session.

• Bill Derivation is a cross-stage domain concern → relocate to domain/billing/. Bill / EnergyBreakdown / BillDerivation / sap_fuel were under domain/property_baseline/ only because Baseline was built first. Two stages now consume them, and a modelling → property_baseline import would couple two stages ADR-0011 keeps independent under a name that wrongly implies ownership. They move to a neutral domain/billing/ (Fuel/FuelRates already live in the shared domain/fuel_rates/). Mechanical move + import rewrite; covered by the existing Baseline tests.

• Modelling bills the simulated end-state, never adjusts the baseline bill. The post-retrofit bill is BillDerivation.derive(EnergyBreakdown.from_sap_result(post_package_sap_result)), where the SapResult comes from scoring the fully-overlaid EpcPropertyData (all selected Simulation Overlays + injected dependencies). This is what makes fuel-switch measures correct for free: a measure that switches heating fuel (e.g. oil → electric ASHP) changes the heating fuel code on that SapResult, so sap_code_to_fuel prices it at the new fuel automatically — no per-measure fuel bookkeeping. Savings are baseline − post, both priced at the same FuelRates snapshot (read once per run), so the delta is never polluted by a rate change.

• No second calculator pass. The post-package SapResult is the one the optimiser's whole-package re-score (role 2) already computed; it rides on the Score (Score.sap_result, populated by PackageScorer, ignored by the optimiser — so the optimiser stays Score-only and its stub-scorer tests are unaffected). Likewise the baseline SapResult is the one the orchestrator already scores for the role-3 cascade and the target gain. Billing reuses both — zero extra calculate.

• FuelRatesRepository is constructor-injected into ModellingOrchestrator, mirroring the Baseline orchestrator — get_current() once per run(), one BillDerivation reused across the batch. Not on the UnitOfWork (read-once reference data, ADR-0011). The extra per-pipeline read (Baseline + Modelling each resolve rates) is accepted; a shared/injected snapshot is a future optimisation.

• Plan-level first, per-measure savings next (telescoping cascade). This slice fills the plan columns (post_energy_bill, post_energy_consumption, energy_bill_savings, energy_consumption_savings). Per-measure recommendation.kwh_savings / energy_cost_savings come from a bill cascade over the role-3 best-practice order (fabric → heating → renewables) — re-bill each cumulative prefix and diff, telescoping exactly to the plan totals (mirroring the SAP role-3 attribution; reuses the per-prefix sap_results, no extra calls). Per-measure savings can be negative (ventilation increases energy) and still telescope. The legacy recommendation.energy_savings column is vestigial (legacy set it to 0; the canonical delivered-energy field is kwh_savings) — left NULL.

• Limitation carried over. The "Appliances + cooking kWh stubbed at 0" deferral above still applies — Modelling's post-package bill understates by the same unregulated-electricity load until those fields land on SapResult. Baseline and Modelling share the gap, so baseline-vs-post savings remain consistent.

Amendment (2026-06-24): off-peak is a whole-meter tariff, day/night split per end use from the calculator's Table 12a fractions

Resolves the §4 / Deferred "off-peak day/night split" — forced by a modelling_e2e failure (no rate for fuel ELECTRICITY_OFF_PEAK) on an off-peak-metered dwelling, where the snapshot's E7 day/night entry was deliberately skipped (no single unit_rate_p_per_kwh) and BillDerivation raised UnpricedFuel. Decided in a /grill-with-docs session.

• Off-peak is a property of the METER, not of an end use. An Economy-7-style dwelling has one dual-rate meter, and every electric end use — heating, hot water, and lighting / pumps-fans / appliances / cooking — bills on it (the calculator already prices the whole meter this way via Table 12a Grid 1 + Grid 2). The pre-amendment EnergyBreakdown modelled off-peak as a per-end-use fuel that only heating/HW could carry, hard-wiring the other electric lines to standard ELECTRICITY — a latent under-pricing of the daytime appliance/lighting load (standard 24.67p vs off-peak day 29.73p), the mirror of the heating over-pricing that folding-to-standard would cause. So the breakdown now models off-peak as a whole-meter tariff: on an off-peak meter, from_sap_result routes all electric lines to Fuel.ELECTRICITY_OFF_PEAK. This needs an off-peak-meter signal on SapResult (a fraction of 1.0 on the off-peak day rate is not the same as standard electricity).

• The day/night split is the calculator's existing Table 12a high-rate fraction, surfaced as a calculator output. Each end use's High-Rate Fraction (fraction of its kWh at the day/high rate; remainder at night/low) is already computed inside cert_to_inputs (_main_space_heating_high_rate_fraction, water_heating_high_rate_fraction, the Table-13 immersion blend, the HP-DHW exception, other_use_high_rate_fraction for Grid 2) and folded into a blended Table-32 cost the bill discards. Per the "fuel is a calculator output" amendment above, the fraction is now also calculator output — threaded CalculatorInputs → SapResult per end use — so the bill reuses it rather than re-deriving a second day/night model that would drift from the rating. Unregulated loads SAP does not rate (appliances, cooking) have no Table 12a fraction; they inherit the ALL_OTHER_USES fraction (same Grid-2 row + daytime-weighted profile as lighting) — a documented extension, since SAP itself never splits them.

• Representation: fraction-on-the-line; rates stay in FuelRates. EnergyLine gains an optional high_rate_fraction (None = single-rate fuel). FuelRates gains a day/night accessor for ELECTRICITY_OFF_PEAK (the repository now reads the snapshot's existing day_p_per_kwh / night_p_per_kwh). BillDerivation prices an off-peak line at kwh × (frac × day_rate + (1−frac) × night_rate). Rejected: (a) pre-splitting kWh into day/night lines upstream — leaks rate-tier structure into the calculator/adapter and doubles the section roll-up; (b) the blended day/night rate handed over by the calculator — that is the calculator pricing at real rates, the exact boundary §2 draws.

• Rejected shortcuts (both re-open closed decisions). Fold ELECTRICITY_OFF_PEAK → standard ELECTRICITY: knowingly mis-prices in both directions (≈+45% on night-shifted storage heat, under-prices daytime loads). A single hand-blended off-peak unit_rate in the snapshot: reintroduces the legacy AnnualBillSavings.py blended PRICE_FACTOR anti-pattern this ADR's Context rewrote away from — the day/night weighting is per-end-use and per-dwelling, not a property of the rate.

• Safety: no-op for unaffected certs. Standard-tariff dwellings are unchanged (every fraction is 1.0 and the carrier stays ELECTRICITY). Off-peak dwellings currently hard-crash (UnpricedFuel aborts the batch, §1), so there is no passing bill value to regress — they move from crash → priced. SAP scores are untouched: the bill is output-only and never feeds the rating.