junte/Model
1
0
Fork 0
mirror of https://github.com/Hestia-Homes/Model.git synced 2026-07-12 13:29:04 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Merge pull request #1373 from Hestia-Homes/fix/1361-class-b-predicted-null-lodged

Browse source 
Drop the phantom Lodged Performance on predicted Properties (#1361 Class B)
This commit is contained in:
Daniel Roth 2026-07-01 08:35:58 +01:00 • committed by GitHub
commit 0c701ba3de
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
12 changed files with 484 additions and 31 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
CONTEXT.md
View file

@ -120,11 +120,11 @@ Deterministically translating an **old / reduced-data EPC schema** into the curr
_Avoid_: gap-fill (means the neighbour-ML path), reduced-data expansion (overloaded with the calculator's Table-5 step), remapping (the schema-translation part only)
**Baseline Performance**:
A Property's current performance aggregate, holding both Lodged Performance and Effective Performance plus the energy block: delivered kWh **per end use** (heating, hot water, lighting, appliances, cooking, pumps/fans, cooling) and the **annual bill** composed into per-section costs plus a total, produced by **Bill Derivation** from SAP10 Calculation's per-end-use kWh × current Fuel Rates. Persisted as one row (flat typed columns, per-section kWh + cost + total); surfaced as one block in the UI.
A Property's current performance aggregate, holding both Lodged Performance and Effective Performance plus the energy block: delivered kWh **per end use** (heating, hot water, lighting, appliances, cooking, pumps/fans, cooling) and the **annual bill** composed into per-section costs plus a total, produced by **Bill Derivation** from SAP10 Calculation's per-end-use kWh × current Fuel Rates. Persisted as one row (flat typed columns, per-section kWh + cost + total); surfaced as one block in the UI. The **Lodged half is optional**: a predicted Property has no lodged record (see Lodged Performance), so its `lodged_*` are `NULL` and only the Effective half + energy block are populated (ADR-0004 amendment, #1361).
_Avoid_: baseline predictions, predicted baseline, rebaselined values
**Lodged Performance**:
The SAP / EPC Band / carbon emissions / Primary Energy Intensity recorded on the public EPC (or the Site Notes' as-surveyed values when Site Notes are the source) — unmodified by modelling. The half of Baseline Performance that says "what the government register says about this Property".
The SAP / EPC Band / carbon emissions / Primary Energy Intensity recorded on the public EPC (or the Site Notes' as-surveyed values when Site Notes are the source) — unmodified by modelling. The half of Baseline Performance that says "what the government register says about this Property". **Requires a record _of this Property_** — a lodged cert or a Site Notes survey — so it is **absent (`None`/NULL) for a predicted Property**: **EPC Prediction** synthesises the picture from neighbours, copying a comparable's recorded figures, so there is no government-register record of _this_ Property to lodge. Only the Effective half is persisted then (ADR-0004 amendment, #1361). Reading a predicted EPC's recorded fields as Lodged Performance manufactures a phantom — a neighbour's SAP presented as this Property's lodged figure.
_Avoid_: original performance, raw EPC values, recorded baseline
**Effective Performance**:

55
docs/adr/0004-baseline-performance-lodged-effective-pair.md
View file

@ -39,3 +39,58 @@ than churning the table twice.
The SQLModel row is defined in `infrastructure/postgres/` so the ephemeral-Postgres tests build it
via `create_all`; the production migration is FE-owned (Drizzle ORM) and tracked in
`docs/migrations/`.
### Amendment (2026-06-30, #1361 Class B): the Lodged half is absent for a predicted Property
The original invariant — **every** `PropertyBaselinePerformance` populates **both** halves, even when
equal — assumed every Property has a record *of its own* performance to lodge: a public EPC cert (the
`epc_with_overlay` path) or a Site Notes survey. It does **not** hold for the **EPC Prediction** path
(ADR-0029/0031). A predicted Property has no record of itself — its `EpcPropertyData` is deterministic
neighbour synthesis that copies a representative comparable's structure wholesale, so the
`energy_rating_current` / band / CO2 / Primary Energy Intensity on the synthesised picture are a
*different dwelling's* lodged figures. Reading **Lodged Performance** off it manufactures a **phantom**:
a borrowed neighbour's SAP presented as this Property's government-register figure. This affected
**every** predicted Property (~12,236 rows estate-wide), not the 8 the `effective-lodged-divergence`
audit surfaced — that audit only fires when the borrowed figure lands ≥15 SAP from the Effective, so it
under-counts the phantom by three orders of magnitude.
**Decision.** The Lodged half is **optional**. When `source_path == "predicted"` there is no Lodged
Performance: `PropertyBaselinePerformance.lodged` is `None` and the four `lodged_*` columns are `NULL`.
The **Effective half is unchanged and still persisted** — a predicted Property is a first-class modelled
output (it flows through Rebaselining, Bill Derivation, and Modelling like any other; its Effective
Performance and bill block are correct and load-bearing for the FE and the plan-vs-effective audit
checks). `rebaseline_reason` stays `physical_state_changed` / `both`, which already records that the
Effective figure was scored from a changed picture.
**Principle (the boundary).** Lodged Performance requires a record **of this Property** — a lodged cert
*or* a Site Notes survey (the as-surveyed values are a real observation of this dwelling). EPC
Prediction borrows a neighbour's, so it has none. The branch is therefore `source_path == "predicted"` —
**not** `physical_state_changed` (true for Site Notes and Landlord Overrides too) and **not** "no public
EPC" (Site Notes has none yet keeps a legitimate Lodged Performance). Whether the Site Notes as-surveyed
values are truly "lodged" is a separate question, deferred.
**Rejected — a sentinel (`lodged = 0`).** Considered, to make "no record" visibly present rather than an
empty cell. Rejected: `0` is a valid-looking SAP score that (a) re-trips `effective-lodged-divergence`
for every predicted Property (`|effective − 0| ≥ 15`), (b) poisons every `AVG(lodged_*)` aggregate that
`NULL` is correctly excluded from, and (c) has no coherent `lodged_epc_band` enum member. The "no lodged
record" signal belongs in `NULL` (honest absence) plus the **structural provenance** already carried by
the distinct predicted-EPC slot — which the FE renders as a predicted badge — not overloaded onto the
score column.
**Consequences.**
- The four `lodged_*` columns become **nullable**. The production table is **FE-owned (Drizzle)**, so the
migration (`ALTER … DROP NOT NULL`) lands in the FE repo and **must precede** any backend write of a
`NULL` lodged, or the predicted-Property INSERT violates the constraint and aborts the batch
(ADR-0012). The SQLModel mirror in `infrastructure/postgres/` is updated to `Optional` so the
ephemeral-Postgres tests build the nullable shape.
- The fix lives in `PropertyBaselineOrchestrator.run()` — the single chokepoint both the First Run
pipeline and `applications/modelling_e2e/handler.py` call — so one change repairs both entry points.
The Rebaseliner port takes `Optional[Performance]`; for a predicted Property `physical_state_changed`
is always true, so `CalculatorRebaseliner` adopts the calculator output and never reads the absent
lodged half (the divergence-log path is the pristine-cert case only, where lodged is non-null).
- A one-time backfill (`scripts/`, dry-run default + `--apply`, idempotent) NULLs the four `lodged_*`
columns on existing predicted-source rows (~12,236). It corrects only the Lodged half; Effective, the
bill block, and `rebaseline_reason` are left intact.
- `effective-lodged-divergence` already short-circuits on `lodged_sap IS NULL`, so it goes green for
predicted Properties once backfilled. Class C of #1361 (a floor that ignores implausibly-low lodged
scores on *real* certs) is an additive guard, independent of this change.

23
docs/migrations/property-baseline-performance-table.md
View file

@ -18,10 +18,10 @@ straight lift-and-shift of the columns below.
|---|---|---|
| `id` | serial PK | |
| `property_id` | int, FK → `property.id`, **unique** | one Baseline Performance per Property |
| `lodged_sap_score` | int | Lodged Performance — gov register, off the Effective EPC |
| `lodged_epc_band` | text | the `Epc` enum, stored as its string value (e.g. `"C"`) |
| `lodged_co2_emissions_t_per_yr` | float | tonnes CO₂/yr (whole dwelling) |
| `lodged_primary_energy_intensity_kwh_per_m2_yr` | int | PEUI (kWh/m²/yr); **not** "heat demand" — see CONTEXT.md |
| `lodged_sap_score` | int, **nullable** | Lodged Performance — gov register, off the Effective EPC. **NULL for a predicted Property** (no lodged cert — #1361, see below) |
| `lodged_epc_band` | text, **nullable** | the `Epc` enum, stored as its string value (e.g. `"C"`) |
| `lodged_co2_emissions_t_per_yr` | float, **nullable** | tonnes CO₂/yr (whole dwelling) |
| `lodged_primary_energy_intensity_kwh_per_m2_yr` | int, **nullable** | PEUI (kWh/m²/yr); **not** "heat demand" — see CONTEXT.md |
| `effective_sap_score` | int | Effective Performance — what modelling scored against |
| `effective_epc_band` | text | |
| `effective_co2_emissions_t_per_yr` | float | tonnes CO₂/yr (whole dwelling) |
@ -30,6 +30,21 @@ straight lift-and-shift of the columns below.
| `space_heating_kwh` | float | EPC `renewable_heat_incentive` recorded demand. **Superseded** by `heating_kwh` (delivered) when the bill block populates; kept until then to avoid an empty-kWh gap, dropped in the population slice. |
| `water_heating_kwh` | float | EPC `renewable_heat_incentive`; **superseded** by `hot_water_kwh`. |
### Lodged half is nullable (#1361 Class B, 2026-06-30)
The four `lodged_*` columns are **nullable**. A **predicted** Property (EPC Prediction, ADR-0029/0031:
no lodged cert, its `EpcPropertyData` synthesised from neighbours) has **no Lodged Performance** — the
backend now writes `lodged_* = NULL` for it and persists only the Effective half (ADR-0004 amendment).
Reading the synthesised EPC's recorded fields as a lodged figure had been manufacturing a phantom (a
neighbour's SAP), which the migration + a one-time backfill remove.
**FE-owned Drizzle migration required:** `ALTER TABLE property_baseline_performance ALTER COLUMN
lodged_sap_score DROP NOT NULL` (and the same for `lodged_epc_band`, `lodged_co2_emissions_t_per_yr`,
`lodged_primary_energy_intensity_kwh_per_m2_yr`). This **must land before** the backend deploys the
orchestrator change or runs `scripts/null_predicted_lodged_performance.py` — a `NULL` write against a
`NOT NULL` column aborts the batch (ADR-0012). The backfill NULLs the four columns on the ~12,236
existing predicted-source rows; Effective, the bill block, and `rebaseline_reason` are left intact.
### Bill block (ADR-0014) — the energy bill, composed per section
Produced by **Bill Derivation**: the calculator's **delivered** kWh per end use priced at current

7
domain/property_baseline/calculator_rebaseliner.py
View file

@ -57,7 +57,7 @@ class CalculatorRebaseliner(Rebaseliner):
self,
property_id: int,
effective_epc: "EpcPropertyData",
lodged: Performance,
lodged: Optional[Performance],
*,
physical_state_changed: bool = False,
) -> RebaselineResult:
@ -87,6 +87,11 @@ class CalculatorRebaseliner(Rebaseliner):
reason=reason,
sap_result=result,
)
# The pristine-lodged path: not pre-SAP10 and not physical-state-changed, so
# there is a real lodged cert to validate against. ``lodged is None`` only
# happens for a predicted Property, which is physical_state_changed and so
# took the branch above — it never reaches here.
assert lodged is not None
self._log_divergence(
property_id=property_id, sap_version=sap_version, result=result, lodged=lodged
)

7
domain/property_baseline/property_baseline_performance.py
View file

@ -15,7 +15,10 @@ class PropertyBaselinePerformance:
Holds both halves — ``lodged`` (what the gov register says) and
``effective`` (what the modelling pipeline scored against) — plus the
``rebaseline_reason`` recording *why* they differ (``"none"`` when equal).
Both halves are always populated, even when equal.
Both halves are populated for a Property with a real record of itself; the
``lodged`` half is ``None`` for a predicted Property, which has no lodged cert
to read a Lodged Performance off — only the Effective half is established then
(ADR-0004 amendment, #1361).
Carries the part of the energy block that needs no derivation: annual
``space_heating_kwh`` / ``water_heating_kwh`` read off the EPC's RHI.
@ -25,7 +28,7 @@ class PropertyBaselinePerformance:
ran (the stub path produced no ``SapResult`` to price).
"""
lodged: Performance
lodged: Optional[Performance]
effective: Performance
rebaseline_reason: RebaselineReason
space_heating_kwh: float

14
domain/property_baseline/rebaseliner.py
View file

@ -62,11 +62,15 @@ class Rebaseliner(ABC):
self,
property_id: int,
effective_epc: EpcPropertyData,
lodged: Performance,
lodged: Optional[Performance],
*,
physical_state_changed: bool = False,
) -> RebaselineResult:
"""Produce Effective Performance. ``physical_state_changed`` is True when
"""Produce Effective Performance. ``lodged`` is ``None`` for a predicted
Property — it has no lodged cert, so there is no Lodged Performance to
compare against (which is also why ``physical_state_changed`` is always
True there: the calculator output IS the Effective Performance).
``physical_state_changed`` is True when
the Effective EPC was assembled from something other than a pristine
lodged cert — Landlord Overrides, Site Notes, or EPC Prediction moved the
physical picture (Rebaselining trigger (b)/(c)) — so the accredited lodged
@ -91,7 +95,7 @@ class StubRebaseliner(Rebaseliner):
self,
property_id: int,
effective_epc: EpcPropertyData,
lodged: Performance,
lodged: Optional[Performance],
*,
physical_state_changed: bool = False,
) -> RebaselineResult:
@ -108,4 +112,8 @@ class StubRebaseliner(Rebaseliner):
"Property needs rebaselining (physical state changed by overrides "
"/ prediction); this stub does not run the calculator"
)
# Only a pristine SAP10 cert reaches here, and that always has a Lodged
# Performance — ``lodged is None`` implies a predicted Property, which is
# ``physical_state_changed`` and so raised above.
assert lodged is not None
return RebaselineResult(effective=lodged, reason="none", sap_result=None)

51
infrastructure/postgres/property_baseline_performance_table.py
View file

@ -48,12 +48,16 @@ class PropertyBaselinePerformanceModel(SQLModel, table=True):
# Postgres won't coerce to the enum. Bind the native enums explicitly
# (``create_type=False``: the types already exist). Values stay plain
# strings on the domain side (``Epc(...)`` / the RebaselineReason Literal).
lodged_sap_score: int
lodged_epc_band: str = Field(
sa_column=Column(SAEnum(*_EPC_BANDS, name="epc", create_type=False), nullable=False)
# The four lodged_* columns are nullable as a unit: a predicted Property has no
# lodged cert, so it has no Lodged Performance (ADR-0004 amendment, #1361). They
# are written all-set or all-NULL; lodged_sap_score is the read discriminator.
lodged_sap_score: Optional[int] = None
lodged_epc_band: Optional[str] = Field(
default=None,
sa_column=Column(SAEnum(*_EPC_BANDS, name="epc", create_type=False), nullable=True),
)
lodged_co2_emissions_t_per_yr: float
lodged_primary_energy_intensity_kwh_per_m2_yr: int
lodged_co2_emissions_t_per_yr: Optional[float] = None
lodged_primary_energy_intensity_kwh_per_m2_yr: Optional[int] = None
effective_sap_score: int
effective_epc_band: str = Field(
@ -102,12 +106,17 @@ class PropertyBaselinePerformanceModel(SQLModel, table=True):
def from_domain(
cls, baseline: PropertyBaselinePerformance, property_id: int
) -> "PropertyBaselinePerformanceModel":
# A predicted Property has no Lodged Performance — the four lodged_* columns
# are all NULL then (ADR-0004 amendment, #1361).
lodged = baseline.lodged
model = cls(
property_id=property_id,
lodged_sap_score=baseline.lodged.sap_score,
lodged_epc_band=baseline.lodged.epc_band.value,
lodged_co2_emissions_t_per_yr=baseline.lodged.co2_emissions,
lodged_primary_energy_intensity_kwh_per_m2_yr=baseline.lodged.primary_energy_intensity,
lodged_sap_score=lodged.sap_score if lodged is not None else None,
lodged_epc_band=lodged.epc_band.value if lodged is not None else None,
lodged_co2_emissions_t_per_yr=lodged.co2_emissions if lodged is not None else None,
lodged_primary_energy_intensity_kwh_per_m2_yr=(
lodged.primary_energy_intensity if lodged is not None else None
),
effective_sap_score=baseline.effective.sap_score,
effective_epc_band=baseline.effective.epc_band.value,
effective_co2_emissions_t_per_yr=baseline.effective.co2_emissions,
@ -139,12 +148,7 @@ class PropertyBaselinePerformanceModel(SQLModel, table=True):
def to_domain(self) -> PropertyBaselinePerformance:
return PropertyBaselinePerformance(
lodged=Performance(
sap_score=self.lodged_sap_score,
epc_band=Epc(self.lodged_epc_band),
co2_emissions=self.lodged_co2_emissions_t_per_yr,
primary_energy_intensity=self.lodged_primary_energy_intensity_kwh_per_m2_yr,
),
lodged=self._read_lodged(),
effective=Performance(
sap_score=self.effective_sap_score,
epc_band=Epc(self.effective_epc_band),
@ -157,6 +161,23 @@ class PropertyBaselinePerformanceModel(SQLModel, table=True):
bill=self._read_bill(),
)
def _read_lodged(self) -> Optional[Performance]:
"""The Lodged half, or None for a predicted Property whose four lodged_*
columns are NULL (ADR-0004 amendment, #1361). They are written as a unit
(`from_domain`), so lodged_sap_score is the discriminator and the other
three are non-null alongside it."""
if self.lodged_sap_score is None:
return None
assert self.lodged_epc_band is not None
assert self.lodged_co2_emissions_t_per_yr is not None
assert self.lodged_primary_energy_intensity_kwh_per_m2_yr is not None
return Performance(
sap_score=self.lodged_sap_score,
epc_band=Epc(self.lodged_epc_band),
co2_emissions=self.lodged_co2_emissions_t_per_yr,
primary_energy_intensity=self.lodged_primary_energy_intensity_kwh_per_m2_yr,
)
def _read_bill(self) -> Optional[Bill]:
"""Reconstruct the Bill from the ``bill_*`` columns. The total is the
not-None discriminator: a persisted bill always sets it, so its absence

12
orchestration/property_baseline_orchestrator.py
View file

@ -8,7 +8,7 @@ from domain.billing.bill import EnergyBreakdown
from domain.sap10_calculator.calculator import SapResult
from domain.billing.bill_derivation import BillDerivation
from domain.property_baseline.property_baseline_performance import PropertyBaselinePerformance
from domain.property_baseline.performance import lodged_performance
from domain.property_baseline.performance import Performance, lodged_performance
from domain.property_baseline.rebaseliner import Rebaseliner
from repositories.fuel_rates.fuel_rates_repository import FuelRatesRepository
from repositories.unit_of_work import UnitOfWork
@ -50,7 +50,15 @@ class PropertyBaselineOrchestrator:
properties = uow.property.get_many(property_ids)
for property_id, prop in zip(property_ids, properties, strict=True):
effective_epc = prop.effective_epc
lodged = lodged_performance(effective_epc)
# A predicted Property has no lodged cert — its Effective EPC is a
# neighbour-synthesised picture, so its recorded performance fields
# are a different dwelling's. There is no Lodged Performance to read
# (ADR-0004 amendment, #1361); only the Effective half is established.
lodged: Optional[Performance] = (
None
if prop.source_path == "predicted"
else lodged_performance(effective_epc)
)
rebaselined = self._rebaseliner.rebaseline(
property_id,
effective_epc,

111
scripts/null_predicted_lodged_performance.py Normal file
View file

@ -0,0 +1,111 @@
"""One-time backfill: NULL the phantom Lodged Performance on predicted Properties.
#1361 Class B: ``PropertyBaselineOrchestrator`` used to read Lodged Performance
off a predicted Property's neighbour-synthesised EPC, persisting a phantom set of
``lodged_*`` figures on ``property_baseline_performance`` — a *different
dwelling's* SAP / band / carbon / Primary Energy Intensity presented as this
Property's government-register record. The orchestrator no longer does this
(``lodged`` is ``None`` when ``source_path == "predicted"``, ADR-0004 amendment),
but the ~12k rows written before the fix still carry the phantom.
This NULLs the four ``lodged_*`` columns on every predicted-source baseline row,
leaving the **Effective** half, the **bill** block, and ``rebaseline_reason``
untouched (a predicted Property's Effective Performance is correct — it is a
first-class modelled output).
A predicted-source Property is identified exactly as the orchestrator's
``source_path == "predicted"``: it has a **predicted** EPC and **no lodged** EPC.
Site Notes keep their as-surveyed Lodged Performance and so are excluded — though
no Site-Notes-sourced EPC exists in ``epc_property`` today, the predicate (a
predicted EPC, no lodged EPC) holds regardless.
DRY-RUN BY DEFAULT: prints the count it would change and writes nothing. Pass
``--apply`` to execute inside a transaction. **Idempotent** — only rows whose
``lodged_sap_score`` is still non-NULL are touched, so a second run is a no-op.
Requires the FE-owned Drizzle ``ALTER ... DROP NOT NULL`` on the four ``lodged_*``
columns to have landed first; without it the UPDATE to NULL violates the
constraint.
"""
from __future__ import annotations
import argparse
from sqlalchemy import Connection, text
from scripts.e2e_common import build_engine, load_env
# A predicted-source baseline row: a predicted EPC exists for the property and no
# lodged one does (``source_path == "predicted"``). ``lodged_sap_score IS NOT
# NULL`` makes it idempotent — a row already nulled is skipped on a re-run.
_PREDICTED_PHANTOM_PREDICATE = """
pbp.lodged_sap_score IS NOT NULL
AND EXISTS (
SELECT 1 FROM epc_property e
WHERE e.property_id = pbp.property_id AND e.source = 'predicted'
)
AND NOT EXISTS (
SELECT 1 FROM epc_property e
WHERE e.property_id = pbp.property_id AND e.source = 'lodged'
)
"""
_COUNT = text(
f"""
SELECT count(*) FROM property_baseline_performance pbp
WHERE {_PREDICTED_PHANTOM_PREDICATE}
"""
)
_NULL_LODGED = text(
f"""
UPDATE property_baseline_performance AS pbp
SET lodged_sap_score = NULL,
lodged_epc_band = NULL,
lodged_co2_emissions_t_per_yr = NULL,
lodged_primary_energy_intensity_kwh_per_m2_yr = NULL
WHERE {_PREDICTED_PHANTOM_PREDICATE}
"""
)
def backfill(conn: Connection, *, apply: bool) -> int:
"""NULL the four ``lodged_*`` columns on predicted-source baseline rows.
Returns the number of phantom rows found (those that ``--apply`` would / did
null). Reads the count first so the dry-run reports it without writing.
"""
found = conn.execute(_COUNT).scalar() or 0
if apply:
conn.execute(_NULL_LODGED)
return found
def main() -> None:
load_env()
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"--apply",
action="store_true",
help="execute the update (default: dry-run, writes nothing)",
)
args = parser.parse_args()
engine = build_engine()
with engine.begin() as conn:
conn.execute(text("SET statement_timeout = 120000"))
found = backfill(conn, apply=args.apply)
verb = "NULLed" if args.apply else "would NULL"
print(
f"{verb} the Lodged Performance (lodged_* → NULL) on {found} "
"predicted-source baseline row(s); Effective / bill / rebaseline_reason "
"left intact."
)
if not args.apply:
print("\nDRY-RUN — nothing written. Re-run with --apply to execute.")
if __name__ == "__main__":
main()

79
tests/orchestration/test_property_baseline_orchestrator.py
View file

@ -1,6 +1,6 @@
from __future__ import annotations
from typing import TYPE_CHECKING
from typing import Optional, TYPE_CHECKING
import pytest
@ -161,10 +161,11 @@ class _ScoringRebaseliner(Rebaseliner):
self,
property_id: int,
effective_epc: EpcPropertyData,
lodged: Performance,
lodged: Optional[Performance],
*,
physical_state_changed: bool = False,
) -> RebaselineResult:
assert lodged is not None # this stub is only used on the real-cert path
return RebaselineResult(
effective=lodged, reason="none", sap_result=self._result
)
@ -244,6 +245,80 @@ def test_run_raises_when_no_rhi_and_no_scored_result() -> None:
assert uow.commits == 0
class _PredictedRebaseliner(Rebaseliner):
"""Scores a predicted picture into a fixed Effective Performance, ignoring the
(absent) lodged half — a predicted Property always has ``physical_state_changed``,
so the calculator output IS the Effective Performance (ADR-0039)."""
def __init__(self, effective: Performance, result: SapResult) -> None:
self._effective = effective
self._result = result
def rebaseline(
self,
property_id: int,
effective_epc: EpcPropertyData,
lodged: Optional[Performance],
*,
physical_state_changed: bool = False,
) -> RebaselineResult:
return RebaselineResult(
effective=self._effective,
reason="physical_state_changed",
sap_result=self._result,
)
def _predicted_property() -> Property:
"""A predicted Property (ADR-0031): no lodged EPC, only a neighbour-synthesised
predicted EPC whose recorded performance fields are a *different dwelling's* —
the phantom Lodged Performance this fix must not propagate. Mirrors the #1361
exemplar (uprn 34003067 / property 742072), predicted-only."""
predicted = object.__new__(EpcPropertyData)
predicted.energy_rating_current = 14 # phantom: copied from a neighbour template
predicted.current_energy_efficiency_band = Epc.G
predicted.co2_emissions_current = 5.0
predicted.energy_consumption_current = 400
predicted.sap_version = 10.2
predicted.renewable_heat_incentive = None
return Property(
identity=PropertyIdentity(
portfolio_id=1, postcode="A0 0AA", address="1 Some Street", uprn=34003067
),
epc=None,
predicted_epc=predicted,
)
def test_run_persists_no_lodged_performance_for_a_predicted_property() -> None:
# Arrange — a predicted Property (no lodged cert); the rebaseliner scores its
# predicted picture into a known Effective Performance.
property_baseline_repo = FakePropertyBaselineRepo()
uow = FakeUnitOfWork(
property=FakePropertyRepo({742072: _predicted_property()}),
property_baseline=property_baseline_repo,
)
effective = Performance(
sap_score=57, epc_band=Epc.D, co2_emissions=1.5, primary_energy_intensity=160
)
orchestrator = PropertyBaselineOrchestrator(
unit_of_work=lambda: uow,
rebaseliner=_PredictedRebaseliner(
effective, _sap_result_with_heating(space_kwh=4200.0, water_kwh=1600.0)
),
fuel_rates=FuelRatesStaticFileRepository(),
)
# Act
orchestrator.run([742072])
# Assert — no lodged comparator was manufactured from the neighbour-synthesised
# EPC's recorded fields; the Effective half is still persisted.
(baseline, _) = property_baseline_repo.saved[0]
assert baseline.lodged is None
assert baseline.effective == effective
def test_run_derives_and_persists_a_bill_when_the_rebaseliner_scores() -> None:
# Arrange — a rebaseliner that hands back a SapResult with lighting energy,
# so the orchestrator prices it into a Bill at the committed snapshot.

32
tests/repositories/property_baseline/test_property_baseline_postgres_repository.py
View file

@ -83,6 +83,38 @@ def test_resaving_baseline_for_a_property_replaces_rather_than_duplicating(
assert loaded == rerun
def _predicted_baseline() -> PropertyBaselinePerformance:
"""A predicted Property's baseline: no Lodged Performance (no cert to read one
off), only the Effective half established (ADR-0004 amendment, #1361)."""
return PropertyBaselinePerformance(
lodged=None,
effective=Performance(
sap_score=57, epc_band=Epc.D, co2_emissions=1.5, primary_energy_intensity=160
),
rebaseline_reason="physical_state_changed",
space_heating_kwh=4200.0,
water_heating_kwh=1600.0,
)
def test_predicted_baseline_round_trips_with_no_lodged_half(db_engine: Engine) -> None:
# Arrange — a predicted Property has no Lodged Performance; the four lodged_*
# columns persist as NULL and reconstruct as a None lodged half.
baseline = _predicted_baseline()
with Session(db_engine) as session:
PropertyBaselinePostgresRepository(session).save(baseline, property_id=13)
session.commit()
# Act
with Session(db_engine) as session:
loaded = PropertyBaselinePostgresRepository(session).get_for_property(13)
# Assert — the lodged half round-trips as None; the Effective half is intact.
assert loaded == baseline
assert loaded is not None
assert loaded.lodged is None
def test_get_for_property_returns_none_when_absent(db_engine: Engine) -> None:
# Arrange / Act
with Session(db_engine) as session:

120
tests/scripts/test_null_predicted_lodged_performance.py Normal file
View file

@ -0,0 +1,120 @@
"""The one-time backfill nulls a predicted Property's phantom Lodged Performance,
leaves a lodged Property's intact, and is idempotent (#1361 Class B)."""
from __future__ import annotations
import json
from pathlib import Path
from typing import Any
from sqlalchemy import Engine
from sqlmodel import Session
from datatypes.epc.domain.epc import Epc
from datatypes.epc.domain.epc_property_data import EpcPropertyData
from datatypes.epc.domain.mapper import EpcPropertyDataMapper
from domain.property_baseline.performance import Performance
from domain.property_baseline.property_baseline_performance import (
PropertyBaselinePerformance,
)
from repositories.epc.epc_postgres_repository import EpcPostgresRepository
from repositories.property_baseline.property_baseline_postgres_repository import (
PropertyBaselinePostgresRepository,
)
from scripts.null_predicted_lodged_performance import backfill
_JSON_SAMPLES = Path(__file__).resolve().parents[2] / "backend/epc_api/json_samples"
_PHANTOM_LODGED = Performance(
sap_score=14, epc_band=Epc.G, co2_emissions=5.0, primary_energy_intensity=400
)
_EFFECTIVE = Performance(
sap_score=57, epc_band=Epc.D, co2_emissions=1.5, primary_energy_intensity=160
)
def _epc() -> EpcPropertyData:
raw: dict[str, Any] = json.loads(
(_JSON_SAMPLES / "RdSAP-Schema-21.0.0" / "epc.json").read_text()
)
return EpcPropertyDataMapper.from_api_response(raw)
def _baseline(lodged: Performance) -> PropertyBaselinePerformance:
return PropertyBaselinePerformance(
lodged=lodged,
effective=_EFFECTIVE,
rebaseline_reason="physical_state_changed",
space_heating_kwh=4200.0,
water_heating_kwh=1600.0,
)
def _seed(db_engine: Engine) -> None:
"""A predicted Property (id 1, predicted EPC only, phantom lodged) and a
lodged Property (id 2, lodged EPC, real lodged) — both with a baseline row
carrying a populated lodged half (the pre-fix state)."""
epc = _epc()
with Session(db_engine) as session:
epc_repo = EpcPostgresRepository(session)
epc_repo.save(epc, property_id=1, source="predicted")
epc_repo.save(epc, property_id=2, source="lodged")
baseline_repo = PropertyBaselinePostgresRepository(session)
baseline_repo.save(_baseline(_PHANTOM_LODGED), property_id=1)
baseline_repo.save(_baseline(_PHANTOM_LODGED), property_id=2)
session.commit()
def test_apply_nulls_only_the_predicted_propertys_lodged_half(
db_engine: Engine,
) -> None:
# Arrange
_seed(db_engine)
# Act
with db_engine.begin() as conn:
found = backfill(conn, apply=True)
# Assert — one phantom found and nulled; the predicted Property loses its
# lodged half but keeps its Effective half, while the lodged Property is
# untouched.
assert found == 1
with Session(db_engine) as session:
repo = PropertyBaselinePostgresRepository(session)
predicted = repo.get_for_property(1)
lodged = repo.get_for_property(2)
assert predicted is not None
assert predicted.lodged is None
assert predicted.effective == _EFFECTIVE
assert lodged is not None
assert lodged.lodged == _PHANTOM_LODGED
def test_dry_run_reports_the_phantom_but_writes_nothing(db_engine: Engine) -> None:
# Arrange
_seed(db_engine)
# Act
with db_engine.begin() as conn:
found = backfill(conn, apply=False)
# Assert — the phantom is counted, but the row is left untouched.
assert found == 1
with Session(db_engine) as session:
predicted = PropertyBaselinePostgresRepository(session).get_for_property(1)
assert predicted is not None
assert predicted.lodged == _PHANTOM_LODGED
def test_re_running_apply_is_a_no_op(db_engine: Engine) -> None:
# Arrange — the first apply nulls the phantom.
_seed(db_engine)
with db_engine.begin() as conn:
backfill(conn, apply=True)
# Act — a second apply finds nothing left to null.
with db_engine.begin() as conn:
found = backfill(conn, apply=True)
# Assert
assert found == 0