Merge pull request #1242 from Hestia-Homes/feature/landlord-overrides

Feature/landlord overrides
This commit is contained in:
Jun-te Kim 2026-06-17 10:31:52 +01:00 committed by GitHub
commit 8d4cbcccea
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
21 changed files with 1067 additions and 20 deletions

View file

@ -1,4 +1,7 @@
FROM mcr.microsoft.com/playwright/python:v1.58.0-jammy
# jammy (Ubuntu 22.04) ships Python 3.10, which lacks enum.StrEnum — used by
# domain/modelling/measure_type.py, pulled in transitively via the copied
# domain/ package. noble (Ubuntu 24.04) ships Python 3.12.
FROM mcr.microsoft.com/playwright/python:v1.58.0-noble
# Install AWS Lambda RIE
ADD https://github.com/aws/aws-lambda-runtime-interface-emulator/releases/latest/download/aws-lambda-rie /usr/local/bin/aws-lambda-rie

View file

@ -1,4 +1,8 @@
FROM mcr.microsoft.com/playwright/python:v1.58.0-jammy
# jammy (Ubuntu 22.04) ships Python 3.10, which lacks enum.StrEnum — used by
# domain/modelling/measure_type.py, which the handler pulls in transitively
# (handler -> pashub_service -> documents_parser.parser -> ... -> domain).
# noble (Ubuntu 24.04) ships Python 3.12, matching the project's 3.11+ standard.
FROM mcr.microsoft.com/playwright/python:v1.58.0-noble
# Install AWS Lambda RIE
ADD https://github.com/aws/aws-lambda-runtime-interface-emulator/releases/latest/download/aws-lambda-rie /usr/local/bin/aws-lambda-rie

View file

@ -0,0 +1,82 @@
# Landlord-Override EPC overlay (`epc_with_overlay`)
When a Property has a lodged EPC **and** resolved Landlord Overrides, the
Effective EPC is the lodged EPC with those overrides **applied as a simulation**,
so the SAP10 calculator scores a picture that reflects what the landlord knows
beyond the cert (typically before a survey). This records *how* that overlay
works and refines ADR-0031 decision 3, which deferred it ("Landlord Overrides
overlay is a later slice"). Terms in CONTEXT.md (Effective EPC, Landlord
Overrides, Rebaselining, Validation Cohort).
## Status
Accepted (design). Refines ADR-0031 dec-3. Implementation: tracer slice
(`wall_type`) first, coverage expanded per component.
## Decisions
### 1. A Landlord Override is a Simulation Overlay, folded onto the lodged EPC
The overlay reuses the existing `EpcSimulation` / `overlay_applicator` machinery
that every modelling Measure already uses to mutate an `EpcPropertyData`
(`domain/modelling/simulation.py`). "The landlord says the wall is now insulated"
and "a Measure insulates the wall" become **one code path**: a resolved override
maps to an `EpcSimulation`, and `effective_epc`'s `epc_with_overlay` branch folds
it onto the lodged `epc`. CONTEXT.md already frames overrides as "a simulation on
the `EpcPropertyData`" — this makes that literal. Rejected: bespoke field-writing
onto a copied `EpcPropertyData`, which would duplicate the fold and invent a
second way to mutate an EPC.
### 2. The Property's override slot holds domain overlays; the repository maps to them
`Property` gains a landlord-overrides slot typed as **domain** `EpcSimulation`s,
not the repository's `ResolvedPropertyOverrides` (which would make `domain/`
import `repositories/` and invert the dependency). `PropertyPostgresRepository`
hydrates the slot by reading the faithful value-space snapshot
(`PropertyOverridesReader`) and mapping each override value → an `EpcSimulation`
the same place it already hydrates `epc` / `predicted_epc`.
### 3. The fold is partial and per-component
`property_overrides` is partial — only the components a portfolio actually
supplied are present (a Property may carry `property_type` + `wall_type` but no
`roof_type`). The overlay applies **only** the overlays derived from rows that
exist; every other field stays lodged-as-is. This falls out of the faithful
reader, which returns only the rows present.
### 4. `WallType` / `RoofType` decompose into SAP int codes — the calculator reads codes, not descriptions
The SAP calculator derives the wall U-value from the `wall_construction` and
`wall_insulation_type` **int codes**, never from `walls[].description`
(`cert_to_inputs.py`). So a `wall_type` override only moves the score once
translated into those codes. The translation is tractable because a `WallType`
value is *material × insulation state*: material → `wall_construction` (codes 1-5)
and state → `wall_insulation_type` (1 external, 3 internal, 4 as-built, 7 filled),
the code sets already documented in `solid_wall_recommendation.py`. The override's
`building_part` maps directly to the overlay's `BuildingPartIdentifier`
(0 → MAIN, 1 → EXTENSION_1, …). `property_type` / `built_form_type` carry no SAP
weight at this layer (geometry is already explicit) — they overlay as metadata
only.
### 5. An overlaid Property is excluded from the Validation Cohort — on divergence, not source path
Producing a *more accurate* score than the cert is the **point**: the calculated
Effective Performance deliberately diverges from the Lodged Performance (which is
preserved). That divergence means an overlaid Property is no longer a clean
`calc(effective_epc)`-vs-lodged control, so it is excluded from the Validation
Cohort — exactly as a predicted-source Property is. The exclusion signal is
**divergence** ("≥1 override was applied", the overlay slot is non-empty), **not**
`source_path`: `epc_with_overlay` also covers plain-EPC Properties with no
overrides, which *are* valid validation targets. This reuses the same divergence
signal CONTEXT.md already names as a Rebaselining trigger. No Validation-Cohort
code path exists today, so this is a recorded rule, not code in this slice.
## Consequences
- `EpcPropertyData` is untouched — provenance stays structural on the Property
(ADR-0031 dec-3), and every downstream consumer is oblivious to the overlay.
- The override→`EpcSimulation` mapping is the real deliverable and grows per
component; `wall_type` is the tracer (highest SAP impact, existing code-set
prior art). `roof_type` and full `WallType` coverage follow.
- Precedence is unchanged (CONTEXT.md): the overlay lives entirely inside the
`epc_with_overlay` branch and never applies when Site Notes are the source.

View file

@ -0,0 +1,56 @@
"""Map a Landlord-Override enum *value* to the gov-EPC API *code* space.
The `property_overrides` fact layer stores resolved overrides as enum-value
strings ("House", "Detached"); the EPC-API cohort certs carry numeric codes
("0", "2") `EpcPropertyData.property_type = str(schema.property_type)`. EPC
Prediction filters `comparable.epc.property_type == target.property_type`, so a
target attribute sourced from overrides must be translated into the code space
or no comparable ever matches (ADR-0031, the wiring-handover "every cohort
comes back empty" gotcha).
Codes are the gov RdSAP/SAP table values in `datatypes/epc/domain/epc_codes.csv`.
This module owns "unresolvable": a value that maps to no code returns None.
"""
from __future__ import annotations
from typing import Optional
# property_type codes (epc_codes.csv, `property_type` rows — stable across the
# RdSAP/SAP schemas that carry each member). "Park home" exists only from
# SAP-17.0 / RdSAP-17.0 onward; the code itself is stable where present.
_PROPERTY_TYPE_CODES: dict[str, str] = {
"House": "0",
"Bungalow": "1",
"Flat": "2",
"Maisonette": "3",
"Park home": "4",
}
# built_form codes (epc_codes.csv, `built_form` rows). "Not Recorded" lodges as
# the non-numeric "NR", but cohort comparables carry `str(int)` for built_form,
# so an "NR" target could never match — and built_form is the SOFT filter, so a
# non-match only widens the cohort. We therefore treat "Not Recorded" (and the
# classifier "Unknown") as "no usable built-form signal" → None.
_BUILT_FORM_CODES: dict[str, str] = {
"Detached": "1",
"Semi-Detached": "2",
"End-Terrace": "3",
"Mid-Terrace": "4",
"Enclosed End-Terrace": "5",
"Enclosed Mid-Terrace": "6",
}
def property_type_to_code(override_value: str) -> Optional[str]:
"""The gov-EPC `property_type` code for a Landlord-Override value, or None
when it has no code ("Unknown", or any unmapped value) which gates the
Property out of prediction, as `property_type` is the hard cohort filter."""
return _PROPERTY_TYPE_CODES.get(override_value)
def built_form_to_code(override_value: str) -> Optional[str]:
"""The gov-EPC `built_form` code for a Landlord-Override value, or None when
it has no usable code ("Unknown", "Not Recorded", or any unmapped value).
built_form is the soft filter, so None simply leaves it unconditioned."""
return _BUILT_FORM_CODES.get(override_value)

View file

@ -0,0 +1,60 @@
"""Map a Landlord-Override `WallType` value to a wall Simulation Overlay (ADR-0032).
A `WallType` value is one full EPC wall description *material* (cavity, solid
brick, ) combined with *insulation state* (as built / with internal insulation
/ filled cavity / ). The calculator scores the wall from the RdSAP
`wall_construction` (material) and `wall_insulation_type` (state) **int codes**,
never the description string, so the overlay decomposes the value into those two
codes and emits an `EpcSimulation` targeting the override's building part. The
result folds onto the lodged EPC via `apply_simulations`, exactly as a wall
Measure's overlay does. Unresolvable material/state → None (no overlay).
"""
from __future__ import annotations
from typing import Optional
from datatypes.epc.domain.epc_property_data import BuildingPartIdentifier
from domain.modelling.simulation import BuildingPartOverlay, EpcSimulation
# RdSAP `wall_construction` codes by material prefix (domain/sap10_ml/rdsap_uvalues.py).
_MATERIAL_CONSTRUCTION: dict[str, int] = {
"Cavity wall": 4,
"Solid brick": 3,
}
# RdSAP `wall_insulation_type` codes by insulation-state suffix
# (domain/sap10_ml/rdsap_uvalues.py): external 1, filled-cavity 2, internal 3,
# as-built/uninsulated 4, cavity+external 6, cavity+internal 7.
_STATE_INSULATION: dict[str, int] = {
"as built, no insulation (assumed)": 4,
"with internal insulation": 3,
"with external insulation": 1,
"filled cavity": 2,
"filled cavity and internal insulation": 7,
"filled cavity and external insulation": 6,
}
def wall_overlay_for(
wall_type_value: str, building_part: int
) -> Optional[EpcSimulation]:
material, _, state = wall_type_value.partition(", ")
construction = _MATERIAL_CONSTRUCTION.get(material)
insulation = _STATE_INSULATION.get(state)
if construction is None or insulation is None:
return None
identifier = (
BuildingPartIdentifier.MAIN
if building_part == 0
else BuildingPartIdentifier.extension(building_part)
)
return EpcSimulation(
building_parts={
identifier: BuildingPartOverlay(
wall_construction=construction,
wall_insulation_type=insulation,
)
}
)

View file

@ -25,6 +25,10 @@ class BuildingPartOverlay:
A `None` field means "leave the baseline value unchanged".
"""
# The wall material (RdSAP `wall_construction` code). Left `None` by Measures
# — insulating a wall doesn't change its material — but set by a Landlord
# Override that corrects the construction itself (ADR-0032).
wall_construction: Optional[int] = None
wall_insulation_type: Optional[int] = None
# Added solid-wall insulation depth (mm) — drives the calculator's Table 6
# bucket / §5.8 documentary U-value for EWI (`wall_insulation_type=1`) and

View file

@ -1,10 +1,12 @@
from __future__ import annotations
from dataclasses import dataclass
from typing import Literal, Optional
from dataclasses import dataclass, field
from typing import Literal, Optional, Sequence
from datatypes.epc.domain.epc_property_data import EpcPropertyData
from domain.geospatial.planning_restrictions import PlanningRestrictions
from domain.modelling.scoring.overlay_applicator import apply_simulations
from domain.modelling.simulation import EpcSimulation
from domain.property.site_notes import SiteNotes
SourcePath = Literal["site_notes", "epc_with_overlay", "predicted"]
@ -43,6 +45,11 @@ class Property:
# structural). Used as the Effective EPC only as a last resort — when there is
# neither a lodged EPC nor Site Notes; a real source always wins.
predicted_epc: Optional[EpcPropertyData] = None
# Resolved Landlord Overrides as Simulation Overlays, folded onto the lodged
# EPC to form the Effective EPC (ADR-0032). Empty when the Property has no
# overrides — the EPC is then returned unchanged. Only applied on the
# `epc_with_overlay` path; never when Site Notes are the source.
landlord_overrides: Sequence[EpcSimulation] = field(default_factory=tuple)
# The current open-market value (a Property Valuation) — externally sourced
# and mostly absent; feeds the Plan's Valuation Uplift £ forms (ADR-0018).
current_market_value: Optional[float] = None
@ -78,10 +85,11 @@ class Property:
def effective_epc(self) -> EpcPropertyData:
"""The EpcPropertyData the modelling pipeline scores against.
Path 1: the Site Notes' surveyed data. Path 2: the public EPC (Landlord
Overrides overlay is a later slice returned as-is for now). Path 3: a
neighbour-synthesised EPC (EPC Prediction gap-fill, ADR-0031), used only
when neither real source is present.
Path 1: the Site Notes' surveyed data. Path 2: the public EPC with any
Landlord Overrides folded on as Simulation Overlays (ADR-0032) returned
as-is when there are none. Path 3: a neighbour-synthesised EPC (EPC
Prediction gap-fill, ADR-0031), used only when neither real source is
present.
"""
if self.source_path == "site_notes":
assert self.site_notes is not None
@ -90,4 +98,6 @@ class Property:
assert self.predicted_epc is not None
return self.predicted_epc
assert self.epc is not None
if self.landlord_overrides:
return apply_simulations(self.epc, self.landlord_overrides)
return self.epc

View file

@ -0,0 +1,27 @@
"""Map a Property's resolved Landlord Overrides to Simulation Overlays (ADR-0032).
The boundary between the faithful `property_overrides` read model
(`ResolvedPropertyOverrides`, value-space) and the domain overlay surface
(`EpcSimulation`). Lives in `repositories/` because it consumes a repository
type `domain/` never imports `repositories/`. Per-component and partial: an
override produces an overlay only where a component mapping exists and resolves
(the tracer covers `wall_type`); everything else is left to the lodged EPC.
"""
from __future__ import annotations
from domain.epc.wall_type_overlay import wall_overlay_for
from domain.modelling.simulation import EpcSimulation
from repositories.property.property_overrides_reader import ResolvedPropertyOverrides
_WALL_TYPE = "wall_type"
def overlays_from(overrides: ResolvedPropertyOverrides) -> list[EpcSimulation]:
overlays: list[EpcSimulation] = []
for row in overrides.rows:
if row.override_component == _WALL_TYPE:
overlay = wall_overlay_for(row.override_value, row.building_part)
if overlay is not None:
overlays.append(overlay)
return overlays

View file

@ -0,0 +1,55 @@
"""The real ``PredictionTargetAttributesReader`` — landlord-overrides-backed.
Composes the faithful ``PropertyOverridesReader`` with the valuecode mapping:
reads the Property's main-building (building_part 0) ``property_type`` /
``built_form_type`` overrides and translates them into the gov-EPC code space the
cohort filter compares against (ADR-0031). An unresolvable ``property_type``
becomes None, which gates the Property out of prediction downstream
(``build_prediction_target``). Wall/roof overrides are left to the later
``epc_with_overlay`` slice this reader conditions cohort selection only.
"""
from __future__ import annotations
from typing import Optional
from domain.epc.override_code_mapping import (
built_form_to_code,
property_type_to_code,
)
from domain.epc_prediction.prediction_target import PredictionTargetAttributes
from repositories.property.prediction_target_attributes_reader import (
PredictionTargetAttributesReader,
)
from repositories.property.property_overrides_reader import PropertyOverridesReader
_MAIN_BUILDING = 0
_PROPERTY_TYPE_COMPONENT = "property_type"
_BUILT_FORM_COMPONENT = "built_form_type"
class OverrideBackedPredictionAttributesReader(PredictionTargetAttributesReader):
def __init__(self, overrides_reader: PropertyOverridesReader) -> None:
self._overrides_reader = overrides_reader
def attributes_for(self, property_id: int) -> PredictionTargetAttributes:
overrides = self._overrides_reader.overrides_for(property_id)
property_type_value = overrides.value(_PROPERTY_TYPE_COMPONENT, _MAIN_BUILDING)
built_form_value = overrides.value(_BUILT_FORM_COMPONENT, _MAIN_BUILDING)
property_type: Optional[str] = (
property_type_to_code(property_type_value)
if property_type_value is not None
else None
)
built_form: Optional[str] = (
built_form_to_code(built_form_value)
if built_form_value is not None
else None
)
return PredictionTargetAttributes(
property_type=property_type,
built_form=built_form,
)

View file

@ -0,0 +1,45 @@
"""Postgres adapter for the ``property_overrides`` read side.
Read-only and uow-independent: ``property_overrides`` is committed reference
data the ``bulk_upload_finaliser`` Lambda writes at Finalise, long before First
Run executes there is no transactional coupling to the ingestion run, so this
opens its own short read session per call via the injected session factory
(mirroring the composition root's ``lambda: Session(engine)``).
"""
from __future__ import annotations
from collections.abc import Callable
from sqlmodel import Session, col, select
from infrastructure.postgres.property_override_table import PropertyOverrideRow
from repositories.property.property_overrides_reader import (
PropertyOverridesReader,
ResolvedPropertyOverride,
ResolvedPropertyOverrides,
)
class PropertyOverridesPostgresReader(PropertyOverridesReader):
def __init__(self, session_factory: Callable[[], Session]) -> None:
self._session_factory = session_factory
def overrides_for(self, property_id: int) -> ResolvedPropertyOverrides:
with self._session_factory() as session:
rows = session.exec(
select(PropertyOverrideRow).where(
col(PropertyOverrideRow.property_id) == property_id
)
).all()
return ResolvedPropertyOverrides(
rows=tuple(
ResolvedPropertyOverride(
override_component=row.override_component,
building_part=row.building_part,
override_value=row.override_value,
)
for row in rows
)
)

View file

@ -0,0 +1,52 @@
"""Read port for the per-Property ``property_overrides`` fact layer (ADR-0006).
The write side (``PropertyOverrideRepository.upsert_all``) materialises the fact
layer at Finalise; this is the read side. It is deliberately *faithful* it
returns the resolved enum-value snapshots exactly as stored ("House",
"Detached", "Solid brick, …"), every ``(override_component, building_part)`` for
the Property, making no judgement about what is resolvable. Consumers translate:
EPC Prediction maps property_type/built_form into the gov-EPC code space and
gates on it (see ``domain/epc/override_code_mapping.py``); the later
``epc_with_overlay`` slice will read wall/roof here too.
"""
from __future__ import annotations
from abc import ABC, abstractmethod
from dataclasses import dataclass
from typing import Optional
@dataclass(frozen=True)
class ResolvedPropertyOverride:
"""One ``property_overrides`` row, in enum-value space (as stored)."""
override_component: str
building_part: int
override_value: str
@dataclass(frozen=True)
class ResolvedPropertyOverrides:
"""Every resolved override for one Property — a faithful value-space snapshot."""
rows: tuple[ResolvedPropertyOverride, ...]
def value(self, override_component: str, building_part: int) -> Optional[str]:
"""The resolved value for one ``(component, building_part)``, or None when
the Property has no such override row."""
for row in self.rows:
if (
row.override_component == override_component
and row.building_part == building_part
):
return row.override_value
return None
class PropertyOverridesReader(ABC):
@abstractmethod
def overrides_for(self, property_id: int) -> ResolvedPropertyOverrides:
"""Every resolved Landlord Override for the Property, as stored. Empty when
the Property has no overrides."""
...

View file

@ -9,10 +9,13 @@ from sqlalchemy.dialects.postgresql import insert as pg_insert
from sqlmodel import Session, col, select
from domain.geospatial.planning_restrictions import PlanningRestrictions
from domain.modelling.simulation import EpcSimulation
from domain.property.properties import Properties
from domain.property.property import Property, PropertyIdentity
from infrastructure.postgres.property_table import PropertyRow
from repositories.epc.epc_repository import EpcRepository
from repositories.property.landlord_override_overlays import overlays_from
from repositories.property.property_overrides_reader import PropertyOverridesReader
from repositories.property.property_repository import (
PropertyIdentityInsert,
PropertyRepository,
@ -37,10 +40,12 @@ class PropertyPostgresRepository(PropertyRepository):
session: Session,
epc_repo: Optional[EpcRepository] = None,
spatial_repo: Optional[SpatialRepository] = None,
overrides_reader: Optional[PropertyOverridesReader] = None,
) -> None:
self._session = session
self._epc_repo = epc_repo
self._spatial_repo = spatial_repo
self._overrides_reader = overrides_reader
# ``__table__`` is injected at runtime on table=True classes but the stubs
# don't expose it; pin to ``Table`` so the dialect insert is typed.
self._table: Table = cast(Table, getattr(PropertyRow, "__table__"))
@ -53,6 +58,13 @@ class PropertyPostgresRepository(PropertyRepository):
)
return self._epc_repo
def _landlord_overrides(self, property_id: int) -> list[EpcSimulation]:
"""The Property's Landlord Overrides as Simulation Overlays — empty when
no reader is wired (the overlay stays off) or the Property has none."""
if self._overrides_reader is None:
return []
return overlays_from(self._overrides_reader.overrides_for(property_id))
def get(self, property_id: int) -> Property:
row = self._session.get(PropertyRow, property_id)
if row is None:
@ -74,6 +86,7 @@ class PropertyPostgresRepository(PropertyRepository):
identity=identity,
epc=self._epc().get_for_property(property_id),
predicted_epc=self._epc().get_predicted_for_property(property_id),
landlord_overrides=self._landlord_overrides(property_id),
planning_restrictions=_restrictions_of(row.uprn, restrictions),
)
@ -105,6 +118,7 @@ class PropertyPostgresRepository(PropertyRepository):
),
epc=epcs.get(property_id),
predicted_epc=predicted_epcs.get(property_id),
landlord_overrides=self._landlord_overrides(property_id),
planning_restrictions=_restrictions_of(row.uprn, restrictions),
)
)

View file

@ -62,6 +62,13 @@ _REPO_ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(_REPO_ROOT)) # worktree root first — avoid the import trap
from datatypes.epc.domain.epc_property_data import EpcPropertyData # noqa: E402
from domain.property.property import Property, PropertyIdentity # noqa: E402
from repositories.property.landlord_override_overlays import ( # noqa: E402
overlays_from,
)
from repositories.property.property_overrides_postgres_reader import ( # noqa: E402
PropertyOverridesPostgresReader,
)
from domain.geospatial.planning_restrictions import PlanningRestrictions # noqa: E402
from domain.geospatial.spatial_reference import SpatialReference # noqa: E402
from domain.modelling.considered_measures import ( # noqa: E402
@ -140,6 +147,27 @@ def _uprns_for(engine: Engine, property_ids: list[int]) -> dict[int, Optional[in
return {int(pid): (int(uprn) if uprn is not None else None) for pid, uprn in rows}
def _dump_overrides(engine: Engine, property_ids: list[int]) -> None:
"""Print each target Property's ``property_overrides`` rows (read-only), so the
Landlord Overrides folded into the Effective EPC are visible before modelling."""
with engine.connect() as conn:
rows = conn.execute(
text(
"SELECT property_id, building_part, override_component, override_value "
"FROM property_overrides WHERE property_id = ANY(:ids) "
"ORDER BY property_id, building_part, override_component"
),
{"ids": property_ids},
).fetchall()
if not rows:
print("landlord overrides: none for the target propertie(s)\n")
return
print("landlord overrides (folded into the Effective EPC):")
for property_id, building_part, component, value in rows:
print(f" property {property_id} · part {building_part} · {component} = {value}")
print()
def _scenario_for(session: Session, scenario_id: int) -> Scenario:
"""Read the Scenario the run targets (read-only). An Increasing-EPC Scenario
must carry a ``goal_value`` (band) the old null-band rows were a fixed bug
@ -358,6 +386,10 @@ def main() -> None:
_parse_measures(args.measures), _parse_measures(args.exclude_measures)
)
uprns = _uprns_for(engine, args.property_ids)
# Landlord Overrides are read from property_overrides and folded onto the lodged
# EPC to form the Effective EPC the calculator scores (ADR-0032).
overrides_reader = PropertyOverridesPostgresReader(lambda: Session(engine))
_dump_overrides(engine, args.property_ids)
# One read-only session for the live `material` catalogue, reused across the
# batch so both store and no-store runs price against the same DB rows.
catalogue_session = Session(engine)
@ -403,6 +435,21 @@ def main() -> None:
epc: Optional[EpcPropertyData] = epc_client.get_by_uprn(uprn)
if epc is None:
raise ValueError(f"no EPC found for UPRN {uprn}")
# Fold any Landlord Overrides onto the lodged EPC; with none, the
# Effective EPC is the lodged EPC unchanged (ADR-0032).
overlaid_property = Property(
identity=PropertyIdentity(
portfolio_id=args.portfolio_id or 0,
postcode="",
address="",
uprn=uprn,
),
epc=epc,
landlord_overrides=overlays_from(
overrides_reader.overrides_for(property_id)
),
)
effective_epc: EpcPropertyData = overlaid_property.effective_epc
spatial: Optional[SpatialReference] = _spatial_for(geospatial, uprn)
restrictions: PlanningRestrictions = (
spatial.restrictions if spatial is not None else PlanningRestrictions()
@ -411,7 +458,7 @@ def main() -> None:
None if args.no_solar else _solar_insights_for(solar_client, spatial)
)
plan: Plan = run_modelling(
epc,
effective_epc,
goal_band=args.goal,
planning_restrictions=restrictions,
solar_insights=solar_insights,

View file

View file

@ -0,0 +1,93 @@
"""The Landlord-Override value → gov-EPC code mapping (ADR-0031 wiring).
`property_type` is the HARD cohort filter, so its mapping is exhaustive over
`PropertyType` and the only one that can silently empty a cohort; `built_form`
is the SOFT filter. Both collapse an unresolvable value to None gating lives
downstream, the mapping just reports "no usable code".
"""
from __future__ import annotations
from typing import Optional
import pytest
from domain.epc.built_form_type import BuiltFormType
from domain.epc.override_code_mapping import (
built_form_to_code,
property_type_to_code,
)
from domain.epc.property_type import PropertyType
def test_house_maps_to_gov_code_zero() -> None:
# Act
code = property_type_to_code("House")
# Assert
assert code == "0"
@pytest.mark.parametrize(
("override_value", "expected_code"),
[
(PropertyType.HOUSE.value, "0"),
(PropertyType.BUNGALOW.value, "1"),
(PropertyType.FLAT.value, "2"),
(PropertyType.MAISONETTE.value, "3"),
(PropertyType.PARK_HOME.value, "4"),
],
)
def test_each_resolvable_property_type_maps_to_its_gov_code(
override_value: str, expected_code: str
) -> None:
# Act
code = property_type_to_code(override_value)
# Assert
assert code == expected_code
@pytest.mark.parametrize(
"override_value",
[PropertyType.UNKNOWN.value, "Castle", ""],
)
def test_unresolvable_property_type_has_no_code(override_value: str) -> None:
# Act
code = property_type_to_code(override_value)
# Assert
assert code is None
@pytest.mark.parametrize(
("override_value", "expected_code"),
[
(BuiltFormType.DETACHED.value, "1"),
(BuiltFormType.SEMI_DETACHED.value, "2"),
(BuiltFormType.END_TERRACE.value, "3"),
(BuiltFormType.MID_TERRACE.value, "4"),
(BuiltFormType.ENCLOSED_END_TERRACE.value, "5"),
(BuiltFormType.ENCLOSED_MID_TERRACE.value, "6"),
],
)
def test_each_resolvable_built_form_maps_to_its_gov_code(
override_value: str, expected_code: str
) -> None:
# Act
code = built_form_to_code(override_value)
# Assert
assert code == expected_code
@pytest.mark.parametrize(
"override_value",
[BuiltFormType.UNKNOWN.value, BuiltFormType.NOT_RECORDED.value, "Castle", ""],
)
def test_built_form_without_usable_code_returns_none(override_value: str) -> None:
# Act
code: Optional[str] = built_form_to_code(override_value)
# Assert
assert code is None

View file

@ -0,0 +1,72 @@
"""The Landlord-Override `WallType` → wall Simulation Overlay mapping (ADR-0032).
A `WallType` value decomposes into the RdSAP `wall_construction` (material) and
`wall_insulation_type` (state) int codes the calculator reads; the overlay
targets the override's building part. Unresolvable values produce no overlay.
"""
from __future__ import annotations
import pytest
from datatypes.epc.domain.epc_property_data import BuildingPartIdentifier
from domain.epc.wall_type_overlay import wall_overlay_for
def test_solid_brick_with_internal_insulation_overlays_main_wall() -> None:
# Act
simulation = wall_overlay_for("Solid brick, with internal insulation", 0)
# Assert — solid brick (wall_construction 3) + internal insulation (type 3)
# on the main building part.
assert simulation is not None
overlay = simulation.building_parts[BuildingPartIdentifier.MAIN]
assert overlay.wall_construction == 3
assert overlay.wall_insulation_type == 3
@pytest.mark.parametrize(
("wall_type_value", "construction", "insulation"),
[
("Cavity wall, as built, no insulation (assumed)", 4, 4),
("Cavity wall, with internal insulation", 4, 3),
("Cavity wall, with external insulation", 4, 1),
("Cavity wall, filled cavity", 4, 2),
("Cavity wall, filled cavity and internal insulation", 4, 7),
("Cavity wall, filled cavity and external insulation", 4, 6),
("Solid brick, as built, no insulation (assumed)", 3, 4),
("Solid brick, with external insulation", 3, 1),
],
)
def test_material_and_state_decompose_to_their_gov_codes(
wall_type_value: str, construction: int, insulation: int
) -> None:
# Act
simulation = wall_overlay_for(wall_type_value, 0)
# Assert
assert simulation is not None
overlay = simulation.building_parts[BuildingPartIdentifier.MAIN]
assert overlay.wall_construction == construction
assert overlay.wall_insulation_type == insulation
def test_overlay_targets_the_extension_building_part() -> None:
# Act — building_part 1 is the first extension.
simulation = wall_overlay_for("Solid brick, with internal insulation", 1)
# Assert
assert simulation is not None
assert BuildingPartIdentifier.EXTENSION_1 in simulation.building_parts
@pytest.mark.parametrize(
"wall_type_value",
["Unknown", "Granite or whin, as built, no insulation (assumed)", ""],
)
def test_unresolvable_wall_type_produces_no_overlay(wall_type_value: str) -> None:
# Act
simulation = wall_overlay_for(wall_type_value, 0)
# Assert
assert simulation is None

View file

@ -0,0 +1,82 @@
"""Effective EPC on the `epc_with_overlay` path folds Landlord Overrides (ADR-0032).
When a Property has a lodged EPC and resolved Landlord Overrides (as Simulation
Overlays), the Effective EPC is the lodged EPC with those overlays applied so
the calculator scores what the landlord knows beyond the cert. With no overrides
the lodged EPC is returned unchanged.
"""
from __future__ import annotations
import json
from pathlib import Path
from typing import Any
from datatypes.epc.domain.epc_property_data import (
BuildingPartIdentifier,
EpcPropertyData,
)
from domain.epc.wall_type_overlay import wall_overlay_for
from domain.property.property import Property, PropertyIdentity
_JSON_SAMPLES = Path(__file__).resolve().parents[3] / "backend/epc_api/json_samples"
def _epc() -> EpcPropertyData:
raw: dict[str, Any] = json.loads(
(_JSON_SAMPLES / "RdSAP-Schema-21.0.0" / "epc.json").read_text()
)
from datatypes.epc.domain.mapper import EpcPropertyDataMapper
return EpcPropertyDataMapper.from_api_response(raw)
def _identity() -> PropertyIdentity:
return PropertyIdentity(
portfolio_id=1, postcode="A0 0AA", address="1 Some Street", uprn=12345
)
def _main_wall(epc: EpcPropertyData) -> Any:
return next(
part
for part in epc.sap_building_parts
if part.identifier is BuildingPartIdentifier.MAIN
)
def test_effective_epc_folds_the_wall_override_onto_the_main_part() -> None:
# Arrange — a Property with a lodged EPC and a solid-brick/internal-insulation
# wall override.
overlay = wall_overlay_for("Solid brick, with internal insulation", 0)
assert overlay is not None
prop = Property(identity=_identity(), epc=_epc(), landlord_overrides=[overlay])
# Act
main = _main_wall(prop.effective_epc)
# Assert — the override's codes are present on the main wall.
assert main.wall_construction == 3
assert main.wall_insulation_type == 3
def test_effective_epc_is_the_lodged_epc_when_there_are_no_overrides() -> None:
# Arrange — a Property with an EPC and no Landlord Overrides.
prop = Property(identity=_identity(), epc=_epc())
# Act
effective = prop.effective_epc
# Assert — the lodged EPC is returned untouched (same object, no fold).
assert effective is prop.epc
def test_baseline_wall_is_unchanged_when_no_override_applies() -> None:
# Arrange — the lodged main wall is cavity (construction 4).
prop = Property(identity=_identity(), epc=_epc())
# Act
main = _main_wall(prop.effective_epc)
# Assert
assert main.wall_construction == 4

View file

@ -28,11 +28,11 @@ from datatypes.epc.domain.epc_property_data import EpcPropertyData
from datatypes.epc.domain.mapper import EpcPropertyDataMapper
from datatypes.epc.search.epc_search_result import EpcSearchResult
from domain.epc_prediction.epc_prediction import EpcPrediction
from domain.epc_prediction.prediction_target import PredictionTargetAttributes
from domain.geospatial.coordinates import Coordinates
from domain.geospatial.planning_restrictions import PlanningRestrictions
from domain.geospatial.spatial_reference import SpatialReference
from domain.property.property import Property
from infrastructure.postgres.property_override_table import PropertyOverrideRow
from infrastructure.postgres.property_table import PropertyRow
from orchestration.ingestion_orchestrator import IngestionOrchestrator
from repositories.comparable_properties.epc_comparable_properties_repository import (
@ -41,6 +41,12 @@ from repositories.comparable_properties.epc_comparable_properties_repository imp
from repositories.epc.epc_postgres_repository import EpcPostgresRepository
from repositories.geospatial.geospatial_repository import GeospatialRepository
from repositories.postgres_unit_of_work import PostgresUnitOfWork
from repositories.property.override_backed_prediction_attributes_reader import (
OverrideBackedPredictionAttributesReader,
)
from repositories.property.property_overrides_postgres_reader import (
PropertyOverridesPostgresReader,
)
from repositories.property.property_postgres_repository import (
PropertyPostgresRepository,
)
@ -105,14 +111,6 @@ class _NoSolarFetcher:
return {}
class _FakeAttributesReader:
"""Stands in for Jun-te's property_overrides read adapter: the landlord-known
property type (here a House, code "0", matching the cohort)."""
def attributes_for(self, property_id: int) -> PredictionTargetAttributes:
return PredictionTargetAttributes(property_type="0", built_form="2")
def _cohort_results() -> list[EpcSearchResult]:
return [
EpcSearchResult(
@ -135,7 +133,9 @@ def test_epc_less_property_is_predicted_persisted_and_resolved_end_to_end(
db_engine: Engine,
) -> None:
# Arrange — an EPC-less Property exists in the database (postcode + UPRN known,
# no EPC lodged), plus its postcode cohort behind the faked EPC API.
# no EPC lodged), plus its postcode cohort behind the faked EPC API, plus the
# landlord overrides the finaliser resolved for it (House / Semi-Detached) that
# the real read adapter will translate into the gov-code space ("0" / "2").
with Session(db_engine) as session:
row = PropertyRow(
portfolio_id=1, postcode=_POSTCODE, address="1 Target Street", uprn=10000
@ -145,6 +145,28 @@ def test_epc_less_property_is_predicted_persisted_and_resolved_end_to_end(
property_id = row.id
assert property_id is not None
session.add(
PropertyOverrideRow(
property_id=property_id,
portfolio_id=1,
building_part=0,
override_component="property_type",
override_value="House",
original_spreadsheet_description="3-bed semi",
)
)
session.add(
PropertyOverrideRow(
property_id=property_id,
portfolio_id=1,
building_part=0,
override_component="built_form_type",
override_value="Semi-Detached",
original_spreadsheet_description="3-bed semi",
)
)
session.commit()
cohort_coords = {20000 + i: Coordinates(longitude=-1.55, latitude=53.81) for i in range(3)}
comparables_repo = EpcComparablePropertiesRepository(
_FakeCohortEpcClient(_cohort_results()), _FakeGeospatialRepo(cohort_coords)
@ -155,7 +177,9 @@ def test_epc_less_property_is_predicted_persisted_and_resolved_end_to_end(
geospatial_repo=_FakeGeospatialRepo({10000: Coordinates(longitude=-1.55, latitude=53.81)}),
solar_fetcher=_NoSolarFetcher(),
comparables_repo=comparables_repo,
prediction_attributes_reader=_FakeAttributesReader(),
prediction_attributes_reader=OverrideBackedPredictionAttributesReader(
PropertyOverridesPostgresReader(lambda: Session(db_engine))
),
epc_prediction=EpcPrediction(),
)

View file

@ -0,0 +1,86 @@
"""The landlord-overrides-backed PredictionTargetAttributesReader (ADR-0031).
Unit-level: a fake ``PropertyOverridesReader`` supplies value-space snapshots so
these tests pin the composition main-building selection, valuecode mapping,
and the gate (unresolvable property_type None) without a database.
"""
from __future__ import annotations
from repositories.property.override_backed_prediction_attributes_reader import (
OverrideBackedPredictionAttributesReader,
)
from repositories.property.property_overrides_reader import (
PropertyOverridesReader,
ResolvedPropertyOverride,
ResolvedPropertyOverrides,
)
class _FakeOverridesReader(PropertyOverridesReader):
def __init__(self, *rows: ResolvedPropertyOverride) -> None:
self._snapshot = ResolvedPropertyOverrides(rows=rows)
def overrides_for(self, property_id: int) -> ResolvedPropertyOverrides:
return self._snapshot
def test_main_building_property_type_is_mapped_to_its_gov_code() -> None:
# Arrange
reader = OverrideBackedPredictionAttributesReader(
_FakeOverridesReader(
ResolvedPropertyOverride("property_type", 0, "House"),
)
)
# Act
attributes = reader.attributes_for(1)
# Assert
assert attributes.property_type == "0"
def test_built_form_is_mapped_and_only_the_main_building_is_read() -> None:
# Arrange — main building is a House/Detached; an extension (part 1) carries a
# different property type that must not be read.
reader = OverrideBackedPredictionAttributesReader(
_FakeOverridesReader(
ResolvedPropertyOverride("property_type", 0, "House"),
ResolvedPropertyOverride("built_form_type", 0, "Detached"),
ResolvedPropertyOverride("property_type", 1, "Flat"),
)
)
# Act
attributes = reader.attributes_for(1)
# Assert — built_form mapped to its code; the part-1 "Flat" is ignored.
assert attributes.property_type == "0"
assert attributes.built_form == "1"
def test_unresolvable_property_type_gates_the_property_out() -> None:
# Arrange — the landlord override resolved only to "Unknown".
reader = OverrideBackedPredictionAttributesReader(
_FakeOverridesReader(
ResolvedPropertyOverride("property_type", 0, "Unknown"),
)
)
# Act
attributes = reader.attributes_for(1)
# Assert — None property_type makes build_prediction_target skip the Property.
assert attributes.property_type is None
def test_property_with_no_overrides_yields_no_attributes() -> None:
# Arrange — nothing resolved for the Property.
reader = OverrideBackedPredictionAttributesReader(_FakeOverridesReader())
# Act
attributes = reader.attributes_for(1)
# Assert
assert attributes.property_type is None
assert attributes.built_form is None

View file

@ -0,0 +1,117 @@
"""Integration tests for the ``property_overrides`` read adapter.
The reader is *faithful*: it returns the resolved enum-value snapshots exactly
as the finaliser wrote them, every ``(override_component, building_part)`` for
the Property, with no translation or gating. Verified against a real Postgres
(the ``db_engine`` fixture) because the value is in reading what was actually
persisted.
"""
from __future__ import annotations
from sqlalchemy import Engine
from sqlmodel import Session
from infrastructure.postgres.property_override_table import PropertyOverrideRow
from repositories.property.property_overrides_postgres_reader import (
PropertyOverridesPostgresReader,
)
def _seed(
session: Session,
*,
override_component: str,
override_value: str,
property_id: int = 1,
portfolio_id: int = 1,
building_part: int = 0,
) -> None:
row = PropertyOverrideRow(
property_id=property_id,
portfolio_id=portfolio_id,
building_part=building_part,
override_component=override_component,
override_value=override_value,
original_spreadsheet_description="detached house",
)
session.add(row)
def test_reads_a_resolved_override_in_value_space(db_engine: Engine) -> None:
# Arrange — the finaliser wrote one property_type override for the Property.
with Session(db_engine) as session:
_seed(
session,
property_id=42,
override_component="property_type",
override_value="House",
)
session.commit()
reader = PropertyOverridesPostgresReader(lambda: Session(db_engine))
# Act
resolved = reader.overrides_for(42)
# Assert — the stored enum value, untranslated.
assert resolved.value("property_type", 0) == "House"
def test_returns_every_component_and_building_part_for_the_property(
db_engine: Engine,
) -> None:
# Arrange — three overrides across two building parts for the target Property,
# plus an override for a *different* Property that must not leak in.
with Session(db_engine) as session:
_seed(
session,
property_id=7,
building_part=0,
override_component="property_type",
override_value="House",
)
_seed(
session,
property_id=7,
building_part=0,
override_component="built_form_type",
override_value="Detached",
)
_seed(
session,
property_id=7,
building_part=1,
override_component="wall_type",
override_value="Solid brick, with internal insulation",
)
_seed(
session,
property_id=8,
override_component="property_type",
override_value="Flat",
)
session.commit()
reader = PropertyOverridesPostgresReader(lambda: Session(db_engine))
# Act
resolved = reader.overrides_for(7)
# Assert — all three of the Property's rows, faithfully; none from Property 8.
assert len(resolved.rows) == 3
assert resolved.value("property_type", 0) == "House"
assert resolved.value("built_form_type", 0) == "Detached"
assert resolved.value("wall_type", 1) == "Solid brick, with internal insulation"
def test_property_without_overrides_reads_empty(db_engine: Engine) -> None:
# Arrange — nothing seeded for this Property.
reader = PropertyOverridesPostgresReader(lambda: Session(db_engine))
# Act
resolved = reader.overrides_for(999)
# Assert
assert resolved.rows == ()
assert resolved.value("property_type", 0) is None

View file

@ -0,0 +1,114 @@
"""PropertyPostgresRepository hydrates Landlord Overrides as overlays (ADR-0032).
Real Postgres end-to-end for the read side: a lodged EPC and a `wall_type`
override row are persisted, and the reloaded Property's Effective EPC reflects
the override folded onto the lodged wall proving reader overlay aggregate.
"""
from __future__ import annotations
import json
from pathlib import Path
from typing import Any
from sqlalchemy import Engine
from sqlmodel import Session
from datatypes.epc.domain.epc_property_data import (
BuildingPartIdentifier,
EpcPropertyData,
)
from datatypes.epc.domain.mapper import EpcPropertyDataMapper
from infrastructure.postgres.property_override_table import PropertyOverrideRow
from infrastructure.postgres.property_table import PropertyRow
from repositories.epc.epc_postgres_repository import EpcPostgresRepository
from repositories.property.property_overrides_postgres_reader import (
PropertyOverridesPostgresReader,
)
from repositories.property.property_postgres_repository import (
PropertyPostgresRepository,
)
_JSON_SAMPLES = Path(__file__).resolve().parents[3] / "backend/epc_api/json_samples"
def _epc() -> EpcPropertyData:
raw: dict[str, Any] = json.loads(
(_JSON_SAMPLES / "RdSAP-Schema-21.0.0" / "epc.json").read_text()
)
return EpcPropertyDataMapper.from_api_response(raw)
def test_reloaded_property_effective_epc_reflects_the_wall_override(
db_engine: Engine,
) -> None:
# Arrange — a Property with a lodged EPC (cavity main wall) and a solid-brick
# / internal-insulation wall override.
with Session(db_engine) as session:
row = PropertyRow(portfolio_id=1, postcode="A0 0AA", address="1 St", uprn=1)
session.add(row)
session.commit()
property_id = row.id
assert property_id is not None
EpcPostgresRepository(session).save(_epc(), property_id=property_id)
session.add(
PropertyOverrideRow(
property_id=property_id,
portfolio_id=1,
building_part=0,
override_component="wall_type",
override_value="Solid brick, with internal insulation",
original_spreadsheet_description="solid brick, insulated",
)
)
session.commit()
# Act — reload through the real repository with the overrides reader wired.
with Session(db_engine) as session:
repo = PropertyPostgresRepository(
session,
EpcPostgresRepository(session),
overrides_reader=PropertyOverridesPostgresReader(lambda: Session(db_engine)),
)
prop = repo.get(property_id)
main = next(
part
for part in prop.effective_epc.sap_building_parts
if part.identifier is BuildingPartIdentifier.MAIN
)
# Assert — the lodged cavity wall (4) is overlaid to solid brick (3) / internal (3).
assert main.wall_construction == 3
assert main.wall_insulation_type == 3
def test_property_without_overrides_keeps_its_lodged_wall(db_engine: Engine) -> None:
# Arrange — a Property with a lodged EPC but no override rows.
with Session(db_engine) as session:
row = PropertyRow(portfolio_id=1, postcode="A0 0AA", address="2 St", uprn=2)
session.add(row)
session.commit()
property_id = row.id
assert property_id is not None
EpcPostgresRepository(session).save(_epc(), property_id=property_id)
session.commit()
# Act
with Session(db_engine) as session:
repo = PropertyPostgresRepository(
session,
EpcPostgresRepository(session),
overrides_reader=PropertyOverridesPostgresReader(lambda: Session(db_engine)),
)
prop = repo.get(property_id)
main = next(
part
for part in prop.effective_epc.sap_building_parts
if part.identifier is BuildingPartIdentifier.MAIN
)
# Assert — the lodged cavity wall (4) is untouched.
assert main.wall_construction == 4