mirror of
https://github.com/Hestia-Homes/Model.git
synced 2026-06-30 13:10:47 +00:00
re run ddd tests
This commit is contained in:
Jun-te Kim
parent
027ba6d89f
commit
1b53c57a07
5 changed files with 88 additions and 98 deletions
|
|
@ -199,6 +199,27 @@ Pattern: `with E.session() as (ctx,page): E.goto(...); E.set_text/set_select(...
|
|||
authoritative validation — if the Elmhurst rebuild diverges, suspect an omitted
|
||||
lodged feature (secondary / meter), confirm via engine-on-Elmhurst-inputs ≈
|
||||
worksheet, and pin the engine = lodged value.
|
||||
⚠ **`present=No` does NOT clear the shared assessment's leftover secondary** — the
|
||||
prior cert's secondary SYSTEM (fuel + SAP code) persists server-side even when the
|
||||
UI dropdown reads "No", and it silently re-enters the worksheet. On a storage-heater
|
||||
cert (uprn_100020665611, RdSAP-20.0.0 end-terrace house) a leftover House-Coal
|
||||
closed-room-heater (SAP 633, cheap solid fuel) inflated the Elmhurst worksheet to
|
||||
44 vs engine 36 / lodged 37 until OVERWRITTEN. Always set the secondary EXPLICITLY
|
||||
to the lodged appliance: storage-heater certs lodge "Portable electric heaters
|
||||
(assumed)" → present=Yes → `ButtonSecondaryHeatingCode` cascade Electric → Electric
|
||||
→ Room Heaters → "REA Panel, convector or radiant heaters" (= SAP 691, eff 100%).
|
||||
That dropped the worksheet 44 → 35 ≡ engine-on-Elmhurst-inputs 35 (faithful).
|
||||
- **Openings: standard double/triple-glazing bands REQUIRE Frame Type + Glazing Gap.**
|
||||
The window grid's `DropDownListExtFrameType` (PVC/Wood/Metal) and
|
||||
`DropDownListExtGlazingGap` (6 mm / 12 mm / 16 mm or more) are required for any
|
||||
band other than the new-build "Double post or during 2022" default — leaving them
|
||||
empty fails the Recommendations gate ("Openings: Frame Type / Glazing Gap must be
|
||||
entered"). Set both in `_add_window` BEFORE clicking Add (cert `pvc_window_frames`
|
||||
→ PVC; `glazing_gap` mm → the band). Glazing bands available: Single / Double|Triple
|
||||
{pre 2002, between 2002 and 2021, post or during 2022, with unknown install date} /
|
||||
Secondary / *…known data*. For RdSAP double-glazing with no lodged install year
|
||||
(engine U 2.8) pick "Double pre 2002" (~U3.1, sub-1-SAP diff) — the "…known data"
|
||||
options demand per-row U/g values.
|
||||
- **Non-boiler (storage-heater) main heating — SOLVED** (uprn_10022893721, RdSAP-
|
||||
18.0 GF flat, electric storage heaters + immersion). The SpaceHeating page has NO
|
||||
inline system-type selector and a `ButtonMainHeatingCode` button only APPEARS
|
||||
|
|
@ -301,6 +322,23 @@ Pattern: `with E.session() as (ctx,page): E.goto(...); E.set_text/set_select(...
|
|||
and blocks the report. For a boiler programmer+room-stat+TRVs (SAP 2106):
|
||||
`set_heating_dialog(..ButtonMainHeatingControls, '^Boilers', '^Standard', 'CBE
|
||||
Programmer, room thermostat and TRVs')`.
|
||||
⚠ **Zone-control codes are under L2 "Zone control", NOT "Standard".** SAP 2110
|
||||
"Time and temperature zone control" = `set_heating_dialog(.., '^Boilers', 'Zone
|
||||
control', 'CBI Time and temperature zone control')` (Boilers→Standard L3 only has
|
||||
CBA…CBH — no zone option, so a wrong L2 leaves the cascade incomplete and the OK
|
||||
click is swallowed by the modalBackground, hanging the run). CBL is the PCDF
|
||||
variant. Match the lodged `main_heating_control`: 2110 → CBI, 2106 → CBE (Standard).
|
||||
- **Removing an inherited secondary when the cert lodges NONE** — `present=No` ALONE
|
||||
does NOT remove it: a persisted secondary CODE (e.g. an electric REA/691 from a
|
||||
prior cert) survives the dropdown=No and the worksheet still applies it at Table-11
|
||||
fraction 0.1 (worth ~1.5 SAP on a gas-combi cert — the electric secondary is priced
|
||||
at the 13.19p peak rate vs gas 3.48p). The flag and the code are independent; the
|
||||
worksheet keys off the CODE. ✅ FIX: `present=Yes` (exposes `TextBoxSecondaryHeating
|
||||
Code`) → JS-clear that textbox to `''` + dispatch change → `present=No` → Save&Close.
|
||||
An EMPTY code drops the worksheet's "Secondary Heating" to None / fraction 0.0.
|
||||
Verify by re-downloading: worksheet line (201) "Fraction of space heat from
|
||||
secondary" must read 0.0000. Seen on uprn_10090944225 (worksheet 79→80; closed the
|
||||
cross-check gap to the residual +1 floor-U difference).
|
||||
|
||||
## Limitations / next improvements (make the campaign scale)
|
||||
- **Per-assessment GUID** — `elmhurst_lib.py` reuses one `ASSESSMENT_GUID`
|
||||
|
|
|
|||
|
|
@ -74,9 +74,9 @@ Skip the 🚩 MVHR / 🚩 heat-pump-fuel and ⛔ sparse certs.
|
|||
- [x] 10093115480 — SAP-17.1 (full-SAP END-TERRACE BUNGALOW single-storey, band L 2016, combi PCDB 16841, party wall 10.99m² CF filled, natural vent + 2 fans, AP50 3.85, 9 LED, measured U 0.19/0.11/0.12, TFA 56) · eng 81 / elm 78 (lodged 81) · PINNED engine 81 = lodged EXACT. +3 = documented full-SAP→RdSAP residual (measured U beats band-L defaults); engine-on-Elmhurst-inputs 78.29 ≈ worksheet 78 → faithful. Built combi. No mapper change.
|
||||
- [🔍] 68151071 — RdSAP-17.0 (semi-detached BUNGALOW single-storey, band H 1991-1995, cavity insulated, REGULAR boiler PCDB 17550 Worcester Greenstar 18Ri + cylinder size2/foam/50mm, pitched 200mm loft, suspended uninsulated floor, party wall 9.48 lodged, TFA 50) · eng 68 / elm 71 (lodged 70) · 🔍 FLAGGED — MAPPER GAP (cylinder insulation thickness dropped). engine -3 vs Elmhurst, -2 vs lodged; engine HW 3446 kWh vs Elmhurst 2911. ROOT CAUSE confirmed: raw cert lodges sap_heating.cylinder_insulation_thickness=50 but the RdSAP-17.0 mapper maps cylinder_size + cylinder_insulation_type only, leaving EpcPropertyData.sap_heating.cylinder_insulation_thickness_mm=None → engine assumes a poorly-insulated cylinder → over-counts HW cylinder loss → under-rates. FIX: carry cylinder_insulation_thickness → cylinder_insulation_thickness_mm in the RdSAP mapper (check blast radius across schemas + regression). engine-on-Elmhurst-inputs 71.34 ≈ worksheet 71 → calculator faithful (gap is purely the dropped input). build_68151071.py + evidence saved.
|
||||
- [x] 100021985993 — SAP-16.2 (END-TERRACE BUNGALOW single-storey, band C, solid-brick internal insulation, combi PCDB 10328, double glazed, 100mm loft, suspended uninsulated floor) · eng 74 / elm 72 (lodged 70) · PINNED engine 74. Built in Elmhurst (end-terrace, boiler 10328, control CBE/2106, water from primary, party wall 6.89 solid masonry — Elmhurst requires non-zero for an end-terrace). +2 vs Elmhurst = documented 16.2 reduced-field party-wall gap (gov-API lodges no party_wall_length → engine models none; worksheet's only extra element is party wall 16.19m²×U0.25=4.05 W/K ≈ 2 SAP). engine-on-Elmhurst-PDF-inputs 67 is PDF-parser noise (HW over-parsed), not a calc bug.
|
||||
- [ ] 100020665611 — RdSAP-20.0.0 · eng 36 / lodged 37
|
||||
- [x] 100020665611 — RdSAP-20.0.0 (END-TERRACE HOUSE 2-storey, band E 1967-1975, cavity UNINSULATED, ELECTRIC STORAGE HEATERS SAP 402 SEB + manual charge CSA/2401, immersion off-peak Economy-7 Dual + cylinder Normal/110L foam 38mm, double glazed PVC, party wall 9.5m/floor U0.25, secondary portable electric heaters/SAP 691, TFA 87.4) · eng 36 / elm 35 (lodged 37) · PINNED engine 36. Built in Elmhurst (storage SEB, CSA control, immersion Dual + cylinder, **Dual electricity meter**, secondary electric room heater). engine-on-Elmhurst-inputs 35 ≡ worksheet 35 → calculator faithful; eng 36 ≈ lodged 37 (−1). Cylinder thickness 38mm correctly carried by RdSAP-20.0.0 mapper (no cylinder-gap here). ⚠ Secondary had to be set EXPLICITLY: shared-assessment leftover House-Coal secondary (633) survived `present=No` and inflated worksheet to 44 until overwritten with electric room heater. build_100020665611.py. No mapper change.
|
||||
- [⚠] 10093388044 — SAP-17.1 · eng 87 / lodged 93 · 🚩 heat-pump fuel-39 (flagged)
|
||||
- [ ] 10090944225 — SAP-17.0 · eng 81 / lodged 82
|
||||
- [x] 10090944225 — SAP-17.0 (full-SAP GROUND-FLOOR FLAT, band L 2015, combi PCDB 17045 Ideal Logic Combi ES35, control 2110 CBI time+temp zone, NATURAL vent + 2 extract fans, AP50 3.48, party wall 3.41m² U0, measured U walls 0.27/floor 0.14/window 1.41, TFA 49.97, no cylinder/no secondary) · eng 81 / elm 80 (lodged 82) · PINNED engine 81. Built in Elmhurst (boiler 17045 via search, control CBI/2110 under Boilers→Zone control, water from primary, AP50 Blower Door 3.48+cert, Single meter). engine-on-Elmhurst-inputs 80 = worksheet 80 (exact) → faithful; eng 81 ≈ lodged 82 (−1). Remaining +1 = documented full-SAP→RdSAP residual localised to the FLOOR (cert measured U 0.14 vs Elmhurst RdSAP solid default 0.23; walls/party match) — engine correctly uses measured 0.14, NOT a mapper bug. ⚠ Initial worksheet read 79 due to a spurious leftover REA/691 electric secondary (Table-11 frac 0.1, ~1.5 SAP) the shared assessment retained — REMOVED via present=Yes→clear secondary code to empty→present=No (79→80). 🔧 Build notes: storage→combi reset (clear leftover SEB MainHeatingCode pass-1, reset meter Single); control 2110 is Boilers→**Zone control**→CBI (not Standard). No mapper change.
|
||||
- [⚠] 10090341811 — SAP-17.0 · eng 80 / lodged 89 · 🚩 MVHR idx 500352 not credited (flagged)
|
||||
- [ ] 10010215568 — RdSAP-17.1 · eng 75 / lodged 74
|
||||
- [⚠] 10093117227 — SAP-17.1 · eng 90 / lodged 80 · 🚩 heat-pump fuel-39 (flagged)
|
||||
|
|
|
|||
|
|
@ -28,7 +28,7 @@ SESSION_DIR = HERE / ".elmhurst-session"
|
|||
SAMPLE_DIR = (
|
||||
HERE.parent.parent
|
||||
/ "backend/epc_api/json_samples/real_life_examples"
|
||||
/ "RdSAP-Schema-17.0/uprn_68151071"
|
||||
/ "SAP-Schema-17.0/uprn_10090944225"
|
||||
)
|
||||
|
||||
ASSESSMENT_GUID = "B44A0DB4-4C08-4241-B818-86F060172105"
|
||||
|
|
|
|||
|
|
@ -510,6 +510,53 @@ _EXPECTATIONS: Final[tuple[RealCertExpectation, ...]] = (
|
|||
cert_num="8393-7438-5230-3319-1996",
|
||||
sap_score=81,
|
||||
),
|
||||
# UPRN 100020665611 → cert 2846-1002-5208-6669-0204. RdSAP-Schema-20.0.0 —
|
||||
# native RdSAP, END-TERRACE HOUSE, 2-storey, band E (1967-1975), cavity
|
||||
# UNINSULATED walls (U 1.5), pitched roof "limited insulation" (U 1.5), solid
|
||||
# uninsulated floor (U 0.63), double glazed PVC (~12.9 m², U 2.8), party wall
|
||||
# 9.5 m/floor (construction code 0 → U 0.25 "unable to determine, house"), TFA
|
||||
# 87.4. Heating: ELECTRIC STORAGE HEATERS (SAP 402 = SEB Modern slimline, manual
|
||||
# charge control 2401 = CSA) + electric immersion off-peak (Economy-7 Dual meter)
|
||||
# with cylinder (size 2 = Normal/110 L, foam, 38 mm); secondary "Portable
|
||||
# electric heaters (assumed)" (Elmhurst SAP 691, eff 100%). Lodged 37; engine 36.
|
||||
# Built in Elmhurst RdSAP10 (evidence saved): worksheet SAP 35, on the same
|
||||
# off-peak Dual tariff. engine on Elmhurst's own parsed inputs = 35 ≡ worksheet
|
||||
# 35 → calculator faithful; engine 36 ≈ lodged 37 (−1). The cylinder insulation
|
||||
# thickness (38 mm) IS carried by the RdSAP-20.0.0 mapper (line ~1701), so the
|
||||
# banked RdSAP-17.0 cylinder-thickness gap does NOT bite here. NB the Elmhurst
|
||||
# build needed the secondary set EXPLICITLY (Electric room heater): the shared
|
||||
# assessment's leftover House-Coal secondary (633) survived `present=No` and
|
||||
# inflated the worksheet to 44 until overwritten. PINNED to the observed engine
|
||||
# 36 — mapping untuned.
|
||||
RealCertExpectation(
|
||||
schema="RdSAP-Schema-20.0.0",
|
||||
sample="uprn_100020665611",
|
||||
cert_num="2846-1002-5208-6669-0204",
|
||||
sap_score=36,
|
||||
),
|
||||
# UPRN 10090944225 → cert 2298-4036-7306-3186-4930. SAP-Schema-17.0 — full-SAP
|
||||
# GROUND-FLOOR FLAT, band L (built 2015), measured U walls 0.27 / floor 0.14 /
|
||||
# window 1.41, mains-gas COMBI (PCDB 17045 Ideal Logic Combi ES35), control 2110
|
||||
# (CBI time + temperature zone control), NATURAL ventilation + 2 intermittent
|
||||
# extract fans, AP50 3.48, party wall only 3.41 m² (U 0), ~6.6 m² double glazing,
|
||||
# no cylinder, no secondary, TFA 49.97. Lodged 82; engine 81. Built in Elmhurst
|
||||
# RdSAP10 (evidence saved): worksheet SAP 80. engine on Elmhurst's own parsed
|
||||
# inputs = 80 = worksheet 80 (exact) → calculator faithful; engine 81 ≈ lodged 82
|
||||
# (−1). The remaining +1 (engine 81 vs Elmhurst 80) is the documented full-SAP→
|
||||
# RdSAP residual, localised to the FLOOR: the cert lodges measured floor U 0.14
|
||||
# but Elmhurst (U-known override disabled) recomputes the RdSAP band-L solid-floor
|
||||
# default 0.23 (~4.5 W/K on 50 m²); walls match (measured 0.27 ≈ Elmhurst 0.28),
|
||||
# party wall U 0 both sides. The engine correctly uses the measured 0.14 — NOT a
|
||||
# mapper bug. (An earlier build read 79 from a spurious leftover REA/691 electric
|
||||
# secondary the shared assessment retained at Table-11 fraction 0.1 — worth ~1.5
|
||||
# SAP; removed by present=Yes → clear the secondary code to empty → present=No,
|
||||
# taking the worksheet 79→80.) PINNED to the observed engine 81 — mapping untuned.
|
||||
RealCertExpectation(
|
||||
schema="SAP-Schema-17.0",
|
||||
sample="uprn_10090944225",
|
||||
cert_num="2298-4036-7306-3186-4930",
|
||||
sap_score=81,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,95 +0,0 @@
|
|||
"""End-to-end smoke of the Hyde override script for ONE property, against a real
|
||||
(ephemeral) Postgres. Seeds the landlord vocab (simulating post-classify, so no
|
||||
ChatGPT) + a minimal ``property`` row, then runs the script's real
|
||||
``write`` + ``verify`` paths and asserts property_overrides + overlays land.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import Engine, text
|
||||
from sqlmodel import Session
|
||||
|
||||
import scripts.hyde.build_property_overrides as b
|
||||
from domain.epc.property_overrides.built_form_type import BuiltFormType
|
||||
from domain.epc.property_overrides.construction_age_band import ConstructionAgeBand
|
||||
from domain.epc.property_overrides.glazing_type import GlazingType
|
||||
from domain.epc.property_overrides.main_fuel_type import MainFuelType
|
||||
from domain.epc.property_overrides.main_heating_system_type import MainHeatingSystemType
|
||||
from domain.epc.property_overrides.property_type import PropertyType
|
||||
from domain.epc.property_overrides.roof_type import RoofType
|
||||
from domain.epc.property_overrides.wall_type import WallType
|
||||
from domain.epc.property_overrides.water_heating_type import WaterHeatingType
|
||||
from infrastructure.landlord_overrides.landlord_overrides_postgres_repository import (
|
||||
LandlordOverridesRepository,
|
||||
)
|
||||
from repositories.property.landlord_override_overlays import overlays_from
|
||||
from repositories.property.property_overrides_postgres_reader import (
|
||||
PropertyOverridesPostgresReader,
|
||||
)
|
||||
|
||||
PORTFOLIO = 795
|
||||
ORG_REF = "55180004001"
|
||||
EXCEL = "scripts/hyde/hyde_property_overrides.xlsx"
|
||||
|
||||
# What ChatGPT WOULD resolve this property's 9 descriptions to (component ->
|
||||
# (raw Excel entry, enum member)). Seeded into the landlord ledger.
|
||||
SEED = {
|
||||
"property_type": ("House: MidTerrace", PropertyType.HOUSE),
|
||||
"built_form_type": ("House: MidTerrace", BuiltFormType.MID_TERRACE),
|
||||
"wall_type": ("TimberFrame: AsBuilt", WallType.TIMBER_FRAME_AS_BUILT_NO_INSULATION_ASSUMED),
|
||||
"roof_type": ("PitchedNormalLoftAccess: 300mm", RoofType.PITCHED_LOFT_300MM),
|
||||
"construction_age_band": ("L: 2012-2022", ConstructionAgeBand.L_2012_2022),
|
||||
"main_fuel": ("Gas: Mains Gas", MainFuelType.MAINS_GAS),
|
||||
"glazing": ("100% Double glazing 2002 or later", GlazingType.DOUBLE_POST_2002),
|
||||
"water_heating": ("From main heating system: Mains Gas", WaterHeatingType.FROM_MAIN_MAINS_GAS),
|
||||
"main_heating_system": ("Boiler: C rated Combi", MainHeatingSystemType.GAS_COMBI),
|
||||
}
|
||||
|
||||
|
||||
def test_one_property_end_to_end(db_engine: Engine, monkeypatch: Any) -> None:
|
||||
specs = b._specs_by_component() # pyright: ignore[reportPrivateUsage]
|
||||
|
||||
# minimal FE-owned `property` table + the one row we'll match by org_ref
|
||||
with Session(db_engine) as s:
|
||||
s.execute(text( # pyright: ignore[reportDeprecated]
|
||||
"CREATE TABLE property (id bigint PRIMARY KEY, portfolio_id bigint, "
|
||||
"landlord_property_id text)"))
|
||||
s.execute(text("INSERT INTO property VALUES (1, :p, :ref)"), # pyright: ignore[reportDeprecated]
|
||||
{"p": PORTFOLIO, "ref": ORG_REF})
|
||||
# seed the classifier ledger (keyed on normalised description)
|
||||
for comp, (raw, member) in SEED.items():
|
||||
repo: LandlordOverridesRepository[Any] = LandlordOverridesRepository(
|
||||
s, specs[comp].row_type)
|
||||
repo.upsert_all(PORTFOLIO, {b._norm(raw): member}) # pyright: ignore[reportPrivateUsage]
|
||||
s.commit()
|
||||
|
||||
# point the script at the ephemeral engine
|
||||
monkeypatch.setattr(b, "_db_session", lambda: Session(db_engine))
|
||||
|
||||
# --- run the real write() for this one property ---
|
||||
b.write(argparse.Namespace(excel=EXCEL, sheet="AddressProfilingResults",
|
||||
portfolio_id=PORTFOLIO, org_ref=ORG_REF, limit=None, apply=True))
|
||||
|
||||
with Session(db_engine) as s:
|
||||
rows = list(s.execute(text( # pyright: ignore[reportDeprecated]
|
||||
"SELECT override_component, building_part, override_value "
|
||||
"FROM property_overrides WHERE property_id = 1 ORDER BY override_component")))
|
||||
got = {c: v for c, _, v in rows}
|
||||
# every seeded component produced a property_overrides row with the resolved value
|
||||
assert got["main_fuel"] == "mains gas"
|
||||
assert got["glazing"] == "Double glazing, 2002 or later"
|
||||
assert got["construction_age_band"] == "L"
|
||||
assert got["main_heating_system"] == "Gas boiler, combi"
|
||||
assert got["water_heating"] == "From main system, mains gas"
|
||||
assert len(rows) == 9 # all 9 components
|
||||
|
||||
# --- the overrides reach the SAP overlay surface ---
|
||||
b.verify(argparse.Namespace(portfolio_id=PORTFOLIO, org_ref=ORG_REF)) # exercises verify()
|
||||
overlays = overlays_from(
|
||||
PropertyOverridesPostgresReader(lambda: Session(db_engine)).overrides_for(1))
|
||||
assert len(overlays) == 9
|
||||
assert any(o.heating is not None and o.heating.main_fuel_type == 26 for o in overlays)
|
||||
assert any(o.glazing is not None and o.glazing.glazing_type == 2 for o in overlays)
|
||||
Loading…
Add table
Reference in a new issue