Model/scripts/lisasrequest/reclassify_wood_logs_main_fuel.py
Jun-te Kim 5330bc20d9 Add individual wood-logs main-fuel archetype (API 6 / RdSAP Table 32 code 20)
MainFuelType had no individual wood-logs member — only "biomass (community)" —
so the LLM classifier funnelled "Solid Fuel: Wood Logs" into the community fuel,
inventing a community heat network the dwelling isn't on (and mislabelling the
connection). main_fuel had no deterministic guard at all, so nothing caught it.

Verified against domain/sap10_calculator/docs/specs: RdSAP 10 Specification
Table 32 lists "wood logs" as a solid fuel (code 20, 0.028 kgCO2e/kWh); the
calculator's input scheme (the gov EPC API fuel enum) codes it 6 -> Table 32 20
(sap_efficiencies._API_TO_TABLE32), and water_heating_overlay already pins the
same fuel to 6. So _FUEL_CODES["wood logs"] = 6 is confirmed, not guessed.

Adds MainFuelType.WOOD_LOGS + the _FUEL_CODES entry, a main_fuel_guard mirroring
water_heating_guard (claims the "wood log" token; dual fuel keeps its own member
since it has no "wood log" substring), and wires main_fuel through a
GuardedColumnClassifier so the live path is deterministic.

Applied the scoped backfill to portfolio 796 (Hyde): 21 rows off
"biomass (community)" -> "wood logs". property_overrides (TEXT) only; the
classifier-cache pgEnum member is deferred to the FE Drizzle migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 17:00:24 +00:00

124 lines
4.6 KiB
Python

"""Backfill main-fuel overrides where an individual wood-logs dwelling was funnelled
into "biomass (community)".
The ``MainFuelType`` taxonomy had no individual wood-logs member (only
``biomass (community)``), so the LLM force-picked the community fuel for
``"Solid Fuel: Wood Logs"`` — inventing a community heat network the dwelling
isn't on. ``MainFuelType.WOOD_LOGS`` (API fuel enum 6 / RdSAP Table 32 code 20)
now exists and ``main_fuel_guard`` resolves the wood-logs token deterministically,
so this fixes the rows written before that change.
Uses the SAME guard as the live path, so the backfill and the classifier cannot
drift. SCOPED TO ONE PORTFOLIO (``--portfolio``, default 796 = Hyde) and only
touches ``property_overrides.override_value`` (TEXT — what the modelling reads);
the global ``landlord_main_fuel_overrides`` classifier cache is left alone (its
``value`` is the Drizzle-owned pgEnum, which needs the FE migration to add the
member first — the Class-A/B deferral Khalim's scripts use).
DRY-RUN BY DEFAULT: prints what it would change and writes nothing. Pass
``--apply`` to execute inside a transaction; it also writes an audit CSV of every
row changed (property_id, uprn, old value, new value) so the change is reversible.
Idempotent — only rows whose stored value differs from the guard's target member
are touched.
python -m scripts.lisasrequest.reclassify_wood_logs_main_fuel # dry run
python -m scripts.lisasrequest.reclassify_wood_logs_main_fuel --apply # write
"""
from __future__ import annotations
import argparse
import csv
import sys
from collections import Counter
from pathlib import Path
from sqlalchemy import text
_REPO_ROOT = Path(__file__).resolve().parents[2]
sys.path.insert(0, str(_REPO_ROOT))
from domain.epc.property_overrides.main_fuel_guard import main_fuel_guard # noqa: E402
from scripts.e2e_common import ENV_PATH, build_engine, load_env # noqa: E402
_SELECT = text(
"""
SELECT po.property_id, pr.uprn,
po.original_spreadsheet_description AS description,
po.override_value AS value
FROM property_overrides po
JOIN property pr ON pr.id = po.property_id
WHERE po.portfolio_id = :portfolio
AND po.override_component = 'main_fuel'
"""
)
_UPDATE = text(
"""
UPDATE property_overrides
SET override_value = :new_value
WHERE portfolio_id = :portfolio
AND override_component = 'main_fuel'
AND property_id = :property_id
AND override_value <> :new_value
"""
)
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--portfolio", type=int, default=796)
parser.add_argument(
"--apply",
action="store_true",
help="execute the updates (default: dry-run, writes nothing)",
)
args = parser.parse_args()
load_env(ENV_PATH)
engine = build_engine()
audit: list[tuple[int, object, str, str, str]] = [] # pid, uprn, descr, old, new
tally: Counter[str] = Counter()
with engine.begin() as conn:
conn.execute(text("SET statement_timeout = 120000"))
for property_id, uprn, description, value in conn.execute(
_SELECT, {"portfolio": args.portfolio}
):
member = main_fuel_guard(description or "")
if member is None or value == member.value:
continue
tally[f"{value!r} -> {member.value!r}"] += 1
audit.append((property_id, uprn, description, value, member.value))
if args.apply:
conn.execute(
_UPDATE,
{
"portfolio": args.portfolio,
"property_id": property_id,
"new_value": member.value,
},
)
if not args.apply:
conn.rollback()
verb = "re-classified" if args.apply else "would re-classify"
print(f"portfolio {args.portfolio}: {verb} {len(audit)} main-fuel override row(s)")
for change, n in tally.most_common():
print(f" {n:5d} {change}")
if args.apply and audit:
out = _REPO_ROOT / "scripts" / "lisasrequest" / (
f"reclassify_wood_logs_main_fuel_{args.portfolio}_audit.csv"
)
with out.open("w", newline="") as fh:
w = csv.writer(fh)
w.writerow(["property_id", "uprn", "description", "old_value", "new_value"])
w.writerows(audit)
print(f"\naudit trail: {out}")
if not args.apply:
print("\nDRY-RUN — nothing written. Re-run with --apply to execute.")
return 0
if __name__ == "__main__":
raise SystemExit(main())