Add the DB-round-trip corpus gate that closes the blind spot 🟩

The accuracy corpus scores straight off the mapped fixture and never saves, so it
is structurally blind to any field the calculator reads but the schema drops
(#1665). This gate maps -> saves to the ephemeral test DB -> reads back ->
re-scores every corpus cert and asserts the reloaded SAP equals the in-memory SAP
to 1e-9. It would have caught cylinder_heat_loss / room-in-roof / roof-windows /
wall_u_value on day one; a new drop now fails here immediately.

Save+read+rollback per cert keeps the graph off disk (bounded, ~18s over 1000
certs). One documented, precise skip: the pre-#1666 decimal main_heating_fraction
cohort, whose INTEGER column truncates the raw decimal on save (0.8 -> 1) — fixed
upstream by #1666 (mapper normalisation, on #1672). Remove the skip once #1672 is
in main so the gate asserts a hard zero.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Khalim Conn-Kowlessar 2026-07-23 09:23:26 +00:00
parent 50493f07c6
commit eba2142cdf

View file

@ -0,0 +1,97 @@
"""Durable DB-round-trip gate for the persistence blind spot (#1665).
Production scores a dwelling AFTER a Postgres save+reload; the accuracy corpus
(`test_sap_accuracy_corpus.py`) scores straight off the mapped fixture and never
saves, so it is structurally blind to any field the calculator reads but the
schema drops. This gate closes that hole: every corpus cert is mapped, saved to
the ephemeral test DB, read back, and re-scored and the reloaded SAP must equal
the in-memory SAP to 1e-9. Any field the calculator reads but the schema drops
makes the two diverge, so a new persistence blind spot fails here immediately
instead of being found by hand months later (it would have caught
cylinder_heat_loss / room-in-roof / roof-windows / wall_u_value on day one).
Known, documented exception the `main_heating_fraction` DECIMAL cohort. The
gov-EPC API lodges a two-main split as either a decimal fraction (`[0.8, 0.2]`)
or a percent (`[80, 20]`); the `epc_main_heating_detail.main_heating_fraction`
column is INTEGER, so a decimal pair truncates on save (`0.8 -> 1`). The real fix
is #1666 (normalise to percent AT THE MAPPER, before save) which lands on the
SAP-accuracy branch (#1672). Until #1672 is in `main`, those certs cannot
round-trip; they are skipped here via a precise predicate (a non-integer lodged
fraction). Once #1672 merges and this branch rebases, delete `_is_pre_1666_*`
and the skip so the gate asserts a hard zero.
"""
from __future__ import annotations
import json
from pathlib import Path
from typing import Any
from sqlalchemy import Engine
from sqlmodel import Session
from datatypes.epc.domain.epc_property_data import EpcPropertyData
from datatypes.epc.domain.mapper import EpcPropertyDataMapper
from domain.sap10_calculator.calculator import Sap10Calculator
from repositories.epc.epc_postgres_repository import EpcPostgresRepository
_CORPUS = (
Path(__file__).resolve().parents[3]
/ "backend/epc_api/json_samples/RdSAP-Schema-21.0.1/corpus.jsonl"
)
_TOLERANCE = 1e-9
def _is_pre_1666_decimal_fraction(epc: EpcPropertyData) -> bool:
"""True when a lodged main_heating_fraction is non-integer, so the INTEGER
column truncates it on save. #1666 (mapper normalisation) removes this;
remove this predicate + skip once #1672 is in main."""
for main in epc.sap_heating.main_heating_details:
frac = main.main_heating_fraction
if frac is not None and float(frac) != int(frac):
return True
return False
def test_round_trip_preserves_sap_over_the_corpus(db_engine: Engine) -> None:
# Arrange
calc = Sap10Calculator()
lines = _CORPUS.read_text().splitlines()
# Act — map -> save -> reload -> re-score every cert, collecting any drift.
diverged: list[str] = []
skipped_pre_1666 = 0
for line in lines:
raw: dict[str, Any] = json.loads(line)
epc = EpcPropertyDataMapper.from_api_response(raw)
if epc is None:
continue
if _is_pre_1666_decimal_fraction(epc):
skipped_pre_1666 += 1
continue
try:
direct = calc.calculate(epc).sap_score_continuous
except Exception:
continue
# Save -> flush (assigns FKs, writes the graph within the txn) -> read
# back in the SAME session -> roll back, so the round-trip exercises the
# real save/reconstruct without accumulating 1000 graphs on disk.
with Session(db_engine) as session:
repo = EpcPostgresRepository(session)
epc_property_id = repo.save(epc)
session.flush()
reloaded = repo.get(epc_property_id)
session.rollback()
reloaded_sap = calc.calculate(reloaded).sap_score_continuous
if abs(reloaded_sap - direct) > _TOLERANCE:
diverged.append(
f"uprn {raw.get('uprn')}: in-memory {direct:.6f} != reloaded "
f"{reloaded_sap:.6f}{reloaded_sap - direct:+.4f})"
)
# Assert — no field the calculator reads is dropped on the DB round-trip.
assert not diverged, (
f"{len(diverged)} cert(s) lost calculator-read fields on the DB round-trip "
f"({skipped_pre_1666} pre-#1666 decimal-fraction certs skipped):\n"
+ "\n".join(diverged[:20])
)