datatypes/epc/domain/mapper.py's from_rdsap_schema_{17_0,17_1,18_0,19_0,
20_0_0,21_0_0} all stored a lodged sap_windows[].glazing_type verbatim
instead of routing it through _api_cascade_glazing_type (which
_api_sap_window, used by 21_0_1, already does correctly). The raw API code
is the RdSAP-21 glazing enum (e.g. code 1 = "DG pre-2002"), not the SAP
10.2 Table 6b cascade enum the calculator's U/g-value tables are keyed on
(cascade code 1 = single glazed) -- so any lodged window whose raw code
happened to collide with a different cascade meaning was silently
mis-rated.
Found while validating PR #1503 (property 753950 / uprn 100021969385)
against Elmhurst: our engine's windows_w_per_k was 82.27 vs Elmhurst's
51.44 -- the dwelling's larger window (91% of glazed area) lodges raw
glazing_type=1 and was being modelled as single glazed instead of double,
the dominant contributor to a SAP 42 vs Elmhurst 44 gap. Fixing this
narrows windows_w_per_k to 50.26 (Elmhurst 51.44, ~2% off) and moves the
engine's score to 43.
Extracted the 5 duplicated reduced-schema-window blocks (17.0/17.1/18.0/
19.0/20.0.0, which lodge a minimal window shape with no glazing_gap/
frame_factor/transmission fields) into a shared _reduced_field_api_sap_
window helper that also now populates window_transmission_details/
frame_factor via the SAP10 lookup instead of leaving them None. 21.0.0's
richer inline block gets the same one-line cascade fix.
+6 regression tests (one per affected schema seam + 21.0.0). Full domain/
schema/prediction/comparable-properties/sap10_calculator/modelling_e2e
suites pass (2746 passed, only the 2 pre-existing unrelated failures);
component-accuracy gate 26/26 unaffected.
Address PR review (dancafc):
- introduce UprnMatch NamedTuple (datatypes/address_match.py) for the
(uprn, address, lexiscore, certificate_number) return, replacing the bare
4-tuple in get_uprn_from_epc_df / get_uprn_from_historic_epc /
HistoricEpcResolver.resolve_uprn. Tuple-compatible, so unpacking is unchanged.
- rename get_uprn_with_epc_df -> get_uprn_from_epc_df (+ callers).
- type resolve_group_ambiguity via a GroupDecision NamedTuple and trim its
docstring.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- unit tests for resolve_group_ambiguity (distinct addresses withheld,
same-address re-listing kept, order preserved)
- Postgres integration tests for upsert_all's backstop dedup
- ADR-0057 recording the "confirm UPRNs before finalise" decision
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 1 of confirming UPRNs before finalise. address2uprn matched each
row independently, so one UPRN could be the best match for two distinct
addresses (a coarse EPC record absorbing several real addresses, e.g.
flats in a block). Those distinct addresses were then silently merged by
the property identity insert, and collided in property_overrides.
resolve_group_ambiguity() withholds a UPRN claimed by >=2 distinct
normalised addresses within a postcode group (keeps genuine same-address
re-listings), and the handler now emits an address2uprn_status column
(matched | ambiguous_duplicate | unmatched | invalid_postcode | error).
Withheld rows drop to a null UPRN but keep their lexiscore for triage on
the (upcoming) confirmation page.
Also adds the ADR-0057 backstop dedup in property_overrides upsert_all so
the ON CONFLICT statement can never double-touch a row.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On reflection the certificate number only needs to travel through the
address2uprn pipeline (result CSV, S3 output) as an internal value —
not persisted to property. Reverts the PropertyIdentityInsert /
property_table.py / property_postgres_repository.py changes; keeps
certificate_number flowing through get_epc_data_with_postcode,
get_uprn_with_epc_df, get_uprn_from_historic_epc, and the
address2uprn_certificate_number result column.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
EpcClientService.search_by_postcode already returns the matched
certificate number alongside the UPRN, but it was dropped before
persistence. Thread it through get_epc_data_with_postcode ->
get_uprn_with_epc_df / get_uprn_from_historic_epc (using the historic
dataset's lmk_key) -> the address2uprn_certificate_number result
column -> PropertyIdentityInsert -> the property table's new
certificate_number column (assessment-model PR #362).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Review feedback (#1481): the address batcher and the Modelling Run batcher
implemented the same greedy packing; the core moves to
utilities/grouped_batching.py and both become thin wrappers.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ADR-0056 amended: override and EPC own type/form facts; a property with
neither is Unknown whatever the legacy columns say. Mirror columns removed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Remote state + MODELLING_E2E_SQS_URL env var + send policy ARN, and
fast_api_lambda deploys after modelling_e2e_lambda so the remote state
exists before it applies.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
DB access via parameterised SQL, not the infrastructure SQLModel mirrors:
importing those alongside the legacy backend.app.db.models mirrors of the
same tables double-registers them in the shared metadata and crashes the
app at import. Contained to the modelling package until the DDD cut-over.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
