Address PR review (dancafc):
- introduce UprnMatch NamedTuple (datatypes/address_match.py) for the
(uprn, address, lexiscore, certificate_number) return, replacing the bare
4-tuple in get_uprn_from_epc_df / get_uprn_from_historic_epc /
HistoricEpcResolver.resolve_uprn. Tuple-compatible, so unpacking is unchanged.
- rename get_uprn_with_epc_df -> get_uprn_from_epc_df (+ callers).
- type resolve_group_ambiguity via a GroupDecision NamedTuple and trim its
docstring.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 1 of confirming UPRNs before finalise. address2uprn matched each
row independently, so one UPRN could be the best match for two distinct
addresses (a coarse EPC record absorbing several real addresses, e.g.
flats in a block). Those distinct addresses were then silently merged by
the property identity insert, and collided in property_overrides.
resolve_group_ambiguity() withholds a UPRN claimed by >=2 distinct
normalised addresses within a postcode group (keeps genuine same-address
re-listings), and the handler now emits an address2uprn_status column
(matched | ambiguous_duplicate | unmatched | invalid_postcode | error).
Withheld rows drop to a null UPRN but keep their lexiscore for triage on
the (upcoming) confirmation page.
Also adds the ADR-0057 backstop dedup in property_overrides upsert_all so
the ON CONFLICT statement can never double-touch a row.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On reflection the certificate number only needs to travel through the
address2uprn pipeline (result CSV, S3 output) as an internal value —
not persisted to property. Reverts the PropertyIdentityInsert /
property_table.py / property_postgres_repository.py changes; keeps
certificate_number flowing through get_epc_data_with_postcode,
get_uprn_with_epc_df, get_uprn_from_historic_epc, and the
address2uprn_certificate_number result column.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
EpcClientService.search_by_postcode already returns the matched
certificate number alongside the UPRN, but it was dropped before
persistence. Thread it through get_epc_data_with_postcode ->
get_uprn_with_epc_df / get_uprn_from_historic_epc (using the historic
dataset's lmk_key) -> the address2uprn_certificate_number result
column -> PropertyIdentityInsert -> the property table's new
certificate_number column (assessment-model PR #362).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
EpcSource widens to "expired" (ADR-0054; the column is TEXT — no
migration). "predicted"/"expired" form one slot family: _slot_sources
routes every slot read and slot-clearing delete through the family, so a
re-ingestion flipping the flavour replaces the row instead of stranding
its sibling. FakeEpcRepo mirrors the family.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Review ask (dancafc): resolver/repository locals now carry explicit types
(matches: list[ScoredHistoricEpc], records: list[HistoricEpc], df:
pd.DataFrame, ...) so the flow reads without chasing callee signatures.
CLAUDE.md's Type Safety section gains the rule so future sessions enforce it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The row→domain mapper now names all 93 constructor arguments explicitly
instead of splatting a lowercased dict, takes a plain Mapping (a
DataFrame.to_dict("records") row) instead of a pandas Series, and ignores
columns the domain type doesn't know. A missing/renamed CSV column fails
loudly as a KeyError at the row. Both iterrows() call sites move to
to_dict("records") — pandas-stubs types iterrows' Series unparameterized,
which strict mode rejects. pandas-stubs + boto3-stubs[s3] make the stack
check clean: pyright strict is now 0 errors across the PR's files.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Wire flag_fuel_mismatch into the two override-resolution paths (the property
repository and the modelling-e2e handler), keeping overlays_from a pure mapping.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Compare fuel families (not exact codes), so a solid-fuel room heater refined to
smokeless/dual/biomass is consistent; only a different family (gas/electric on a
solid heater) is logged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
overlays_from now applies main_fuel after main_heating_system (stable sort), so
an explicit landlord fuel wins the natural-fuel default the heating archetype
drags, regardless of override row order. apply_simulations is last-wins.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
HistoricEpcS3Repository reached into utils/s3.py (read_csv_gz_from_s3 +
parse_s3_uri), the legacy utility that self-constructs boto3 inside free
functions. The other S3 repositories deliberately depend on the
infrastructure/s3 layer instead (UnstandardisedAddressListCsvS3Repository
injects a CsvS3Client). Bring historic EPC into line.
- Add GzipCsvS3Client(S3Client) in infrastructure/s3: read_csv_gz(key) ->
DataFrame (get_object + gzip decode).
- Inject it into HistoricEpcS3Repository; the bucket lives in the client and
the repo only builds the per-postcode key + maps rows (no S3/HTTP code).
Add with_default_s3_client(s3_root) for composition roots.
- Update main.py and the match_addresses_for_postcode seam to the factory.
- Repo tests inject a real GzipCsvS3Client over a controlled boto stub
(exact key assertions + AccessDenied); add a moto-based client test and a
factory test covering s3_root -> bucket+key.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MQE5TsSuQTeNSCSz9A9GQf
The port accepts a normalised Postcode and rejects malformed/empty ones via
Postcode.is_valid() (PostcodeNotFound) — dropping the per-lookup postcodes.io
HTTP call and the cross-module use of the private _sanitise_postcode. Mapper
helper promoted to public.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The deterministic calculator reads sap_ventilation.extract_fans_count (which
already round-trips); the top-level epc.extract_fans_count is its mirror (the
mapper sets both from one source). Reconstruct it from the same column so
EpcPropertyData round-trips complete, dropping the allow-list exception.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire community heating fuel + CHP fraction (epc_main_heating_detail),
alt-wall is_sheltered + wall insulation thermal conductivity
(epc_building_part), and pv_diverter_present / measured cylinder volume /
AP50 air permeability (epc_property) through save + _compose/_to_*. All
deep-equal round-trip; coverage guard now enforces their reconstruction.
Columns live (FE migration applied).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add EpcPhotovoltaicArrayModel (epc_photovoltaic_array child table) and wire
save / delete / read so sap_energy_source.photovoltaic_arrays survives
load->save->load in order. Threaded through both the single get() and the
bulk _for_properties() paths via _compose -> _to_energy_source. Column
names match the FE migration (feature/epc-pv-and-floor-heatloss-schema).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add is_exposed_floor / is_above_partially_heated_space to
EpcFloorDimensionModel and wire from_domain + _to_floor_dimension. Column
names match the FE schema (feature/epc-pv-and-floor-heatloss-schema).
Live DB migration is run post-merge (drizzle-kit generate picks them up).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Even after batching the data writes, the handler still wrote to the DB per
property through the orchestrator's SubTask bookkeeping: create + start +
complete each self-committed, and _cascade re-listed every sibling and re-saved
the parent on every transition — ~5 writes per property plus an O(N^2) cascade.
- TaskOrchestrator.run_subtasks: create all children in one INSERT, run each
(failures isolated per child), then persist all terminal states in one bulk
save and cascade the parent once. Children go WAITING -> terminal; the
transient IN_PROGRESS row is never written.
- SubTaskRepository.create_many / save_many (bulk INSERT / bulk fetch + update).
- _cascade short-circuits when the Task is already FAILED (terminal) — skips the
sibling roll-up entirely.
- modelling_e2e handler fans out via run_subtasks instead of per-property
create_child_subtask + run_subtask.
Per N-property batch the SubTask bookkeeping drops from ~5N writes + an O(N^2)
cascade to ~2 writes + 1 cascade.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
