Commit graph

15 commits

Author SHA1 Message Date
Khalim Conn-Kowlessar
e80be44fd1 fix(modelling_e2e): persist predicted EPC + baseline for predicted properties
A predicted Property (no lodged EPC) got a Plan but nothing else: the synthesised
EPC was never written to epc_property, and Baseline Performance was skipped — so
property 729529 (portfolio 796 / scenario 1268), predicted from its DA16 1QZ
cohort, was "missed" with no predicted-EPC row and no baseline row.

Persist the synthesised EPC in the predicted slot (uow.epc.save(..., source=
"predicted"), ADR-0031) inside the Plan UoW, then run the Baseline orchestrator
for predicted Properties too — it re-hydrates the predicted EPC and establishes
the baseline from it. The earlier "lodged only" guard is dropped: by the write
block the Property always has a persisted EPC (lodged or predicted); one that
could be neither fetched nor predicted raised earlier.

Verified against the DB by invoking the real handler for 729529: predicted
epc_property rows 0->1 and property_baseline_performance rows 0->1. Baseline on
the predicted picture builds cleanly (RHI present, reason pre_sap10). Tests
updated: prediction + broadening paths now assert the predicted-slot epc.save and
the baseline run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 17:49:08 +00:00
Khalim Conn-Kowlessar
2ee1b35dca fix(modelling_e2e): persist Baseline Performance for lodged properties
The handler wrote epc/spatial/solar/plan and marked the property modelled, but
never established its Baseline Performance — so no row was created in
property_baseline_performance for any property modelled through the Lambda
(noticed on portfolio 796 / scenario 1268 / property 727218, a lodged property).

Mirror the e2e runner: after the plan UoW commits (so the EPC is persisted for
the orchestrator to re-hydrate), run PropertyBaselineOrchestrator for lodged
properties. Predicted properties have no lodged figures and no persisted EPC, so
they are skipped — consistent with the e2e runner and the ara_first_run Baseline
stage.

Verified 727218's baseline pipeline builds end-to-end in-memory (lodged_performance
→ CalculatorRebaseliner → bill → PropertyBaselinePerformance, reason pre_sap10).
Tests: lodged path asserts the orchestrator runs once; prediction path asserts it
does not.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 17:21:03 +00:00
Khalim Conn-Kowlessar
4f4ec32e51 Merge remote-tracking branch 'origin/main' into feature/e2e-runs
# Conflicts:
#	repositories/comparable_properties/epc_comparable_properties_repository.py
#	tests/repositories/comparable_properties/test_epc_comparable_properties_repository.py
2026-06-23 17:07:27 +00:00
Khalim Conn-Kowlessar
de7fb94ff7 docs(adr): record nearby-postcode broadening (0034) + share HTTP retry primitive
Closes out the cohort-broadening work with its decision record and consolidates
the retry plumbing.

ADR-0034 documents broadening the EPC-Prediction cohort to the real unit
postcodes nearest the target (via postcodes.io) when its own postcode holds no
same-type comparable — extending ADR-0031 decision 5. Records why postcodes.io
was chosen over council[] (whole-LA, no property_type in rows), a bulk Code-Point
Open / ONSPD dataset, and the OS Places radius API, and the lazy / nearest-first
early-stop / soft-fail policy. Broadening-specific docstrings now cite 0034.

Retry consolidation: extract the EPC client's call_with_retry into a shared
infrastructure/http_retry.py keyed off a generic TransientHttpError marker, so
the mechanism (exponential backoff, Retry-After) is shared while each client
keeps its own transient policy. EpcRateLimitError now subclasses TransientHttpError
(still an EpcApiError); PostcodesIoClient routes through the same helper, raising
TransientHttpError on 429/5xx and soft-failing to the seed once exhausted (the EPC
client propagates instead). Direct tests for the shared helper; EPC + postcodes.io
suites repointed at the shared sleep.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 16:54:06 +00:00
Daniel Roth
a4aaf0e81f Merge branch 'main' into trigger-e2e-locally 2026-06-23 16:38:03 +00:00
Daniel Roth
79a76b10a9 fix db issues 2026-06-23 16:36:59 +00:00
Khalim Conn-Kowlessar
0bd2db4f03 feat(modelling_e2e): price gap measures via overlay + broaden prediction to nearby postcodes
Two reconciliations to make the modelling_e2e Lambda handler production-ready.

1. Price through the off-catalogue overlay, drop the workarounds
   The handler priced through a plain ProductPostgresRepository and excluded
   secondary_heating_removal / system_tune_up / system_tune_up_zoned to dodge
   ProductNotFound (and a poisoning pgEnum DataError). Those measures are now
   priced by catalogue_with_off_catalogue_overrides (already used by the e2e
   runner and PostgresUnitOfWork), so the exclusions are removed and ALL measure
   types are considered. This also fixes gas-boiler / single-glazed properties,
   which Dan's handler never excluded and so still crashed (the standard
   system_tune_up option is built unconditionally — the considered-measures
   exclusion never actually gated it).

2. Broaden the EPC-Prediction cohort to nearby real postcodes (ADR-0031)
   A property with no lodged EPC and no same-type comparable in its own postcode
   (e.g. the only flat among houses) used to gate out and fail the subtask. The
   gov EPC API cannot search by radius/outcode, so we resolve the real unit
   postcodes physically nearest the target via postcodes.io (keyless; already a
   trusted in-repo dependency) and walk them nearest-first until enough same-type
   comparables surface. New PostcodesIoClient (transient-failure retry with
   exponential backoff, soft-failing to the seed so broadening never breaks
   prediction) and EpcComparablePropertiesRepository.candidates_near. Wired into
   the handler and e2e runner; broadening is lazy (only on gate-out) and memoised
   per (postcode, property_type).

Validated live: property 728476 (gas boiler) prices system_tune_up at GBP295;
property 718580 (lone flat in BR6 6BS) now predicts via nearby BR6 postcodes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 16:25:18 +00:00
Jun-te Kim
00af7b5a54 data types 2026-06-23 12:42:53 +00:00
Khalim Conn-Kowlessar
c3422704f5 revert epc timeout to 10s 2026-06-23 11:19:23 +00:00
Daniel Roth
c49e1f60a2 dont consider system tune up recommendations - causes error 2026-06-23 10:36:27 +00:00
Daniel Roth
25c695186b measured excluded plus get geospatial nearby stuff working 2026-06-22 16:12:16 +00:00
Daniel Roth
7e5af6c8f4 process multiple properties in one message 2026-06-22 15:46:18 +00:00
Daniel Roth
8ee53a3a95 persist EPC data 2026-06-22 14:45:03 +00:00
Daniel Roth
dc830fea63 various fixes 2026-06-22 14:21:52 +00:00
Daniel Roth
c45a27adb0 move handler stuff to applications directory 2026-06-22 13:35:23 +00:00
Renamed from lambdas/modelling_e2e/handler.py (Browse further)