Commit graph

8104 commits

Author SHA1 Message Date
Khalim Conn-Kowlessar
f1e190293b Address review: stream workbook to /tmp, repo the scenario-name SQL
Dan's review on #1546:

- Stream the workbook to a /tmp temp file and S3Client.upload_file it, instead
  of render_workbook -> bytes -> put_object. render_workbook now saves straight
  to a path; the orchestrator renders to a temp file, multipart-uploads it, and
  always cleans it up. Restores ADR-0065's "never an in-memory BytesIO" decision
  (the OOM path at the 100k-row cap).

- Move the raw `SELECT ... FROM scenario` out of the Lambda handler into
  ScenarioNamesPostgresRepository, so the handler stays composition-only and all
  SQL lives in repositories/.

- current_sap_points goes through _float, matching its Optional[float] read-model
  type and the sibling numeric facts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 19:51:51 +00:00
Khalim Conn-Kowlessar
35a0c6e972 Source export fabric from structured SAP fields; fix stale-cert selection
The "latest EPC per UPRN" selection ordered by e.id DESC, but row-id does not
track recency: a gov-EPC cert re-ingested after a PasHub survey lands with a
higher e.id, so the header/perf reads silently picked the stale gov cert (wrong
TFA, lodgement, property_type, current band) on 3 of portfolio 838. Order by the
effective lodgement date (registration -> completion -> inspection) DESC NULLS
LAST, e.id DESC tiebreak.

Compose the descriptive fabric from the structured SAP fields a re-survey lodges
(no gov-EPC prose): walls/roof/floor from the main epc_building_part, heating and
hot water from epc_main_heating_detail + the water-heating codes, decoded with
the SAP engine's own code space (fabric_description.py). Structured fabric wins
where present and falls back to the gov-EPC prose otherwise; windows and lighting
stay on prose (their per-element codes do not reconstruct the dwelling phrase).

Add built_form (epc_property, code-decoded) and property_age_band (building part
construction_age_band -> RdSAP date range) columns.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 19:38:12 +00:00
Khalim Conn-Kowlessar
a820274d4f ARA export: newest plan per (property, scenario) + uprn-matched Effective-EPC
Two fixes so a multi-scenario export over a PasHub-fetched portfolio is correct:

1. Plan selection: take the NEWEST plan per (property, scenario), not the
   is_default one. is_default is one-per-property (not per-scenario), so it
   cannot scope a multi-scenario export — a property's default sits under a
   single scenario, leaving every other scenario with zero rows. _ROWS_SQL and
   _MEASURES_SQL now DISTINCT ON (property_id) ORDER BY created_at DESC, id DESC.

2. Effective-EPC descriptive block: match the header facts + current performance
   by UPRN (the latest epc_property for the source), because the PasHub re-fetch
   lands records with property_id = NULL — a property_id join silently fell back
   to a stale gov-EPC cert (wrong 2013-2025 lodgement dates / int property_type)
   or nothing. Lodgement date coalesces registration -> completion -> inspection
   (PasHub surveys carry only inspection_date). Descriptive prose elements stay
   on the property_id gov-EPC fallback (a survey lodges structured fields, not
   gov-EPC prose).

Result on portfolio 838 / scenarios 1303+1304: both sheets populate 205 rows;
property_type text on 204/205; lodgement dates 2026 on 200/205. Repository tests
updated (uprn fixtures, newest-plan selection + a newest-wins case); 22 export
tests pass, pyright clean.
2026-07-16 01:26:32 +00:00
Khalim Conn-Kowlessar
30c7133b9a Add ara_export Lambda + SQS terraform module 🟩
Self-contained root module mirroring bulk_document_download (4GB memory per
ADR-0065, exports-bucket-only IAM). Cross-module wiring (shared ECR/state,
fast-api remote_state + ARA_EXPORT_SQS_URL env, CI image build) still needs a
terraform plan review — unverifiable in this environment.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 09:38:07 +00:00
Khalim Conn-Kowlessar
8bd9c564dc Add the scenario export route, worker handler and SQS trigger 🟩
Wires POST /v1/exports/scenario -> tasks.inputs recipe -> pinned sub_task ->
ARA_EXPORT_SQS_URL -> ara_export Lambda -> orchestrator. Route resolution and the
trigger body are covered by tests; the Lambda handler mirrors bulk_document_download.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 09:36:14 +00:00
Khalim Conn-Kowlessar
274d71599e Resolve export property ids with explicit ids overriding filters 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 09:32:39 +00:00
Khalim Conn-Kowlessar
ddc1994984 Resolve export property ids with explicit ids overriding filters 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 09:32:14 +00:00
Khalim Conn-Kowlessar
42201e42c5 Record a failure when the selection yields no export rows 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 09:29:45 +00:00
Khalim Conn-Kowlessar
4220ce4bf0 Record a failure when the selection yields no export rows 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 09:29:28 +00:00
Khalim Conn-Kowlessar
5ee8b2df50 Email the requester the export link, best-effort 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 09:28:56 +00:00
Khalim Conn-Kowlessar
e825b58c13 Email the requester the export link, best-effort 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 09:28:16 +00:00
Khalim Conn-Kowlessar
f774ca4424 Build and upload the sheet-per-scenario export workbook 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 09:27:25 +00:00
Khalim Conn-Kowlessar
ace3150363 Build and upload the sheet-per-scenario export workbook 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 09:26:59 +00:00
Khalim Conn-Kowlessar
e313f19ef2 Brand the export header band and freeze the header row 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 09:22:44 +00:00
Khalim Conn-Kowlessar
8f24187d03 Brand the export header band and freeze the header row 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 09:22:09 +00:00
Khalim Conn-Kowlessar
000ebf82be Sanitise and deduplicate scenario sheet names 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 09:20:56 +00:00
Khalim Conn-Kowlessar
a8c90b0d95 Sanitise and deduplicate scenario sheet names 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 09:20:17 +00:00
Khalim Conn-Kowlessar
4bd90bf836 Render the scenario export workbook with a sheet per scenario 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 09:19:40 +00:00
Khalim Conn-Kowlessar
62451aa3c9 Render the scenario export workbook with a sheet per scenario 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 09:18:59 +00:00
Khalim Conn-Kowlessar
183d14f5f4 Resolve Effective-EPC fields by override then lodged then predicted 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:40:53 +00:00
Khalim Conn-Kowlessar
642789db01 Resolve Effective-EPC fields by override then lodged then predicted 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:39:45 +00:00
Khalim Conn-Kowlessar
6ea3f31d82 Populate export rows with the lodged EPC's descriptive fields 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:38:22 +00:00
Khalim Conn-Kowlessar
ed690d9e36 Populate export rows with the lodged EPC's descriptive fields 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:36:43 +00:00
Khalim Conn-Kowlessar
432898444c Map the default plan's selected measures onto the export row 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:30:11 +00:00
Khalim Conn-Kowlessar
5dab7beae5 Map the default plan's selected measures onto the export row 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:29:26 +00:00
Khalim Conn-Kowlessar
14763638e2 Lock model-A scoping: exclude properties without a default plan 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:27:39 +00:00
Khalim Conn-Kowlessar
1bb1e605e2 Read a scenario's default-plan properties as export rows 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:26:51 +00:00
Khalim Conn-Kowlessar
d70c1b65cc Read a scenario's default-plan properties as export rows 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:26:17 +00:00
Khalim Conn-Kowlessar
65fb4fabad Extract per-Property row construction behind a shape helper 🟪
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:16:27 +00:00
Khalim Conn-Kowlessar
9c1e79efc6 Carry Property identity and Effective-EPC fields onto the export row 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:15:40 +00:00
Khalim Conn-Kowlessar
6fc94b1cc4 Carry Property identity and Effective-EPC fields onto the export row 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:15:18 +00:00
Khalim Conn-Kowlessar
c2aca6d2e6 Carry the Plan's post-works figures onto the export row 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:14:41 +00:00
Khalim Conn-Kowlessar
6a6ffb695c Carry the Plan's post-works figures onto the export row 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:14:13 +00:00
Khalim Conn-Kowlessar
91c88743f7 Pivot solar PV with a battery to its own column 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:13:22 +00:00
Khalim Conn-Kowlessar
f758216bf1 Pivot solar PV with a battery to its own column 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:12:51 +00:00
Khalim Conn-Kowlessar
a6eaac5fe3 Total each Property's retrofit cost from its measure costs 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:12:23 +00:00
Khalim Conn-Kowlessar
df768d882e Total each Property's retrofit cost from its measure costs 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:11:59 +00:00
Khalim Conn-Kowlessar
b52b38e6f3 Roll up each Property's SAP points and savings across its measures 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:11:36 +00:00
Khalim Conn-Kowlessar
478ee20aaa Roll up each Property's SAP points and savings across its measures 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:11:01 +00:00
Khalim Conn-Kowlessar
23ff1e73a8 Guarantee every measure column on every scenario sheet 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:10:16 +00:00
Khalim Conn-Kowlessar
04be6b57a2 Guarantee every measure column on every scenario sheet 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:09:40 +00:00
Khalim Conn-Kowlessar
c4e1cc4229 Pivot a Property's measure cost onto its scenario-sheet column 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:09:00 +00:00
Khalim Conn-Kowlessar
1a0e87e593 Pivot a Property's measure cost onto its scenario-sheet column 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:08:26 +00:00
Khalim Conn-Kowlessar
01e8b8c289 Record the Scenario Export design as ADR-0065 with glossary terms 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 23:08:26 +00:00
KhalimCK
e1c99edc3f
Merge pull request #1542 from Hestia-Homes/fix/loft-topup-below-building-regs
Loft insulation eligible below the 270mm building-regs depth (ADR-0063)
2026-07-10 17:11:44 +01:00
Khalim Conn-Kowlessar
3bbe293f68 Gate the loft top-up strictly below the 270mm building-regs depth 🟩
Locks the gate edges — a lodged pitched loft below 270mm is recommended, a
loft at 270mm is left alone, and the 'Nmm+' form ('300mm+') parses to its
number and stays ineligible. These passed on arrival (they fell out of the
numeric-parse gate added in the previous commit); pinned as regression guards.
ADR-0063.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 16:07:48 +00:00
Khalim Conn-Kowlessar
95a0da4828 Recommend loft insulation for a loft below the building-regs depth 🟩
A lodged numeric loft depth below the 270mm building-regs compliance gate is
now eligible (topped up to the 300mm install depth), regardless of whether the
roof type is lodged, since a measured depth is a positive statement of a real
loft. Sentinels keep their ADR-0047 resolution. Fixes property 724702 (50mm
loft) which previously returned no recommendation. ADR-0063.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 16:06:44 +00:00
Khalim Conn-Kowlessar
2fe5b79dc4 Recommend loft insulation for a loft below the building-regs depth 🟥
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 16:03:39 +00:00
Khalim Conn-Kowlessar
ce0940c335 Record below-regs loft eligibility as ADR-0063
The roof generator offered loft insulation only when genuinely uninsulated
(ADR-0021), so a loft with a shallow lodged depth (e.g. 724702's 50mm) got
nothing and never entered the optimiser pool. Grilled with Khalim: the loft
branch becomes eligible when the lodged depth is a known numeric value below
the 270mm building-regs compliance gate, topped up to the 300mm install depth.
Numeric-lodged-depths only; sentinels keep their ADR-0047 resolution. Loft
branch only. CONTEXT 'Roof Insulation Eligibility' updated to match.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 15:59:51 +00:00
Jun-te Kim
59954db488
Merge pull request #1540 from Hestia-Homes/fix/sap15-main-building-identifier
Map SAP-Schema-15.0 "Main building" identifier to MAIN
2026-07-10 16:09:59 +01:00