Dan's review on #1546:
- Stream the workbook to a /tmp temp file and S3Client.upload_file it, instead
of render_workbook -> bytes -> put_object. render_workbook now saves straight
to a path; the orchestrator renders to a temp file, multipart-uploads it, and
always cleans it up. Restores ADR-0065's "never an in-memory BytesIO" decision
(the OOM path at the 100k-row cap).
- Move the raw `SELECT ... FROM scenario` out of the Lambda handler into
ScenarioNamesPostgresRepository, so the handler stays composition-only and all
SQL lives in repositories/.
- current_sap_points goes through _float, matching its Optional[float] read-model
type and the sibling numeric facts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The "latest EPC per UPRN" selection ordered by e.id DESC, but row-id does not
track recency: a gov-EPC cert re-ingested after a PasHub survey lands with a
higher e.id, so the header/perf reads silently picked the stale gov cert (wrong
TFA, lodgement, property_type, current band) on 3 of portfolio 838. Order by the
effective lodgement date (registration -> completion -> inspection) DESC NULLS
LAST, e.id DESC tiebreak.
Compose the descriptive fabric from the structured SAP fields a re-survey lodges
(no gov-EPC prose): walls/roof/floor from the main epc_building_part, heating and
hot water from epc_main_heating_detail + the water-heating codes, decoded with
the SAP engine's own code space (fabric_description.py). Structured fabric wins
where present and falls back to the gov-EPC prose otherwise; windows and lighting
stay on prose (their per-element codes do not reconstruct the dwelling phrase).
Add built_form (epc_property, code-decoded) and property_age_band (building part
construction_age_band -> RdSAP date range) columns.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two fixes so a multi-scenario export over a PasHub-fetched portfolio is correct:
1. Plan selection: take the NEWEST plan per (property, scenario), not the
is_default one. is_default is one-per-property (not per-scenario), so it
cannot scope a multi-scenario export — a property's default sits under a
single scenario, leaving every other scenario with zero rows. _ROWS_SQL and
_MEASURES_SQL now DISTINCT ON (property_id) ORDER BY created_at DESC, id DESC.
2. Effective-EPC descriptive block: match the header facts + current performance
by UPRN (the latest epc_property for the source), because the PasHub re-fetch
lands records with property_id = NULL — a property_id join silently fell back
to a stale gov-EPC cert (wrong 2013-2025 lodgement dates / int property_type)
or nothing. Lodgement date coalesces registration -> completion -> inspection
(PasHub surveys carry only inspection_date). Descriptive prose elements stay
on the property_id gov-EPC fallback (a survey lodges structured fields, not
gov-EPC prose).
Result on portfolio 838 / scenarios 1303+1304: both sheets populate 205 rows;
property_type text on 204/205; lodgement dates 2026 on 200/205. Repository tests
updated (uprn fixtures, newest-plan selection + a newest-wins case); 22 export
tests pass, pyright clean.
Wires POST /v1/exports/scenario -> tasks.inputs recipe -> pinned sub_task ->
ARA_EXPORT_SQS_URL -> ara_export Lambda -> orchestrator. Route resolution and the
trigger body are covered by tests; the Lambda handler mirrors bulk_document_download.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Locks the gate edges — a lodged pitched loft below 270mm is recommended, a
loft at 270mm is left alone, and the 'Nmm+' form ('300mm+') parses to its
number and stays ineligible. These passed on arrival (they fell out of the
numeric-parse gate added in the previous commit); pinned as regression guards.
ADR-0063.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Properties 741843/741872 (uprns 100060714155/100021944594) failed
modelling_e2e subtasks 6ac7841a-9338-4efe-97e5-7e0d19b3055d and
c5450f03-5b0d-4068-a3b3-d1470bc0af57 with a bare StopIteration: their
SAP-Schema-15.0 (2011-era LIG-lodged) certs identify the main dwelling's
building part as "Main building" rather than "Main Dwelling", so
from_api_string fell to OTHER and left the property with no MAIN part —
crashing wall_recommendation.py's unguarded next(...).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Review findings on PR #1527:
- The overlay constants, ScoredOption builder, ventilation dependency and
selected_types helper come from the shared _optimiser_fixtures module
(landed on the fabric-first base) instead of local copies; the boiler
overlay is the shared BOILER_OVERLAY (SAP Table 4a code 104, a mains-gas
combi) rather than code 201, which is neither a boiler nor a heat pump.
_IWI_OVERLAY (solid-wall internal, type 3) stays local — no shared
equivalent — and the carbon stubs stay bespoke (the shared StubScorer has
no CO2 knob).
- The optimise_package_fabric_first import is lifted to module scope.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
An unrecognised schema_type used to raise ValueError and abort the whole
call; now from_api_response logs a warning and returns None so one
unmapped cert doesn't break a batch. Schema 15.0 already has a mapper
(PR #1531) so it's unaffected; only genuinely unmapped versions skip.
OpenHousing logs a job but silently drops the appointment when no bookable
resource is sent, so the survey date never lands. log_job already sends the
configured default_resource; amend_job omitted the parameter unless the
request carried an explicit surveyor. Default amend's resource to the
configured surveyor too, and fail loudly at config load on a blank
ABRI_RELAY_DEFAULT_RESOURCE so the misconfiguration can't ship an empty
resource again.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Review findings on PR #1526:
- tests/domain/modelling/_optimiser_fixtures.py is the one home for the
overlay constants, the ScoredOption builder, the additive per-kind
StubScorer and the forced ventilation dependency; test_optimiser.py and
test_optimiser_fabric_first.py had byte-identical copies of each
(and _StubScorer / _VentStubScorer fold into one parameterised stub).
- Fixture worlds are domain-plausible per team convention: the fabric-vs-
heating contrast is a £12,000 EWI against a £3,200 gas boiler rather
than a £500 heat pump undercutting a £1,000 cavity wall; heating
overlays carry real identities (SAP Table 4a code 104 for the boiler,
a PCDF index for the heat pump) instead of code 201 doubling as both;
whole-dwelling double glazing is £3,500, not £500.
- Dead knobs removed: the unused _ROOF_OVERLAY, the always-zero roof
gain, the duplicate _BOILER_OVERLAY, and the nested conditional
expressions in the interaction stubs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Re-attempted entering this SAP-15.0 cert's 4 real N/E/S/W windows (gov-API
sap_windows, 2.516 m^2 each) as separate Elmhurst Openings rows instead of
the accepted single combined South row, to test whether it closes the
residual ~2pt gap (engine 53 vs Elmhurst worksheet 51).
Reproduced live the documented elmhurst_lib.py window-grid limitation:
adding a 2nd row wipes the previously-added row's width/height to 0.00,
in both same-session and fresh-session-per-window attempts. Not fixable
via current Playwright automation. Restored the assessment to its original
single South 10.06 m^2 row (no PDFs re-downloaded, no worksheet change).
Separately confirmed our own engine already models this cert's windows
per-orientation correctly from gov-API sap_windows via
solar_gains.py's ORIENTATION_BY_SAP10_CODE cascade -- this is an Elmhurst
build-tooling ceiling, not an engine or mapper bug. sap_score=53 remains
unchanged; no calculator/mapper code was touched. Full accuracy suite
re-run clean (66 passed/67 skipped/1 xfailed + corpus test passed).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Both flagged mismatches were Elmhurst input errors (same silent-stale-value
contamination class as the earlier chimneys/wall-thickness bugs), not
parser or calculator bugs:
- cylinder_size: build_100010086084.py's water_heating() selected
DropDownListCylinderSize by raw value "2", but this DOM's option values
ARE their visible litre-band text (no "2" among them) — Playwright
silently no-ops on a non-matching value, leaving a prior cert's "Medium
(131-170L)" selection in place. Fixed to match by text ("Normal"), and
added the missing mapper dict entries (Normal/Large litre-suffixed
labels) to _ELMHURST_CYLINDER_SIZE_LABEL_TO_SAP10 (a real mapper-coverage
gap — the calculator raises UnmappedElmhurstLabel rather than silently
mis-mapping).
- boiler_flue_type: the boiler-code search dialog's combined "Balanced/Open
Flue" Table 4b category doesn't drive the separate, independently
selectable RadioButtonListFlueType field, which was left at an inherited
"Balanced" from a prior cert. Fixed space_heating() to explicitly select
"Open" to match the lodged gov-API code. This field isn't consumed by
Sap10Calculator (ML/generator-only), so it had zero effect on the SAP
score.
Re-downloaded elmhurst_summary.pdf/elmhurst_worksheet.pdf after the fixes
(Recommendations page confirmed clean). Elmhurst-PDF-inputs path moved
46->47 SAP from the cylinder-volume correction; gov-API SAP (53) and
Elmhurst's own worksheet (51) are unchanged. Full accuracy suites re-run
clean (67 passed, 67 skipped, 1 xfailed, same as before).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The Elmhurst rebuild for this cert had two carryover bugs from the shared
Playwright session: TextBoxOpenChimneys silently inherited "2" from a prior
cert's build (this cert's register lodges open_fireplaces_count=0, which
the gov-API mapper already reads correctly), and wall thickness inherited
a stale 280mm. Also fixed a real bug in the space_heating() PCDB-ref-clearing
logic: input_value() returns the string "0" when already cleared, which is
truthy in Python, causing a spurious navigate-away that broke the next
E.goto() call.
Rebuilt cleanly (chimneys=0, wall thickness=220mm per RdSAP convention for
unrecorded solid-wall thickness, since CheckBoxWallThicknessUnknown does not
persist via Playwright in this tool version). Elmhurst's own worksheet score
moves from a contaminated 46/48 to 51, closing the gap against this engine's
53-on-gov-API-inputs from ~5-7pt to ~2pt. No mapper.py change was needed --
the gov-API mapper's open_chimneys_count=0 was correct all along.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Built UPRN 100010086084 (the largest gov-API-vs-lodged outlier from the
20-cert batch) in Elmhurst's accredited RdSAP10 tool to get real ground
truth, per feedback that lodged rating isn't a valid comparison target.
Elmhurst worksheet: SAP 46-48 (orientation-dependent), vs this engine's
53 on gov-API inputs -- a real ~5-7pt residual gap. Ruled out with
direct empirical tests: total_floor_area, missing party_wall_length,
region/climate (SAP rating always uses UK-average weather per Appendix
U -- confirmed inert), boiler/secondary efficiency (exact match, 66%/
63% both sides). Confirmed real but NOT blind-fixed (need more than one
cert's evidence, and a wall-U-value patch tested the wrong direction in
isolation): a wall-thickness-unknown U-value fallback mismatch (this
engine 1.70, Elmhurst's own "unknown" resolution 1.40) and a window
U-value divergence (1.85 vs 2.52 W/m2K for the same nominal glazing
pick). Single-window orientation confirmed a ~2pt partial contributor.
Also fixes a small, separate, evidenced gap the build surfaced: the
Elmhurst summary parser didn't recognise the live tool's "Medium
(131-170L)" cylinder-size label (only the bare "Medium").
Full write-up is in the RealCertExpectation comment for this cert in
test_real_cert_sap_accuracy.py and the worklist. Residual gap is
intentionally left open for follow-up, not force-closed.
