Aligns the historical backfill with the go-forward finaliser: it now writes a
boiler_efficiency_band row for EVERY main_heating_system row — the parseable band
(A-G) where present, else an explicit Unknown — so historical and go-forward data
match. Unknown is fine on non-boilers (inert to modelling). Confirmed with Khalim.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses reviewer feedback:
- #3 (Khalim + kimjunte): the boiler_efficiency_band classifier column is now
GUARD-ONLY — LLM fallback removed. The band format is deterministic, so an LLM
guess could only fabricate a band the landlord never stated (moves SAP +
eligibility). New LoggingUnknownColumnClassifier is the non-fabricating fallback:
maps guard-misses to UNKNOWN (never stored) and logs them for review — kimjunte's
'warn on None', placed in the classifier path not the shared guard (which sees
legitimate None constantly on the modelling/backfill paths).
- #2 (Khalim): backfill --apply now reports rows ACTUALLY written (upsert
rowcount), not the candidate count, so an idempotent re-run reports 0.
- #1 (Khalim): overlay module docstring scrubbed of stale 'slot/pending' wording
to match the cert-native anchor mechanism.
kimjunte's finaliser optional-skip question (declared-vs-shared-column flow) left
for reviewer alignment, not changed. 529 tests green; pyright clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Idempotent, dry-run-by-default script (mirrors reclassify_main_heating). Parses
the SEDBUK band off each main_heating_system row's original_spreadsheet_description
with the SAME guard as the live classifier (no drift), upserting a
boiler_efficiency_band row per boiler that carries one. Pure core band_backfill_rows
unit-tested. FE-enum-gated writes (Class-A/B deferred) (ADR-0068).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reproducible RdSAP-10 build (build_100010472748.py) + downloaded Input Summary
and SAP Worksheets (accredited SAP 62 D) + the entry crib sheet, as the
ground-truth for the community-heating mapping fix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`test_pashub_sap_accuracy_wythenshawe` graded against the manifest `pre_sap`
(`hubspot_deal_data.pre_sap`) — surveyor hand-entered and frequently wrong,
which floated a misleading 79% / MAE 0.75. Grade instead against the DB
PasHub-assessment SAP (`epc_property_energy_performance.energy_rating_current`,
`ep.source='lodged'`), matched by UPRN or postcode+house-number.
`scripts/enrich_pashub_wythenshawe_oracle.py` resolves each fixture's oracle SAP
once (DB required, not run in CI) and bakes `oracle_sap` / `oracle_match` into
`manifest.json`, so the harness stays DB-free. 148/168 resolve (39 UPRN + 109
address); the 20 unmatched carry `oracle_sap` null and are excluded from the
aggregate until PasHub sources their SAP.
With the six from_site_notes fixes, the 148 matched score within-0.5 89.2%
(132/148), MAE 0.297 (was, vs pre_sap, 79.2% / 0.754). Ratchets re-baselined
0.75→0.88 floor, 0.85→0.31 ceiling. The residual tail is the integer-oracle
rounding floor (spec-correct dwellings) plus a couple of spec-literal gaps, not
engine defects.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The deal's third_party_surveyor_identifier is now required by the log
and amend flows: the trigger contract rejects messages without it
(blank treated as missing, since a blank resource makes OpenHousing
silently drop the appointment), and LogJobRequest/AmendJobRequest carry
it as a required field. The ABRI_RELAY_DEFAULT_RESOURCE fallback and
its config/terraform/workflow wiring are removed as dead code.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One-time ingest of Wythenshawe Community Housing Group Elmhurst *Summary*
PDFs: resolve each property's SharePoint folder from the WCHG xlsx, download
the summary, run the existing extractor -> mapper -> SAP10 pipeline, and
(with --commit) push the PDF to S3, record an uploaded_files row, and save
the EpcPropertyData to epc_property keyed on the national UPRN (property_id
NULL, replace-by-(uprn,source)). Dry-run by default; also emits a SAP-accuracy
CSV vs Elmhurst's reported current SAP (% within 0.5 + MAE).
Adds DomnaSites.SOCIAL_HOUSING (SOCIAL_HOUSING_SHAREPOINT_ID) for the site.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
scripts/run_pashub_modelling.py invokes the modelling_e2e handler in-process
over a portfolio using STORED EPCs (refetch_epc=False — the correct source for
PasHub cohorts, issue #1589), parametrised by env (portfolio/scenario/pids/
dry-run/batch). Handover documents the 3 re-extraction stragglers, the
rebaseliner pass-through gotcha (effective_sap_score IS the pashub rating for
unchanged lodged EPCs — never validate the calculator against it), and the 7
root-caused extractor bugs now tracked in #1590.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Foundation for PRD #1555: runs each PAS Hub site-note PDF through the
extractor -> EpcPropertyData -> Sap10Calculator and gauges the computed
SAP against pashub's own SAP-10.2 `pre_sap` (from hubspot_deal_data).
- test_pashub_sap_accuracy.py: hybrid gate. Per-fixture "must compute"
(xfail on the known in-progress mapper gaps MissingMainFuelType /
UnmappedSapCode / UnmappedPasHubLabel) + aggregate within-0.5 ratchet
floor, mirroring test_sap_accuracy_corpus.py.
- 205 image-stripped site-note PDFs + manifest.json. Images stripped so
the repo footprint stays ~52MB while the text layer the extractor reads
is byte-identical.
- build_pashub_accuracy_fixtures.py: provenance/rebuild from S3 +
hubspot_deal_data.
All 206 currently xfail on the known heating-string mapper gaps; each fix
(#1556-1568) flips its fixtures to computing and ratchets the floor.
Refs #1555#1568
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Re-attempted entering this SAP-15.0 cert's 4 real N/E/S/W windows (gov-API
sap_windows, 2.516 m^2 each) as separate Elmhurst Openings rows instead of
the accepted single combined South row, to test whether it closes the
residual ~2pt gap (engine 53 vs Elmhurst worksheet 51).
Reproduced live the documented elmhurst_lib.py window-grid limitation:
adding a 2nd row wipes the previously-added row's width/height to 0.00,
in both same-session and fresh-session-per-window attempts. Not fixable
via current Playwright automation. Restored the assessment to its original
single South 10.06 m^2 row (no PDFs re-downloaded, no worksheet change).
Separately confirmed our own engine already models this cert's windows
per-orientation correctly from gov-API sap_windows via
solar_gains.py's ORIENTATION_BY_SAP10_CODE cascade -- this is an Elmhurst
build-tooling ceiling, not an engine or mapper bug. sap_score=53 remains
unchanged; no calculator/mapper code was touched. Full accuracy suite
re-run clean (66 passed/67 skipped/1 xfailed + corpus test passed).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Both flagged mismatches were Elmhurst input errors (same silent-stale-value
contamination class as the earlier chimneys/wall-thickness bugs), not
parser or calculator bugs:
- cylinder_size: build_100010086084.py's water_heating() selected
DropDownListCylinderSize by raw value "2", but this DOM's option values
ARE their visible litre-band text (no "2" among them) — Playwright
silently no-ops on a non-matching value, leaving a prior cert's "Medium
(131-170L)" selection in place. Fixed to match by text ("Normal"), and
added the missing mapper dict entries (Normal/Large litre-suffixed
labels) to _ELMHURST_CYLINDER_SIZE_LABEL_TO_SAP10 (a real mapper-coverage
gap — the calculator raises UnmappedElmhurstLabel rather than silently
mis-mapping).
- boiler_flue_type: the boiler-code search dialog's combined "Balanced/Open
Flue" Table 4b category doesn't drive the separate, independently
selectable RadioButtonListFlueType field, which was left at an inherited
"Balanced" from a prior cert. Fixed space_heating() to explicitly select
"Open" to match the lodged gov-API code. This field isn't consumed by
Sap10Calculator (ML/generator-only), so it had zero effect on the SAP
score.
Re-downloaded elmhurst_summary.pdf/elmhurst_worksheet.pdf after the fixes
(Recommendations page confirmed clean). Elmhurst-PDF-inputs path moved
46->47 SAP from the cylinder-volume correction; gov-API SAP (53) and
Elmhurst's own worksheet (51) are unchanged. Full accuracy suites re-run
clean (67 passed, 67 skipped, 1 xfailed, same as before).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The Elmhurst rebuild for this cert had two carryover bugs from the shared
Playwright session: TextBoxOpenChimneys silently inherited "2" from a prior
cert's build (this cert's register lodges open_fireplaces_count=0, which
the gov-API mapper already reads correctly), and wall thickness inherited
a stale 280mm. Also fixed a real bug in the space_heating() PCDB-ref-clearing
logic: input_value() returns the string "0" when already cleared, which is
truthy in Python, causing a spurious navigate-away that broke the next
E.goto() call.
Rebuilt cleanly (chimneys=0, wall thickness=220mm per RdSAP convention for
unrecorded solid-wall thickness, since CheckBoxWallThicknessUnknown does not
persist via Playwright in this tool version). Elmhurst's own worksheet score
moves from a contaminated 46/48 to 51, closing the gap against this engine's
53-on-gov-API-inputs from ~5-7pt to ~2pt. No mapper.py change was needed --
the gov-API mapper's open_chimneys_count=0 was correct all along.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Built UPRN 100010086084 (the largest gov-API-vs-lodged outlier from the
20-cert batch) in Elmhurst's accredited RdSAP10 tool to get real ground
truth, per feedback that lodged rating isn't a valid comparison target.
Elmhurst worksheet: SAP 46-48 (orientation-dependent), vs this engine's
53 on gov-API inputs -- a real ~5-7pt residual gap. Ruled out with
direct empirical tests: total_floor_area, missing party_wall_length,
region/climate (SAP rating always uses UK-average weather per Appendix
U -- confirmed inert), boiler/secondary efficiency (exact match, 66%/
63% both sides). Confirmed real but NOT blind-fixed (need more than one
cert's evidence, and a wall-U-value patch tested the wrong direction in
isolation): a wall-thickness-unknown U-value fallback mismatch (this
engine 1.70, Elmhurst's own "unknown" resolution 1.40) and a window
U-value divergence (1.85 vs 2.52 W/m2K for the same nominal glazing
pick). Single-window orientation confirmed a ~2pt partial contributor.
Also fixes a small, separate, evidenced gap the build surfaced: the
Elmhurst summary parser didn't recognise the live tool's "Medium
(131-170L)" cylinder-size label (only the bare "Medium").
Full write-up is in the RealCertExpectation comment for this cert in
test_real_cert_sap_accuracy.py and the worklist. Residual gap is
intentionally left open for follow-up, not force-closed.
Built the property-753950 cert (worklist P4, PR #1503) in the live RdSAP-10
Online tool via Playwright: SAP 44 (E), vs our engine's 42 and the lodged
2014 cert's 45 — a normal RdSAP-2012->SAP-10.2 recalculation spread, not a
defect surfaced by the multiple_glazed_proportion fix.
Also surfaced (and documented as a caveat, not fixed here) a pre-existing
SAP-16.2 gap: the cert lodges party_wall_length_m=0 on both floors despite
being semi-detached, which Elmhurst's validation rejects and this PR's
engine change doesn't touch.
Address PR review (dancafc):
- introduce UprnMatch NamedTuple (datatypes/address_match.py) for the
(uprn, address, lexiscore, certificate_number) return, replacing the bare
4-tuple in get_uprn_from_epc_df / get_uprn_from_historic_epc /
HistoricEpcResolver.resolve_uprn. Tuple-compatible, so unpacking is unchanged.
- rename get_uprn_with_epc_df -> get_uprn_from_epc_df (+ callers).
- type resolve_group_ambiguity via a GroupDecision NamedTuple and trim its
docstring.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The move in 3934e2d3 left the trigger-builder test and local smoke script
importing the deleted etl.hubspot.abri_flow_triggers module; point them at
HubspotDealDiffer.check_abri_triggers_and_construct_message so they collect
and run again.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
30 pairs (28 deterministically scoreable) from the 2,000-postcode sweep,
frozen as ONE anonymised raw-payload JSON (pairs + cohorts + actuals — a
thousand per-cert files would drown the PR diff). The gate replays the
whole conditioning path offline — mapper, conditioning, selection,
synthesis, comparison — in ~9s; floors are the measured values, tighten-
only. comparable_from_payload is extracted from the corpus loader so both
fixture formats share one payload->ComparableProperty path; the builder
(build_expired_pairs_corpus.py) refreezes from the raw-JSON disk cache.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
roof_construction codes group by FORM (empirical: 1=Flat 98%, 4/5/8=
Pitched 88-99%, 3=dwelling-above 100% over 7,974 certs; 7/9=premises-
above per #1452), so the filter matches families — an exact-code filter
would wrongly drop pitched neighbours lodged as 5/8. Historic prefixes
map to the same families; roof rooms and thatch stay unconditioned.
Harness ladder replay and telemetry mirror the new filter.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
JsonCachingEpcClient caches at the raw layer (cert payloads verbatim,
search rows as JSON) so warm re-runs replay through retry+mapper+parsing
in minutes AND the cache doubles as integration-test fixture material —
the ADR-0030 corpus pattern (anonymised raw JSON through the mapper),
which pickled domain objects could never be. Wired into the pairs
harness (--cache-dir). roof_construction_code_sweep pins the RdSAP roof
code table the basement-code-6 way: single-building-part certs only,
code x description-prefix purity table. Also fixes the tfa telemetry key
rename that left the agreement row empty.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Evidence (439-pair harness, PR #1466): historic-vs-new age band agrees
52% exactly but 90% within one band (assessors re-band, skewing newer);
TFA agrees 45% within ±5% but 82% within ±20%. Equality/±5% steered the
cohort toward stale values where they engaged and relaxed everywhere
else. Band definitions are public so the harness's ladder replay shares
one source of truth.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 2000-postcode sweep died at ~884 on a strict-raising RdSAP-17.1 cert
(missing multiple_glazed_proportion) inside the pair-check, losing the
whole run's output. get_by_uprn and the cohort fetch now skip-and-log per
item; telemetry rows append as produced and carry raw values (age band,
TFA, fuel text) so band-width questions are answerable post hoc.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per pair, replay the age->fuel->TFA conditioning ladder over the plain
arm's cohort to record ENGAGED vs RELAXED per filter, and record whether
each historic stable attribute still agrees with the newly lodged cert
(the direct staleness measurement). Emitted as JSONL (--telemetry) and
aggregated into two diagnosis tables appended to the report.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Classification hit-rates for every compare_prediction component, all five
numeric residuals, and the secondary calculator-floored SAP residual
(calc(predicted) − lodged), plain vs conditioned side by side — the same
metric shape as the prediction-corpus gate (ADR-0030). Pair-check now
precedes the cohort fetch so a national postcode sweep only pays the
expensive search-by-postcode for shards that actually hold a pair.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
