Commit graph

27 commits

Author SHA1 Message Date
Jun-te Kim
7987c94876 chore(hyde): harden Elmhurst automation for per-cert builds (window cleanup + reset)
Support for the per-cert Elmhurst worksheet loop:
- goto()/save_close(): fall back networkidle -> domcontentloaded on
  timeout so an incomplete-validation state (half-set RR, 0-area window)
  no longer hangs navigation.
- add_combined_window(): select glazing first (AutoPostBack), then JS
  input+change+blur the width/height with settle waits so the Area cell
  computes before Add (a plain .fill() raced the postback -> 0.00 row).
- set_single_window(): leave the grid holding exactly one row, deleting
  rows carried over from a prior build of the SHARED assessment (the
  window grid is the only accumulating page).
- reset_transient_state(): return the shared assessment to neutral after
  a run (clear RR age band, secondary = No, meter = Single).

KNOWN RESIDUAL: the window grid still intermittently persists a 0.00 area
through Save & Close (Elmhurst's "each Add wipes the previous row" +
last-row-undeletable behaviour), and the secondary-heating dialog's modal
background can intercept the cascade click. Reliable autonomous grinding of
per-cert builds needs these two hardened further; the loop currently
completes clean single-main gas certs (see the 47084930 flat-roof slice)
but stalls on secondary-heating + stale-window combinations. build_
100021924710.py is the WIP template for that harder shape.

No engine change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 23:54:35 +00:00
Jun-te Kim
8e33a8de53 test(accuracy): ratchet 77.3% -> 77.7% via flat-roof insulation-thickness fix (Elmhurst-validated)
A FLAT roof lodges its insulation thickness in the DEDICATED gov-EPC API
`flat_roof_insulation_thickness` field (e.g. "75mm"), leaving
`roof_insulation_thickness` None (that field is for pitched-loft joists).
`heat_transmission_from_cert` read only the latter, so a measured
flat-roof thickness was ignored and the roof billed at the uninsulated
age-band flat default (age E = 1.5) instead of its Table-16 insulated U.

Fixed by preferring `flat_roof_insulation_thickness` when the part is a
flat roof — the exact mirror of the existing rafter-thickness branch.
An "AB"/"NI" (as-built/unknown) value parses to None and keeps the
age-band default, unchanged; only measured thicknesses move.

PER-CERT ELMHURST VALIDATION (cert 47084930, top-floor flat, flat roof
75 mm): built on the lodged inputs in accredited Elmhurst RdSAP10-Online
(evidence saved: elmhurst_summary.pdf / elmhurst_worksheet.pdf). The
worksheet bills "insulated flat roof" at U 0.5 (floor 0.70 + wall 0.25
also matching the engine). The fix takes the engine roof 96.4 -> 32.1 W/K
(= 64.26 x 0.5, Elmhurst-exact), PE +47 -> +0.2, SAP 69.51 -> 74.34 =
lodged 74.

Gauge: within 77.3% -> 77.7%, SAP MAE 0.648 -> 0.641, CO2 0.074 -> 0.072,
PE 3.1 -> 2.97. Unit-pinned in test_heat_transmission
(flat_roof_insulation_thickness -> U 0.5); RealCertExpectation 47084930
= 74 (Elmhurst-validated). Also adds the build script build_47084930.py.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 23:27:57 +00:00
Jun-te Kim
8e487a9bae docs(elmhurst): validate slice-5 wall fix against accredited Elmhurst worksheet (uprn 200004296092)
Built the cert in accredited Elmhurst RdSAP10-Online on the lodged register
inputs; downloaded Input Summary + SAP Worksheets (committed alongside).

Confirmed:
- Slice 5 DIRECTION validated: the externally-insulated solid-brick wall is
  billed insulated by both tools (Elmhurst U 0.48, engine U 0.29), far below
  the uninsulated 1.40 the pre-fix §5.8 gate produced.
- ROOF matches EXACTLY (U 0.30, 9.35 W/K).
- Engine 71.74 = lodged 71; Elmhurst current SAP 66 (its "unknown external
  insulation" default is more conservative + this build used a generic combi
  and no keyed wall thickness, so 66 is not a clean engine-comparison).

Follow-up lead (documented in ELMHURST_FINDINGS.md): the engine feeds the
UNMEASURED lodged wall_thickness (360 mm, measured=N) into the §5.6 formula,
whereas RdSAP §5.3 uses measured thickness / Table-3 default otherwise. The
party-wall "difference" is NOT a bug (engine 0.25 is spec-correct, Table 15
row 4). Also adds the build script scripts/hyde/build_200004296092.py.

No engine change in this commit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 20:49:58 +00:00
Khalim Conn-Kowlessar
3f69c4be02 docs: full-SAP baseline-downgrade follow-ups + Case A population sweep tool
Captures the calc-facing ventilation-type gap (older RdSAP mappers drop
mechanical_ventilation_kind), the FE-side Main Fuel display, the sweep survivor
clusters, and the predicted-property display path — all separate from ADR-0037.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 21:45:44 +00:00
KhalimCK
f7f74ea72b
Merge pull request #1288 from Hestia-Homes/feature/per-cert-mapper-validation
Feature/per cert mapper validation
2026-06-24 12:04:00 +01:00
Khalim Conn-Kowlessar
aad4050915 fix(hyde): harden Elmhurst browser automation for the dev container
In-container Playwright runs were flaky: the chromium renderer crashed
mid-build ("Target crashed") on the 64M /dev/shm, and login intermittently
hung. Added `--disable-dev-shm-usage` + `--no-sandbox` launch args, a
4-attempt login retry loop (domcontentloaded + explicit selector wait),
and an `ELM_GUID` env override so a per-UPRN assessment can be targeted
without editing the module. Tooling only — no calculator impact.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 19:46:41 +00:00
Jun-te Kim
0401797524 sap fixes 2026-06-23 16:48:04 +00:00
Jun-te Kim
8f0432721c test(accuracy): pin RdSAP-20.0.0 PV semi uprn_22086693 (PV-list fix corpus)
Corpus validation of the modelling_e2e photovoltaic_supply-as-list fix. Cert
6102-6227-8000-0083-2292 (RdSAP-20.0.0 semi, gas combi + 2× 1.14 kW PV arrays)
crashed from_rdsap_schema_20_0_0 on the measured-array list; the fix routes it
through the dict-tolerant _map_schema_21_pv. PV correctly credited: engine 61
(no PV) → 66 (+5). Built in Elmhurst (evidence: epc.json + summary + worksheet,
fabric+heating; the PV "New Technologies" Panel-details grid deferred): worksheet
55 = engine-on-Elmhurst-inputs 55 exactly → calculator faithful. The +6 engine-vs-
Elmhurst base-dwelling residual is the documented RdSAP-default gap (band-C cavity-
uninsulated suspended-floor semi). Pinned engine 66.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 16:13:34 +00:00
Jun-te Kim
75341da42d 32 2026-06-23 15:38:38 +00:00
Jun-te Kim
17b1d63f0e test(accuracy): pin SAP-16.0 storage-flat uprn_10070004512 (built_form fix corpus)
Corpus validation of the modelling_e2e built_form fix. Cert 8742-6624-9300-2780-4926
(SAP-Schema-16.0, ground-floor electric-storage-heater flat) omits built_form; the
mapper now derives it from dwelling_type. built_form is ML-only so the fix is
SAP-neutral: engine 66 = lodged 66 exactly. Built in Elmhurst (evidence: epc.json +
summary + worksheet): worksheet 54, engine-on-Elmhurst-inputs 53 ≈ 54 → calculator
faithful. The +12 engine-vs-Elmhurst is a build/input gap (cert size-1 small cylinder
unrepresentable in Elmhurst's Normal/110L-minimum entry → higher HW + reduced-field
16.0 defaults). Pinned engine 66.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 15:35:06 +00:00
Jun-te Kim
00af7b5a54 data types 2026-06-23 12:42:53 +00:00
Jun-te Kim
210ca6397f updated sap scaema to take in inputs 2026-06-23 10:22:40 +00:00
Jun-te Kim
1b53c57a07 re run ddd tests 2026-06-22 08:52:45 +00:00
Jun-te Kim
a3e2566378 landlord override data added 2026-06-20 12:57:54 +00:00
Jun-te Kim
abd4bbc2d0 Merge branch 'feautre/hyde_upload_and_extend_landlord_overrides' into feature/hyde_make_it_more_accurate_with_tests 2026-06-20 07:26:35 +00:00
Jun-te Kim
3044c70202 sap score and elmhirst mapper optimsaiation 2026-06-20 07:25:42 +00:00
Jun-te Kim
6c0545a8ee Add resume prompt to finish portfolio-796 run after Khalim review 📝
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 19:58:53 +00:00
Jun-te Kim
36b8b7a939 Add Hyde portfolio-796 unknowns review doc for Khalim sign-off 📝
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 19:57:34 +00:00
Jun-te Kim
c1ab179d51 Add Hyde property_overrides build script + creds-free smoke test 🟩
Two-pass org_ref-matched builder for property_overrides (classify via ChatGPT
into the landlord ledger, validate+apply user edits, write idempotently);
ephemeral-Postgres smoke proves the one-property chain without creds.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 19:02:42 +00:00
Jun-te Kim
e03fd27357 Type-clean the override_component consistency guard 🟪
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 18:19:07 +00:00
Jun-te Kim
37b0a38425 add more test cases 2026-06-19 10:59:51 +00:00
Jun-te Kim
e6a829aaea more examples 2026-06-19 09:51:49 +00:00
Jun-te Kim
5669676917 Annotate full worklist with autonomous-run triage results
Every worklist UPRN now carries schema · engine SAP / lodged · flag. Tally:
64 healthy, 19 MVHR-not-credited (🚩 flag B), 6 heat-pump fuel-39 (🚩 flag A),
4 sparse/NOT MAPPABLE (), 3 Elmhurst-pinned. MVHR is the largest accuracy gap.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 20:36:28 +00:00
Jun-te Kim
b68710c98f Flag heat-pump fuel-39 + MVHR-heat-recovery gaps for manual review
Autonomous-run triage of the moderate eng-vs-lodged gaps resolves them into two
patterns, both flagged for owner review (not auto-fixable):
- Heat-pump fuel code 39 mis-priced as gas (over-rates; both gap directions).
- MVHR heat recovery modelled as plain extract loss → systematic UNDER-rating
  (~8-12 SAP) on every full-SAP cert carrying a mechanical_vent_system_index_number.
  New memory mvhr-heat-recovery-not-modelled; needs the Appendix Q / PCDB MVHR
  efficiency model.
findings doc updated with the classification.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 20:23:59 +00:00
Jun-te Kim
05e3a25d25 Sanitise millimetre-lodged full-SAP opening dimensions to metres
Some SAP-Schema-17.x/18.0.0 certs lodge sap_openings width/height in MILLIMETRES
mixed with metre rows in the same array (e.g. a 2025x2100 mm window beside a
3.06x1 m one). The 17.1 mapper read them all as metres → a 4.25M m2 window →
HTC in the millions → SAP clamped to 1.

Fix (TDD, datatypes/epc/domain/mapper.py): _sanitise_opening_dimension_m treats
any dimension > 50 m as mm and divides by 1000; _sap_opening_area_m2 applies it
to areas. Wired into the window, roof-window, and door-area-weighting paths.
The 3 broken certs (uprn_10093117227 / 10090317693 / 10091636031) now score
90 / 81 / 79 instead of 1.

3 RED->GREEN slices + refactor; new test class
TestFromSapSchema17_1OpeningUnitSanitisation + sap_17_1_mm_openings.json fixture;
0 new pyright errors; no regressions.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 20:16:27 +00:00
Jun-te Kim
d41798f1c1 Add SAP-18.0.0 + SAP-16.0 schema coverage; autonomous-run triage findings
Schema coverage (datatypes/epc/domain/mapper.py):
- SAP-Schema-18.0.0: full-SAP shape ≡ 17.1 → from_sap_schema_17_1, no normalisation.
- SAP-Schema-16.0: same reduced-field 16.x path; default the omitted `tenure`
  field in _normalize_sap_schema_16_x (metadata; SAP cascade never reads it).
  Genuinely sparse 16.x certs (missing core fabric fields) still fail loud.
- Regression tests + sap_18_0_0.json / sap_16_0.json fixtures; 0 new pyright errors.

Autonomous triage of the worklist (scripts/hyde/autonomous_run_findings.md):
- Found + diagnosed 2 bugs (flagged, NOT fixed): (1) MAPPER — full-SAP openings
  lodged in mm read as m → multi-million-m2 windows → SAP clamps to 1 (uprn_
  10093117227 / 10090317693 / 10091636031); (2) CALCULATOR — database heat-pump
  fuel code 39 mis-priced as gas, over-rates ~14 (uprn_10093114053).
- Most certs map within +/-4 of lodged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 20:04:43 +00:00
Jun-te Kim
2f0eb49eee Checkpoint: UPRN 10093116543 Elmhurst build + devcontainer VNC/Playwright + perms
- Add SAP-accuracy sample for uprn_10093116543 (epc.json, elmhurst_inputs.md,
  summary/worksheet PDFs)
- Persist hyde viewer stack (xvfb/fluxbox/x11vnc/novnc/websockify) and Playwright
  chromium in the backend devcontainer; forward noVNC 6080
- Broaden .claude/settings.local.json allowlist (display/python/grep/tail)
- In-progress campaign mapper/cert_to_inputs work carried from prior cert

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 15:21:56 +00:00