Commit graph

49 commits

Author SHA1 Message Date
Jun-te Kim
c238a0e6a0
Merge pull request #1449 from Hestia-Homes/feat/sap-corpus-campaign
fix(rdsap): RdSAP 10 Table 13 200/280mm solid-brick wall boundary
2026-07-06 12:25:17 +01:00
Jun-te Kim
5cae4a0ce5 fix(rdsap): RdSAP 10 Table 13 200/280mm solid-brick wall boundary
_u_brick_thin_wall_age_a_to_e's `<= 280` check put an uninsulated solid-
brick wall of exactly 280mm in the "200 to 280mm -> 1.7" row; RdSAP 10
§5.7 Table 13 (spec PDF p.41) shares 280 as an unlabelled edge between
that row and "280 to 420mm -> 1.4", so the table text alone doesn't say
which row owns it.

Found by building cert 100031768368 (280mm exactly, band C) in Elmhurst
and comparing every worksheet line to the calculator: volume/ACH/floor/
doors matched exactly, but the wall didn't (1.70 vs Elmhurst's 1.40)
despite an identical input crosswalk. Initially reverted a first attempt
at this fix when it appeared to regress an already-pinned cert
(217091901, band A, also nominally 280mm, previously "confirmed" at
U=1.70) — but that cert's build script never actually set a wall
thickness field, so its shared Elmhurst assessment had silently
inherited a stale 260mm from an earlier build. Fixed that build script
and rebuilt with the correct 280mm entry: Elmhurst's worksheet now also
gives U=1.40, matching 100031768368 and confirming the fix rather than
contradicting it.

Both certs move to (near-)exact lodged matches: 100031768368 59.12 ->
61.21 (lodged 61, within 0.5); 217091901 60.82 -> 61.59 (lodged 62,
exact). Corpus gauge 77.8% -> 78.6% within-0.5, MAE 0.636 -> 0.627 (14
corpus certs lodge exactly 280mm solid brick). TDD'd, 46 pins + full
suite green (2310 passed).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 12:25:07 +00:00
Khalim Conn-Kowlessar
325c0ea257 Merge remote-tracking branch 'origin/main' into fix/full-sap-export-capable-mapping 2026-07-03 12:11:25 +00:00
Khalim Conn-Kowlessar
6656d8dec2 A full-SAP export-capable PV cert reproduces its lodged SAP 10.2 score 🟥
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 12:05:04 +00:00
Jun-te Kim
6c1828e8bf chore(hyde): fix 100061275133 Elmhurst build — 0 fans + PCDB exact boiler
ventilation(): the cert lodges no extract-fan count and RdSAP 10 Table 5's
age-band B default is 0, but the build entered 1 — inflating Elmhurst's own
(8)/(21) infiltration lines relative to both the cert and our engine.

space_heating(): switched from the generic "Combi condens" SAP-Table dialog
code (84% efficiency, no PCDB record) to elmhurst_lib.set_pcdb_boiler(18908)
via clear_main_heating_code — resolves the exact lodged product (Worcester
Greenstar 4000, 88.70%) and its correct keep-hot classification (None),
matching the SAP engine's own PCDB lookup and removing the "Standard Combi"
keep-hot artifact the reused generic code carried.

Also corrected the file's stale docstring (described a different property
entirely — mid-terrace/250mm/DUAL meter/11 windows/lodged 63 — left over
from a copy-paste).

Verified against the refreshed worksheet: (21) shelter-adjusted ACH 0.6938
now EXACT vs our engine's ventilation_from_cert; (61) combi loss monthly
kWh matches ours to ~0.01 kWh (was carrying a ~360 kWh/yr keep-hot artifact).
Both open items from the PR #1439 handoff are resolved.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 09:20:12 +00:00
Jun-te Kim
b9d36f72cb chore(hyde): refresh 100061275133 Elmhurst PDFs after draught-proofing rebuild
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 07:45:58 +00:00
Jun-te Kim
61934af97b chore(hyde): fix 100061275133 Elmhurst build faithfulness (lighting 5 LED, wall thickness 400)
Input reconciliation (Elmhurst summary vs EpcPropertyData vs raw cert) exposed
two Playwright-entry errors: lighting entered as 6 incandescent (cert has 5 LED,
100% low-energy) and wall thickness left at Elmhurst's 260mm default (cert lodges
400mm). Fixing both moved Elmhurst 67->69. Confirmed the MAPPER is faithful
(every fabric/heating/lighting field matches the raw cert) and the engine uses
the correct PCDB 18908 efficiency (88.7%). The remaining Elmhurst gap to lodged
76 is the un-enterable PCDB boiler (automation falls back to generic BGW 84% +
keep-hot); the engine (77) is faithful. Not a mapping error.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 06:13:09 +00:00
Jun-te Kim
6c47951d60 test(accuracy): pin code-195 electric storage-boiler certs as spec-faithful (§12 Rule 2)
Electric-tariff cluster investigation. The 112-cert electric-main cohort is
mixed-sign within every category/meter - no systematic tariff bias. Root cause
of the biggest non-xfail divergences (10012334488 +13.2, 10091578598 +7.81):
SAP code 195 = 'Electric water storage boiler' (Table 4a p.170), which SAP 10.2
§12 Rule 2 correctly bills mostly at the 7-hour off-peak rate. Both are pure
cost gaps (PE/CO2 match lodged -> demand right); the lodged software over-billed
the storage boilers at peak rate. The engine is SPEC-FAITHFUL - matching lodged
would tune against §12/Table-4a. Pinned to the spec-faithful engine values.

Adds scripts/corpus_1000/elec_cohort.py (electric-main cohort analyzer).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 04:55:42 +00:00
Jun-te Kim
bf53d29690 test(accuracy): pin 100051051866 + 10070056075 (clean single-main loop complete)
Final two clean single-main gas certs. Both small cost-side residuals (+1.04 /
-1.01 SAP) on cavity-insulated fabric already Elmhurst-validated on sibling
certs, with properly-insulated cylinders (insulation type 1, 50/80mm - not the
type-0/NULL zero-storage-loss bug). No fabric anomaly; the register data fully
explains the engine output. Pinned to observed engine 71 / 79.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 04:30:22 +00:00
Jun-te Kim
1032a4f128 test(accuracy): validate + pin 100061275133 (slice-5 external-insulation U confirmed)
2-storey semi, solid brick 400mm with EXTERNAL insulation, 300mm loft,
suspended uninsulated ground floor, gas combi, double glazing (glazed_type 13).
Built in accredited Elmhurst RdSAP10: roof (U 0.14, 5.459 W/K) and windows
(15.32 x eff U 1.3258 = 20.311 W/K) match the engine EXACTLY; the
externally-insulated wall is engine U 0.29 vs Elmhurst 0.30, validating the
slice-5 external-insulation default (100mm added over solid brick); suspended
ground floor engine 0.68 vs Elmhurst 0.70. Elmhurst's overall SAP 67 is not
comparable (reused-assessment Standard-Combi keep-hot loss, a build artifact).
Engine 77 vs lodged 76 (within 1.24). Pinned to the observed 77.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 04:28:25 +00:00
Jun-te Kim
17ced10e94 test(accuracy): validate + pin 217091901 (single-glazing confirmed vs Elmhurst)
Top-floor flat, solid brick uninsulated, uninsulated flat roof, gas combi,
SINGLE glazing (glazed_type 5). Built in accredited Elmhurst RdSAP10 and every
heat-loss element matches the engine EXACTLY: walls 53.18, flat roof 145.57,
floor 0, party 0, and — with the faithful single glazing — windows 34.7114 W/K
(8.62 m2 x eff U 4.0268). Confirms the mapper's glazed_type 5 -> single (raw U
4.8) is spec-correct per epc_codes.csv, not the double it superficially
resembles. Engine 61 vs lodged 62; the -1.18 is a cost-side residual, not a
fabric bug. Pinned to the observed engine 61.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 04:16:47 +00:00
Jun-te Kim
6f2b73e613 chore(hyde): fix window-grid automation (keep good row, delete stale below) + pin 100050881708
SOLVED the window-grid blocker (per Junte's "keep one and edit, don't
delete all"): set_single_window now ADDS the fresh window (it inserts at
DOM index 0), then deletes the STALE rows BELOW it by index (high->low,
never index 0) via the per-row Delete image button + JS-clicked Yes modal.
The freshly-added row survives as the single window; no 0.00 junk strands
to block the Recommendations gate. Verified end-to-end: cert 100050881708
built, worksheet downloaded, walls (0.70) + roof (0.19) match the engine
EXACTLY. Combined with the earlier fill+Tab (area postback) + frame/gap
(validation) fixes, the per-cert loop now runs autonomously on clean
single-main certs.

Pin: 100050881708 (semi, cavity filled, pitched 225mm, suspended ground
floor, combi) engine 72 = lodged 70 +2; walls/roof accredited-Elmhurst-
matched. NB the build entered the suspended ground floor as an upper
"above unheated space" floor, so Elmhurst's worksheet SAP (61) is not a
clean comparison — the engine uses the correct §3.12 suspended-ground U
(0.74); the pin rests on the walls+roof match + lodged.

No engine change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 03:55:48 +00:00
Jun-te Kim
8e33a8de53 test(accuracy): ratchet 77.3% -> 77.7% via flat-roof insulation-thickness fix (Elmhurst-validated)
A FLAT roof lodges its insulation thickness in the DEDICATED gov-EPC API
`flat_roof_insulation_thickness` field (e.g. "75mm"), leaving
`roof_insulation_thickness` None (that field is for pitched-loft joists).
`heat_transmission_from_cert` read only the latter, so a measured
flat-roof thickness was ignored and the roof billed at the uninsulated
age-band flat default (age E = 1.5) instead of its Table-16 insulated U.

Fixed by preferring `flat_roof_insulation_thickness` when the part is a
flat roof — the exact mirror of the existing rafter-thickness branch.
An "AB"/"NI" (as-built/unknown) value parses to None and keeps the
age-band default, unchanged; only measured thicknesses move.

PER-CERT ELMHURST VALIDATION (cert 47084930, top-floor flat, flat roof
75 mm): built on the lodged inputs in accredited Elmhurst RdSAP10-Online
(evidence saved: elmhurst_summary.pdf / elmhurst_worksheet.pdf). The
worksheet bills "insulated flat roof" at U 0.5 (floor 0.70 + wall 0.25
also matching the engine). The fix takes the engine roof 96.4 -> 32.1 W/K
(= 64.26 x 0.5, Elmhurst-exact), PE +47 -> +0.2, SAP 69.51 -> 74.34 =
lodged 74.

Gauge: within 77.3% -> 77.7%, SAP MAE 0.648 -> 0.641, CO2 0.074 -> 0.072,
PE 3.1 -> 2.97. Unit-pinned in test_heat_transmission
(flat_roof_insulation_thickness -> U 0.5); RealCertExpectation 47084930
= 74 (Elmhurst-validated). Also adds the build script build_47084930.py.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 23:27:57 +00:00
Jun-te Kim
11a637c6bd test(accuracy): ratchet 77.0% -> 77.3% via unsized-cylinder storage-loss fix
A hot-water cylinder lodging gov-API `cylinder_size=0` ("size not
determined", with has_hot_water_cylinder=true) resolved to a None volume
in `_cylinder_storage_loss_override` (which reads `_cylinder_volume_l_
from_code`, returning None for code 0). The subsequent `if volume_l is
None: return None` then DROPPED the cylinder's Table-2 storage loss
entirely — under-costing the DHW and over-rating the dwelling. Meanwhile
the Table-13 high-rate-fraction path already used `_hot_water_cylinder_
volume_l` (which defaults size 0 to 110 L), so the two DHW paths
disagreed on the same cylinder.

RdSAP 10 §10.5 Table 28 ("if the actual size is not determined, the size
is taken as according to Table 28") makes an unsized present cylinder the
110 L "Normal" baseline, which STILL incurs the storage loss. Fixed by
defaulting the storage-loss volume to 110 L for the explicit size-0 case.

Gated on the EXPLICIT 0 (not None): a full-SAP cert whose RdSAP
cylinder_size is simply unlodged (None, e.g. pinned 10091568921) keeps
its own cylinder handling rather than a forced 110 L RdSAP default.

7 corpus certs lodge cylinder_size=0 + has_cylinder=true. Gauge: within
77.0% -> 77.3%, SAP MAE ~flat (0.648), CO2 0.074 -> 0.073, PE 3.2 -> 3.05
(the previously-dropped storage loss now correctly counted in the demand
cascade). Unit-pinned in test_cert_to_inputs (storage loss non-None for
size 0); RealCertExpectation 200004017091 = 71.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 23:02:27 +00:00
Jun-te Kim
8e487a9bae docs(elmhurst): validate slice-5 wall fix against accredited Elmhurst worksheet (uprn 200004296092)
Built the cert in accredited Elmhurst RdSAP10-Online on the lodged register
inputs; downloaded Input Summary + SAP Worksheets (committed alongside).

Confirmed:
- Slice 5 DIRECTION validated: the externally-insulated solid-brick wall is
  billed insulated by both tools (Elmhurst U 0.48, engine U 0.29), far below
  the uninsulated 1.40 the pre-fix §5.8 gate produced.
- ROOF matches EXACTLY (U 0.30, 9.35 W/K).
- Engine 71.74 = lodged 71; Elmhurst current SAP 66 (its "unknown external
  insulation" default is more conservative + this build used a generic combi
  and no keyed wall thickness, so 66 is not a clean engine-comparison).

Follow-up lead (documented in ELMHURST_FINDINGS.md): the engine feeds the
UNMEASURED lodged wall_thickness (360 mm, measured=N) into the §5.6 formula,
whereas RdSAP §5.3 uses measured thickness / Table-3 default otherwise. The
party-wall "difference" is NOT a bug (engine 0.25 is spec-correct, Table 15
row 4). Also adds the build script scripts/hyde/build_200004296092.py.

No engine change in this commit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 20:49:58 +00:00
Jun-te Kim
0e784cbb1c test(accuracy): ratchet SAP MAE 0.658 -> 0.647 via wall added-insulation unknown-thickness default
A SOLID-BRICK or STONE wall lodging External/Internal insulation
(wall_insulation_type 1/3) but no measured insulation thickness (gov-API
"NI" -> None) was billed at its UNINSULATED §5.6/Table-6 U-value: the §5.8
added-insulation R-value adjustment gates on `insulation_thickness_mm > 0`,
so an unknown thickness fell straight through to the raw solid-wall U
(~1.4-1.7). This over-counted wall heat loss and under-rated the dwelling.

RdSAP 10 §5.4 (PDF p.32): "the U-value with additional insulation is based
on the thickness of insulation of mineral wool type (assume 100 mm if
thickness is unknown)". Apply that 100 mm default so the §5.8 Table-14
R-value fires. Scoped to solid brick / stone (the only constructions with
a §5.8 R-value path in u_wall) so cavity (composite path) and timber /
system-built walls are untouched.

Localised by deep-diving corpus cert 200004296092 (end-terrace, solid
brick 360 mm "with external insulation", NI): walls 101.7 W/K (U~1.40,
HLP 3.46 = absurd for an insulated wall) -> ~0.29; PE +59.6 -> ~0; engine
64.35 (Δ -6.65) -> 71.74 = lodged 71. ~8 solid-brick/stone corpus certs
carry this NI-thickness-with-insulation shape.

Gauge: within-0.5 held at 77.0%, SAP MAE 0.658 -> 0.647, CO2 0.074 ->
0.073, PE 3.2 -> 3.1 (floor ratcheted). Unit-pinned in test_rdsap_uvalues
(brick 360 mm external NI -> 0.29) + RealCertExpectation 200004296092 = 72.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 20:24:56 +00:00
Jun-te Kim
cb089a630c test(accuracy): ratchet SAP MAE 0.668 -> 0.658 via instantaneous-electric-DHW off-peak fix
An instantaneous point-of-use electric water heater (WHC 907/909, no
cylinder) on an off-peak tariff was billed 100% at the off-peak LOW rate
by the generic electric-off-peak else-branch. But SAP 10.2 §12 (PDF line
2680) computes the off-peak on-peak HW proportion via Table 13, "dependent
on the total floor area and the CYLINDER size" — it presumes a stored-water
cylinder charged overnight. An instantaneous heater has NO cylinder, heats
on demand, and cannot shift to the off-peak window, so 100% of its
consumption bills at the HIGH rate (Table 12a WH high-rate fraction 1.0).

Both the SAP-cost rate (`_hot_water_fuel_cost_gbp_per_kwh`) and the
ADR-0014 / CO2-PE fraction (`_hot_water_high_rate_fraction`) fixed
consistently (low-rate scalar -> 7-hour high rate; fraction 0.0 -> 1.0).

Localised by deep-diving corpus cert 74061136 (HHR-storage mid-floor flat,
WHC 909): PE matched lodged (+1.6, roof+floor zero-loss) while SAP over-rated
+7.72 — the cost-only signature. Its DHW was ours £59 vs lodged £344 (5.8x);
the tariff was the whole gap. Fix: +7.72 -> -1.25 (residual is separate small
fabric). 7 corpus certs carry electric-instantaneous DHW on off-peak; the 3
outside 0.5 all move sharply inward (MAE the win, not within-0.5 crossings).

RdSAP 10 §12 tariff routing confirmed spec-correct (Unknown meter + storage
409 -> off-peak 7-hour, Rule 2) — the bug was the DHW rate, not the tariff.
Unit-pinned in test_cert_to_inputs; RealCertExpectation 74061136 = 72.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 20:04:58 +00:00
Jun-te Kim
928ce293d0 docs(ledger): re-rank at 77.0% + triage electric-tariff & fabric tail
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 19:48:10 +00:00
Jun-te Kim
06a2c413f5 test(accuracy): ratchet 75.5% -> 77.0% via secondary-fuel collision fix on cost path
The gov-API `secondary_fuel_type` enums 5 (anthracite) / 9 (dual fuel) /
33 (coal) collide in VALUE with same-valued RdSAP-10 Table-32 codes for
OTHER fuels (5=bulk LPG, 9=LPG SC11F, 33=elec 10h-low). The main-fuel
boundary and the CO2/display `_secondary_fuel_cost_gbp_per_kwh` already
canonicalise these via `canonical_fuel_code`, but the SAP-DRIVING
`_fuel_cost` cascade had a SEPARATE inline secondary rate that passed the
raw enum straight to the price lookup — so the two paths diverged and the
cost path silently mis-priced.

Anthracite (enum 5) was billed at the bulk-LPG code-5 rate 12.19 p/kWh
instead of 3.64 — a 3x over-cost on the secondary that under-rated every
solid-fuel-secondary dwelling. Worst: corpus cert 100050355518 (semi,
2 extensions, anthracite room-heater secondary) lodged 36 / engine 20.8
-> 35.5. Its CO2/PE already matched lodged (the demand cascade was right)
— a pure cost-side gap, which is what localised it.

Fix: canonicalise `secondary_fuel` before the Table-32 lookup in
`_fuel_cost`, mirroring the main-fuel boundary. 55 corpus certs carry a
colliding secondary.

Gauge: within-0.5 75.5% -> 77.0%, SAP MAE 0.708 -> 0.668 (floors
ratcheted). The prior enum-9 test only guarded the display helper; new
test_fuel_cost_secondary_colliding_fuel_priced_at_canonical_rate pins the
_fuel_cost cost path directly. RealCertExpectation pinned for
100050355518 (36 = lodged).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 19:43:31 +00:00
Jun-te Kim
c380377948 test(accuracy): ratchet 74.2% -> 75.5% via 'insulated (assumed)' roof Table-18 fix
RdSAP 10 §5.11.4: a pitched roof lodging thickness "NI" with description
"...insulated (assumed)" was billed at the §5.11.4 observed-retrofit
50 mm row (U 0.68) while the SAME description lodged with "ND" fell
through to the Table 18 age-band default (0.40 for bands A-G). The
ND-vs-NI sentinel is lodging-software noise; "(assumed)" means the
insulation PRESENCE is an age-band assumption, not an observed
retrofit, so Table 18's "thickness cannot be determined" clause
governs both forms. The bare "insulated" description (observed
retrofit, no qualifier) keeps the 50 mm row.

Evidence: the 27-cert loc4+NI corpus cohort was systematically
under-rated (~-1 SAP; zero positive movers among "(assumed)" certs),
vs the 102-cert loc4+ND cohort already on Table 18 at ~0 bias.

Gauge: within-0.5 74.2% -> 75.5%, SAP MAE 0.721 -> 0.708, CO2 MAE
0.09 -> 0.074, PE MAE 3.5 -> 3.2 (floors ratcheted). Unit-pinned in
test_rdsap_uvalues (assumed -> Table 18; bare-insulated -> 0.68
regression guard); integration pin in test_heat_transmission updated;
RealCertExpectation pinned for 10094975827 (66 = lodged, was -1.87).

Ledger: C002 103001004 deferred pending Elmhurst arbitration
(single-immersion off-peak pricing + declared cylinder loss vs
Table-2); C006 quadruplet marked ⚠ (LIG-21.0 software honours lodged
0 extract fans literally vs the Elmhurst-validated 0=unknown->default
convention, worksheet case 46).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 19:06:26 +00:00
Jun-te Kim
b9f1f428b9 test(accuracy): corpus-1000 campaign ledger + C001 4510053280 pinned 75 (lodged-software PSR gap)
Setup for the RdSAP-21.0.1 corpus gauge campaign (74.2% within-0.5,
SAP MAE 0.721 at HEAD, 1000/1000 computed):

- scripts/corpus_1000/build_worklist.py — runs all 1000 corpus certs
  through the gauge's own path (from_api_response -> Sap10Calculator),
  ranks |engine - lodged| descending, clusters by dwelling signature,
  and (re)writes scripts/corpus_1000/worklist.md preserving per-cert
  statuses and notes. 258 certs outside 0.5 in 249 clusters.

- C001 (worst cert, Δ +23.9): uprn 4510053280, ground-floor flat 47 m²
  on ASHP PCDB 100053 (Mitsubishi Ecodan 5 kW, PSR table 0.2-2.0).
  Dwelling PSR 2.031 -> SAP 10.2 N2 footnotes 44/45 heat-pump extension
  (reciprocal interpolation toward 100% at 2x largest PSR) = 305%
  space efficiency, which accredited Elmhurst also applies (golden
  case 56, record 100061). The LODGED software instead treated the
  out-of-range record as invalid and billed 100% direct electric +
  standard schedule (reproduces lodged CO2 to 1%). Lodged-software
  methodology gap, worklist ⚠, engine untouched; observed engine 75
  pinned in test_real_cert_sap_accuracy.

Gauge floors unchanged (no engine change).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 18:28:20 +00:00
Jun-te Kim
e79e83bbbe Add UPRN 100020942571 (8 Booth Close) to the SAP accuracy corpus
Pins the engine's baseline SAP (54) for this electric-room-heater +
gas-multipoint end-terrace ground-floor maisonette (portfolio 814 pid
742121) against accredited Elmhurst RdSAP-10 (worksheet SAP 53, within
1 point), with the Elmhurst Input Summary + SAP Worksheet PDFs saved as
evidence alongside the cert's epc.json.

Companion to the HHRSH storage-heater off-peak-pricing fix PR: this
baseline pin confirms the calculator is faithful on the as-lodged
dwelling; that PR fixes the HHRSH recommendation overlay (engine 48->73
vs Elmhurst 71).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 11:31:28 +00:00
Jun-te Kim
8f0432721c test(accuracy): pin RdSAP-20.0.0 PV semi uprn_22086693 (PV-list fix corpus)
Corpus validation of the modelling_e2e photovoltaic_supply-as-list fix. Cert
6102-6227-8000-0083-2292 (RdSAP-20.0.0 semi, gas combi + 2× 1.14 kW PV arrays)
crashed from_rdsap_schema_20_0_0 on the measured-array list; the fix routes it
through the dict-tolerant _map_schema_21_pv. PV correctly credited: engine 61
(no PV) → 66 (+5). Built in Elmhurst (evidence: epc.json + summary + worksheet,
fabric+heating; the PV "New Technologies" Panel-details grid deferred): worksheet
55 = engine-on-Elmhurst-inputs 55 exactly → calculator faithful. The +6 engine-vs-
Elmhurst base-dwelling residual is the documented RdSAP-default gap (band-C cavity-
uninsulated suspended-floor semi). Pinned engine 66.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 16:13:34 +00:00
Jun-te Kim
75341da42d 32 2026-06-23 15:38:38 +00:00
Jun-te Kim
17b1d63f0e test(accuracy): pin SAP-16.0 storage-flat uprn_10070004512 (built_form fix corpus)
Corpus validation of the modelling_e2e built_form fix. Cert 8742-6624-9300-2780-4926
(SAP-Schema-16.0, ground-floor electric-storage-heater flat) omits built_form; the
mapper now derives it from dwelling_type. built_form is ML-only so the fix is
SAP-neutral: engine 66 = lodged 66 exactly. Built in Elmhurst (evidence: epc.json +
summary + worksheet): worksheet 54, engine-on-Elmhurst-inputs 53 ≈ 54 → calculator
faithful. The +12 engine-vs-Elmhurst is a build/input gap (cert size-1 small cylinder
unrepresentable in Elmhurst's Normal/110L-minimum entry → higher HW + reduced-field
16.0 defaults). Pinned engine 66.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 15:35:06 +00:00
Jun-te Kim
00af7b5a54 data types 2026-06-23 12:42:53 +00:00
Jun-te Kim
210ca6397f updated sap scaema to take in inputs 2026-06-23 10:22:40 +00:00
Jun-te Kim
9c89a0e680 neighbouring properties added 2026-06-22 14:38:00 +00:00
Jun-te Kim
3044c70202 sap score and elmhirst mapper optimsaiation 2026-06-20 07:25:42 +00:00
Jun-te Kim
37b0a38425 add more test cases 2026-06-19 10:59:51 +00:00
Jun-te Kim
e6a829aaea more examples 2026-06-19 09:51:49 +00:00
Jun-te Kim
a955a09e9c Pin uprn_10093116330 (full-SAP gas-combi 2-storey semi): engine 82 vs Elmhurst 78
7th sibling full-SAP cert; documented full-SAP→RdSAP +4 residual. Build clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 10:26:11 +00:00
Jun-te Kim
e519de26a4 Pin uprn_10093116336 (full-SAP gas-combi 2-storey semi): engine 83 vs Elmhurst 79
6th sibling full-SAP cert; same documented full-SAP→RdSAP +4 residual. Build clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 09:41:29 +00:00
Jun-te Kim
317220beba Pin uprn_10093116334 (full-SAP gas-combi bungalow): engine 81 vs Elmhurst 77
5th sibling full-SAP cert validated against Elmhurst (semi-detached bungalow,
same street/boiler PCDB 17505 as 10093116324). Engine 81 (lodged 82); Elmhurst
worksheet 77. The +4 is the documented full-SAP→RdSAP residual. Build verified
clean (storeys=1, no phantom conservatory). Worklist strategy note added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 09:32:57 +00:00
Jun-te Kim
3eb0022034 Pin uprn_10093116324 (full-SAP gas-combi bungalow): engine 79 vs Elmhurst 74
4th sibling full-SAP cert validated against Elmhurst. Engine 79 (lodged 80);
Elmhurst worksheet 74. The +5 is the documented full-SAP→RdSAP residual — engine
uses the cert's measured U-values (wall 0.19/floor 0.12/roof 0.12) + PCDB combi
17505 (88.5%); Elmhurst uses RdSAP band-L defaults + generic 84% BGW combi. Build
verified clean (single-storey bungalow, no phantom conservatory, TFA 52/51.9).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 09:10:36 +00:00
Jun-te Kim
d87718f316 Merge remote-tracking branch 'origin/main' into feature/hyde_make_it_more_accurate_with_tests
# Conflicts:
#	datatypes/epc/domain/mapper.py
2026-06-17 09:05:37 +00:00
Jun-te Kim
d41798f1c1 Add SAP-18.0.0 + SAP-16.0 schema coverage; autonomous-run triage findings
Schema coverage (datatypes/epc/domain/mapper.py):
- SAP-Schema-18.0.0: full-SAP shape ≡ 17.1 → from_sap_schema_17_1, no normalisation.
- SAP-Schema-16.0: same reduced-field 16.x path; default the omitted `tenure`
  field in _normalize_sap_schema_16_x (metadata; SAP cascade never reads it).
  Genuinely sparse 16.x certs (missing core fabric fields) still fail loud.
- Regression tests + sap_18_0_0.json / sap_16_0.json fixtures; 0 new pyright errors.

Autonomous triage of the worklist (scripts/hyde/autonomous_run_findings.md):
- Found + diagnosed 2 bugs (flagged, NOT fixed): (1) MAPPER — full-SAP openings
  lodged in mm read as m → multi-million-m2 windows → SAP clamps to 1 (uprn_
  10093117227 / 10090317693 / 10091636031); (2) CALCULATOR — database heat-pump
  fuel code 39 mis-priced as gas, over-rates ~14 (uprn_10093114053).
- Most certs map within +/-4 of lodged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 20:04:43 +00:00
Jun-te Kim
0b32d9fcee Add SAP-16.3 + SAP-17.0 schema coverage (completes the e2e UPRN set)
- SAP-Schema-16.3: same reduced-field RdSAP shape as 16.2 — generalise the
  normaliser to _normalize_sap_schema_16_x and route both 16.2/16.3 through it.
  uprn_44012843 maps → SAP 79 (lodged 81).
- SAP-Schema-17.0: structurally identical to the full-SAP 17.1 schema (measured
  sap_opening_types), so it parses with the 17.1 dataclass and reuses
  from_sap_schema_17_1 with no normalisation. uprn_10023444324 → 80, uprn_
  10023444320 → 81.
- Regression tests (16.3 dispatch, 17.0 dispatch) + sap_16_3.json / sap_17_0.json
  fixtures; 0 new pyright errors. All 7 e2e UPRNs now map.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 19:24:50 +00:00
Jun-te Kim
36dc55b499 epc json 2026-06-16 18:54:07 +00:00
Jun-te Kim
4d14607e7e Add SAP-16.2 schema coverage + single-glazing fix; flat party-wall fix; pin 2 certs
SAP-Schema-16.2 (datatypes/epc/domain/mapper.py):
- 16.2 is structurally an RdSAP-17.1 cert under a different name; add
  _normalize_sap_schema_16_2 (field renames + defaults) and dispatch to the
  tested from_rdsap_schema_17_1 mapper. uprn_100020933699 maps → SAP 71.
- Honour a "Single glazed" windows description when multiple_glazing_type="ND"
  (was defaulting to double) → RdSAP-21 code 5; eng 72→71 (lodged 70).
- 4 regression tests + sap_16_2.json fixture; 0 new pyright errors.

Flat party-wall fix (domain/sap10_calculator/worksheet/heat_transmission.py):
- Full-SAP flats carry flatness in dwelling_type, not property_type, so the
  party-wall default fell through to the 0.25 house value instead of the RdSAP
  Table-15 flat 0.0. Add _is_flat_or_maisonette_dwelling fallback + regression
  test. uprn_10093116529 80→81 (matches the cert's lodged party u_value 0).

Accuracy corpus pins (tests/domain/sap10_calculator/test_real_cert_sap_accuracy.py):
- uprn_10093116543 (SAP-17.1 gas-combi semi): engine 81 (Elmhurst 77; documented
  full-SAP→RdSAP residual — measured wall/floor U + PCDB boiler vs RdSAP defaults).
- uprn_10093116529 (SAP-17.1 g/f flat): engine 81 (Elmhurst 78).

devcontainer: add poppler-utils (pdfinfo) for the documents-parser PDF fixtures.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 18:53:00 +00:00
Jun-te Kim
2f0eb49eee Checkpoint: UPRN 10093116543 Elmhurst build + devcontainer VNC/Playwright + perms
- Add SAP-accuracy sample for uprn_10093116543 (epc.json, elmhurst_inputs.md,
  summary/worksheet PDFs)
- Persist hyde viewer stack (xvfb/fluxbox/x11vnc/novnc/websockify) and Playwright
  chromium in the backend devcontainer; forward noVNC 6080
- Broaden .claude/settings.local.json allowlist (display/python/grep/tail)
- In-progress campaign mapper/cert_to_inputs work carried from prior cert

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 15:21:56 +00:00
Jun-te Kim
dad3044740 save skills and automation progress 2026-06-16 12:17:43 +00:00
Jun-te Kim
5c11fd35c8 Validate SAP calculator vs Elmhurst; fix reduced-field window U; add accuracy harness
Reduced-field window U: heat_transmission derived the synthesised-window
raw U from u_window(all None) -> the 2.5 placeholder regardless of glazing.
Now routes the (uniform) glazing_type code through u_window (RdSAP Table 24)
so e.g. double pre-2002 reads 2.8, not 2.5. Only the pre-SAP10 reduced-field
path is affected (21.0.1 certs carry per-window U upstream) — the RdSAP-21.0.1
corpus gauge is unchanged at 66.9% within-0.5.

test_real_cert_sap_accuracy: pin uprn_10002468137 (RdSAP-17.1, all-electric
storage heaters) at SAP 61, validated against Elmhurst on identical inputs
(dual off-peak immersion, 110 L cylinder, 2 baths). Our engine reproduces
Elmhurst's fuel cost to the penny; lodged 55 is the old SAP-2012 schema.

Tooling to grow the accuracy corpus:
- scripts/fetch_real_life_epc_sample.py — capture a cert by UPRN into the corpus.
- scripts/compare_epc_paths.py — diff gov-API vs Elmhurst-summary EpcPropertyData
  and run both through the engine, localising mapper vs calculator differences.
- skill validate-cert-sap-accuracy — the end-to-end loop (capture -> Elmhurst
  inputs -> human builds -> compare -> reconcile -> pin in the test).
- skill epc-to-elmhurst-rdsap-inputs reference: corrected immersion (code 1=dual),
  cylinder size (code 2 = Normal/110 L), and bath-count (WWHRS sub-tab) mappings.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 15:26:11 +00:00
Jun-te Kim
0079752eab inviestigation with hyde values 2026-06-15 12:13:11 +00:00
Jun-te Kim
3995433816 Map RdSAP-Schema-17.0 certs to EpcPropertyData 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 12:40:04 +00:00
Jun-te Kim
5178197dc2 Map RdSAP-Schema-19.0 certs to EpcPropertyData 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 12:19:16 +00:00
Jun-te Kim
cfc337f04a Dispatch and map RdSAP-Schema-18.0 certs end-to-end 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 11:12:53 +00:00
Jun-te Kim
3b7d26fe34 added test for a 1000 examples 2026-06-09 16:02:21 +00:00
Khalim Conn-Kowlessar
96ac34f422 adding hacky handling for matching on lmk key or uprn 2026-04-09 14:47:54 +01:00