An as-built cavity wall (wall_insulation_type=4) lodged "Cavity wall, as
built, insulated (assumed)" was routed to RdSAP 10 Table 6's "Filled
cavity" row. Per Table 6 (England, PDF p.41) the Filled-cavity row carries
the "†" footnote ("assumed as built") only at age bands I-M, where it is
numerically identical to "Cavity as built"; at bands A-H the Filled-cavity
row represents a GENUINE fill, not the as-built assumption. So an as-built
cavity must use the "Cavity as built" row at all bands (band G/H = 0.60,
not the filled 0.35).
This is the same latent A-H bug slice S0380.210 fixed for the "partial
insulation (assumed)" variant but left in place for "insulated (assumed)"
by a legacy production convention. The API SAP-accuracy cohort over-rated
"Cavity wall, as built, insulated (assumed)" certs at bands G/H by a clean
+1.38 / +1.61 SAP median (n=37 / n=18); bands I-M were unaffected (rows
coincide), confirming the spec mechanism per-band.
Retires the `_cavity_described_as_filled` description sniffer — as-built
cavities now always use the as-built row regardless of the rendered
insulation adjective; a genuine retrofit fill is still caught by the
explicit wall_insulation_type=2 branch.
API SAP eval: 48.6% -> 52.1% within 0.5; <1.0 63.8% -> 67.2%; median |err|
0.548 -> 0.475; mean|err| 1.561 -> 1.497; 909 computed, 0 raises.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
47.6% → 48.6% within 0.5; immersion code mapping corrected (1=dual,
2=single); next robust leads are under-rating flat/party-fabric scatter.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Electric immersion water heating (WHC 903) on an off-peak tariff billed
100% at the low rate, under-costing the dwelling and over-rating it
(median +0.98 SAP across the off-peak WHC-903 API cohort, n=57).
SAP 10.2 Table 12a "Immersion water heater" row (PDF p.191) routes the
water-heating column to Table 13 (PDF p.197): the high-rate fraction is
a function of cylinder volume V, assumed occupancy N (Appendix J Table
1b) and single-/dual-immersion. The remainder bills at the low rate.
Table 13 Note 2 supplies exact equations equivalent to the rounded grid;
`electric_dhw_high_rate_fraction` evaluates them (validated against the
published 110 L grid cells). Per Note 1 the 10-hour equations cover any
tariff with >=10 hours/day low-rate (so 18-/24-hour use that column).
Immersion code mapping CONFIRMED 1=dual, 2=single via RdSAP 10 §10.5
(PDF p.54 — an immersion is "assumed dual" on a dual/off-peak meter)
cross-checked against the API cohort (code 1 sits 3.6:1 on dual meters;
code 2 on single meters). This INVERTS an earlier handover's unverified
"1=single, 2=dual" note — the dual code carries Table 13's small
fraction, matching the cohort over-rating direction; the single mapping
overshot in a prototype.
API SAP eval: 47.6% -> 48.6% within 0.5; <1.0 62.6% -> 63.8%;
mean|err| 1.586 -> 1.561; 909 computed, 0 raises.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per Property the inspection script now resolves the UPRN's spatial
reference from the Ordnance Survey Open-UPRN parquet in S3
(GeospatialS3Repository over a boto3 ParquetReader) and threads both
levers into run_modelling:
- planning_restrictions: the conservation/listed/heritage flags that gate
the wall + solar measures (ADR-0019/0020).
- solar_insights: a live Google Solar buildingInsights fetch keyed on the
reference coordinates, so the Solar PV Options can fire (ADR-0026).
Mirrors IngestionOrchestrator._fetch's coords->solar flow. Degrades
gracefully per Property: a UPRN S3 doesn't cover -> unrestricted/no-solar;
a point Google has no coverage for (BuildingInsightsNotFoundError) ->
no-solar; both still modelled. --no-solar skips the Google leg. A context
note (restrictions; solar) is printed and written to the md/csv summary.
Verified live: spatial_for + solar fetch round-trip on real UPRNs (S3 via
ambient ~/.aws creds, pyarrow reads parquet bytes). pyright clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Updates the headline (45.1 → 47.6%), records the four shipped fixes + the
roof-8 false-lead closure, documents the two methods that worked
(description-vs-code audit + outlier-robust categorical sweep by net skew +
median), and lists the open robust leads (whc=903 immersion HW, cat-7 storage,
dual immersion) with the scatter buckets to avoid.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`_secondary_fraction` keyed "has a secondary" off the integer
`secondary_heating_type` code. The gov-API path surfaces the secondary as a
DESCRIPTION instead (`secondary_heating.description`, e.g. "Portable electric
heaters (assumed)") and leaves the integer code None. So a gas/oil boiler
main (not in the §A.2.2 forced-secondary set) with an assumed portable-electric
secondary dropped the secondary entirely (sec_kWh=0), under-costing the
dwelling and over-rating its SAP.
Per RdSAP §A.2.2 / SAP 10.2 Table 11, a lodged secondary is costed at its
Table 11 fraction (cat-2 boiler = 0.10, billed at standard-rate electricity per
the §A.2.2 assumed portable-electric default). New
`_has_lodged_secondary_description` treats a real `secondary_heating.description`
as a lodged secondary; passed to `_secondary_fraction` at both call sites. The
description is authoritative — same lesson as floor_heat_loss / roof codes.
(Electric-storage mains were unaffected: they force the secondary already.)
Also adds the Table 11 fraction for main_heating_category=8 (electric underfloor,
"Integrated storage/direct-acting electric systems" = 0.10) — the strict-raise
surfaced this latent gap once cat-8 mains were routed through the lookup.
Eval: 909 computed, 0 raises, 46.9% -> 47.6% within 0.5 (+13 certs: 420 -> 433),
mean|err| 1.633 -> 1.586. 13 improved / 1 regressed (2610, a cat-10 room-heater
cert with an independent over-count). Bucket "Portable electric heaters"
median +2.73 -> ~0 on the gas/cat-2 subset (cat-7 storage was already correct).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Revives the local recommendation-inspection flow for specific Properties.
`scripts/run_modelling_e2e.py` reads each Property's UPRN from the DB
(read-only), fetches the latest EPC live from the gov EPC API by UPRN, runs the
Modelling stage in memory (all Generators → Optimiser → costed, attributed
Plan), and prints a per-Property plan table + writes a Markdown/CSV summary.
Persists nothing — purely for inspection.
The local DB's Properties have no linked ingested EPC (epc_property.property_id
is NULL for all rows; Ingestion's source clients are stubbed, #1136), so the
EPC must be fetched inline rather than read back. Builds the connection from the
`DB_*` env vars in backend/.env and the EPC token from `EPC_AUTH_TOKEN`.
Threads optional solar insights through harness `run_modelling` (so Solar PV
Options can fire once coordinates are wired) and adds the `solar_pv` catalogue
row. Solar + planning restrictions + DB persistence are noted follow-ups.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Publish container ports without a fixed host side in the backend and
asset_list compose files, and switch forwardPorts to service-qualified
form. Lets a second worktree's devcontainer spin up without conflicting
on host 8000/5432 (backend) or 8081 (asset_list), matching the pattern
in assessment-model-sales-frontend.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`_BOILER_NO_ROOM_THERMOSTAT_CONTROL_CODES` held only {2101, 2102} — it was
keyed off the Table 4e "+0.6 °C" annotation rather than the actual interlock
criterion. SAP 10.2 §9.4.11 (PDF p.66): "A boiler system with no room
thermostat (or a device equivalent in this context, such as a flow switch or
boiler energy manager) ... must be considered as having no interlock", and
"TRVs alone ... do not perform the boiler interlock function". A fixed bypass
likewise provides no interlock (it keeps water circulating when TRVs close).
So control 2107 ("Programmer, TRVs and bypass") and 2111 ("TRVs and bypass")
lack interlock and must take the Table 4c(2) −5pp Space+DHW seasonal-efficiency
adjustment and the Table 4f footnote a) ×1.3 circulation-pump uplift — both of
which they previously missed. (2108 flow switch / 2109 boiler energy manager
carry interlock-equivalent devices → excluded; 2103-2106/2113 have a room
thermostat.) All affected certs are cat-2 gas boilers, where §9.4.11 applies.
Eval: 909 computed, 45.3% → 46.9% within 0.5 (+14 certs: 412 → 426), mean|err|
1.659 → 1.633. Bucket means corrected: control 2107 +1.50 → +0.32 (n=38),
2111 +1.48 → +0.16 (n=4). 32 improved / 10 regressed (all small; the six that
crossed out of ±0.5 were coincidentally-accurate offsetting-error certs).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pulls in 42 commits of calculator/mapper accuracy fixes from the per-cert
mapper-validation and floor/roof/heating fronts.
Conflict resolutions:
- mapper `_is_elmhurst_roof_window`: main dropped the branch's "wall location →
vertical" guard (it broke cert 000516's rooflight), but that re-broke cert
001431's two External-wall U>3.0 windows (which must stay vertical). The two
certs lodge a BYTE-IDENTICAL §11 row, so neither location nor U separates
them — the real discriminator is the room-in-roof context. Replaced the
unconditional U>3.0 backstop with one gated on the BP having a room-in-roof
(`_elmhurst_bp_has_room_in_roof`): 000516's Main BP has a "Room in roof type
1" (→ rooflight), 001431's does not (→ vertical). Validated against BOTH —
full Elmhurst worksheet suite 1038 pass + the 001431 window-extraction pin.
- property_postgres_repository: kept main's `ids_by_uprn` method + the branch's
`_restrictions_of` helper.
- sap_fuel.py: the branch relocated it to domain/billing/ (already carrying
main's to_table_32_code normalization), so kept the old path deleted.
Fallout from main's fabric fixes (validated by the boiler-3 real-cert pin which
still reproduces at delta 0):
- re-pinned the boiler-1 + boiler-instant-hw ASHP snapshot scores;
- main's §14.2 gas-boiler main-fuel derivation resolved the BGB/102 baseline
gap, so `test_gas_boiler_instant_hw_before_baselines` is now a passing test
(was an xfail tripwire).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Records the session-4 audit: walls/heating/controls clean; roof_construction=8
("Pitched, insulated", no measured thickness) computing U=2.30 is CORRECT, not a
bug — confirmed by user worksheet sim-case-29 (band C → Elmhurst SAP 55 ≡ our
56.75; lodged 80 is data-fidelity artifact). Lesson: "insulated (assumed)" = the
age-band default insulation level, not "well insulated". DO NOT re-chase roof-8.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Slice 9 of the Solar PV Recommendation Generator (ADR-0026). Pins the
overlay→calculator PV cascade against Elmhurst's before/after re-lodgements of
cert 001431, across the orientation/pitch/overshading config space (the certs
lodge synthetic 1.00 kWp test vectors):
- SE/SW, shaded (overshading 3/2), pitch 30°/45°
- E/W, unshaded, pitch 60°/45°
- NW/N, unshaded, pitch 60°/45° (the low-yield orientations)
Each hand-built SolarOverlay reproduces the relodged after at abs ≤ 1e-4 on
SAP / CO2 / primary energy.
Battery tripwire (per user): the "with battery" cert lodges a §19 5 kWh battery
the current extractor does NOT parse, so it scores identically to its
no-battery twin — the no-battery overlay reproduces it today, and the pin will
fail (alerting us to switch to the with-battery overlay) once the extractor
parses the battery. A companion test pins that the calculator already models
the 5 kWh battery (it raises SAP), so the fix target is established.
All five certs share an EES 'WGK' / SAP-code-502 main-heating lodgement the
mapper doesn't yet derive a fuel for; the pins patch the shared fuel (mains gas
26) identically on before+after to isolate the PV delta (the solar overlay
never touches heating), and `test_solar_before_baselines` xfails as the
forcing-function tripwire for that separate mapper-front gap.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Slice 8 of the Solar PV Recommendation Generator (ADR-0026). The
ModellingOrchestrator now reads each Property's persisted Google Solar
buildingInsights JSON (uow.solar), projects it once per Property into a typed
SolarPotential via `_solar_potential_for` (None for a missing or error
payload), and threads it into `recommend_solar` alongside planning_restrictions
— mirroring the ASHP wiring. Solar fires only when a feasible potential is
present, so dwellings without fetched solar data are unaffected.
FakeSolarRepo now returns None for an unseeded Property (was raising) and
supports `by_property` seeding, so the orchestrator's new solar read is exercised.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Slice 6 of the Solar PV Recommendation Generator (ADR-0026). `recommend_solar`
emits one "Solar PV" Recommendation of up to five conservatively-sized configs
× {no battery, battery} = ≤10 competing Options (a free Optimiser candidate).
Each Option folds a SolarOverlay built from the chosen config: one
PhotovoltaicArray per non-north segment (peak_power = panels × panelCapacityW /
1000; orientation/pitch from geometry; generation-calibrated overshading),
is_dwelling_export_capable set True absolutely, a diverter when the dwelling
has a cylinder (None for a combi), a 5 kWh battery for the battery variant, and
the per-config composite cost from Products.solar_bundle_cost.
Eligibility = house/bungalow ∧ not listed/heritage (blocks_internal, the same
gate as ASHP — a conservation area does NOT block PV) ∧ no existing PV ∧ a
feasible SolarPotential. Flats and existing-PV top-up are deferred.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Slice 7 of the Solar PV Recommendation Generator (ADR-0026). Adds the
composite per-dwelling Solar PV cost on the Products collection (ADR-0025
pattern): pv_system(kWp band, nearest of the ECOPV06-13 EA bands 1.0→4.5 kWp,
floor/cap at the ends) + scaffolding(£900 first elevation + £450 each
additional, default 2) + enabling base (EICR £150 + DNO £50 + 2-way consumer
unit £330) + [diverter £980 if cylinder] + [battery if the with-battery
variant] → Cost(total, contingency_rate 0.15).
Rates are data in the committed solar_rates.json (Southern Housing "SOLAR PV &
BATTERY" EA column), loaded via SolarRates.from_json/.default and injectable.
The £2,000 / 5 kWh battery is NOT on the rate sheet — a flagged estimate
(battery_estimate=true), confirmed with the user to stand in until a DB rate.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Slice 5 of the Solar PV Recommendation Generator (ADR-0026). Adds the flat
`SolarOverlay` and `_fold_solar`, the sixth Simulation Overlay surface: like
the ventilation/lighting overlays it targets no building part and folds its
fields onto `sap_energy_source` (home of the SAP Appendix M PV inputs) —
photovoltaic_arrays (absolute target, one PhotovoltaicArray per non-north
segment, replacing the dwelling's existing arrays), pv_diverter_present,
pv_connection, is_dwelling_export_capable (set True absolutely), pv_batteries.
Omitted fields leave the baseline unchanged (combi → no diverter); the
baseline is never mutated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Slice 4 of the Solar PV Recommendation Generator (ADR-0026).
`select_conservative_configs` turns Google's full solarPanelConfigs ladder
into up to five competing array configs for the Optimiser: drop north-facing
planes (within 30° of due north, wrap-aware), cap usable panels at ~70% of
maxArrayPanelsCount (imagery misses obstructions; MCS edge setback), collapse
rungs that trim to the same usable size keeping the higher-generation layout,
then sample five spanning min→max by expected generation. Returns () when
nothing usable remains.
Real London example → 5 rungs at 4/12/19/26/34 panels (all ≤34.3 = 70% of
49); synthetic cases pin the north-drop and the 70% cap.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Slice 3 of the Solar PV Recommendation Generator (ADR-0026). Per roof segment,
back-solve the effective overshading factor ZPV from Google's expected
generation against SAP's own unshaded annual output:
ZPV = (yearlyEnergyDcKwh × 0.955) / (0.8 × kWp × S)
reusing the calculator's Appendix U3.3 annual solar radiation S via a new
public seam `pv_annual_solar_radiation_kwh_per_m2`. Dividing Google's
generation by SAP's S cancels orientation/tilt and isolates shading; the
result snaps to the RdSAP bucket {1:1.0, 2:0.8, 3:0.5, 4:0.35} via the
ADR-0026 midpoint cutpoints (≥0.90→1, 0.65–0.90→2, 0.425–0.65→3, <0.425→4;
ZPV>1→1). The real London example's planes all back-solve to ZPV>1 → code 1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Slice 2 of the Solar PV Recommendation Generator (ADR-0026). Adds the
strictly-typed `SolarPotential` domain projection over the raw Google Solar
`buildingInsights` JSON that Ingestion persists (SolarRepository): the
`solarPanelConfigs` ladder, each rung broken into its roof segments with
Google's continuous azimuth/tilt mapped to the SAP octant
(`azimuth_to_sap_octant`, 0°=N clockwise → 1=N..8=NW, matching the
calculator's ORIENTATION_BY_SAP10_CODE) and RdSAP §11.1 pitch code
(`pitch_to_sap_code`, snap to {0→1,30→2,45→3,60→4,90→5}).
Pinned against the real London buildingInsights example (mirrored into
fixtures from the user-provided RTF): 400 W panels, maxArrayPanelsCount 49,
46-rung ladder, per-segment SE/NW/NE/SW octants at ~32° → pitch code 2.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Elmhurst §15.1 "Insulated: No Insulation" label was lodged but absent
from `_ELMHURST_CYLINDER_INSULATION_LABEL_TO_SAP10`, so
`_elmhurst_cylinder_insulation_code` raised UnmappedElmhurstLabel — blocking
the parse of every Solar PV example cert (the solar `before` lodges "No
Insulation"). An uninsulated cylinder has no insulation *type*, so per the
no-misleading-insulation convention it maps to `cylinder_insulation_type =
None` rather than naming a material; the lodged 0 mm thickness carries the
storage-loss signal the SAP 10.2 Table 2 dispatch needs.
Slice 1 of the Solar PV Recommendation Generator (ADR-0026).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Designed via grill-with-docs. API-first (Google Solar -> typed SolarPotential):
one Solar PV Recommendation, up to 5 conservatively-sized configs x battery
on/off; new PV overlay surface (per-roof-segment arrays, diverter conditional on
cylinder, export-capable ensured); overshading generation-calibrated from the
API's expected generation; eligibility offers conservation areas (only listed/
heritage block); composite costing from EA rates (contingency 0.15). CONTEXT
gains Solar Potential, Solar PV Recommendation, Solar PV Eligibility.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Code 3 = "(other premises below)" = above partially heated space (§3.12 →
U=0.7), confirmed 9/9 on single-BP certs (the diagnostic that dodged the
lossy-floors[] contamination). Records the 7536 re-pin and the lesson that
"irreducible residual" golden notes can mask a real mapper bug.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The API `floor_heat_loss` code is authoritative — confirmed by joining each
single-BP cert's code to its independent `floors[].description` (which the
gov register publishes alongside the code):
code 1 ↔ "To external air" (exposed, 9/9)
code 2 ↔ "To unheated space" (semi-exposed, 6/6)
code 3 ↔ "(other premises below)" (partially htd, 9/9)
code 6 ↔ "(another dwelling below)" (party, 176/176)
code 7 ↔ "Solid"/"Suspended …" (ground, all)
Code 3 was mis-mapped to "To unheated space" (semi-exposed) and, on
mid-/top-floor flats, had its floor area zeroed entirely by the
dwelling-level exposure heuristic. RdSAP 10 §3.12 (PDF p.25) classes a
flat's floor over non-domestic "other premises … heated, but at different
times" as "above a partially heated space" → the §5.14 (PDF p.47) constant
U=0.7 W/m²K — distinct from semi-exposed (Table 20) and party (no loss).
Fix: the mapper sets `is_above_partially_heated_space` on the floor=0
dimension for code 3 (string → "(other premises below)" for fidelity), and
the heat-transmission step lets that per-BP lodgement override the flat
suppression upward (mirroring the existing exposed / "another dwelling
below" overrides). The cascade already routes is_above_partial → U=0.7.
Re-pins golden cert 7536-3827: its Ext2 (bp3) lodges code 3, but the cert's
lossy `floors[]` summary dropped that description, so a prior agent guessed
"code 3 = ground" (U=1.12) and concluded the residual was an irreducible
"register-rounding" artifact. It was this bug: Ext2 floor U 1.12 → 0.70,
PE -6.1952 → -5.6414, CO2 -0.1639 → -0.1492 (both toward 0), SAP unchanged.
Eval: 909 computed, 45.1% → 45.3% within 0.5, mean|err| 1.702 → 1.659,
<1.0 59.5% → 60.2%. 13 code-3 certs improve (0380 +3.71 → -0.63, 0350
+7.82 → +0.83, 2610 +7.47 → -1.29); the few that overshoot were already
failing and carry independent fabric bugs (9763's walls = 8 W/K for 60 m²).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Record that profiler lead #1 (floor_codes=3) is not a clean single cause
(bimodal + confounded), that the paired worksheet certs confirm only
codes 1/6/7 (code 3 unmapped → needs a worksheet for 0380-2087), and that
immersion_type=2 / main_control=2107 / roof_codes=1 are scatter, not
dispatch bugs. The exposed-floor-on-flats fix (§3.12) shipped at b40e0f67.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A mid-/top-floor flat whose lowest floor is lodged as an exposed floor
(API floor_heat_loss=1) had its floor area zeroed by the dwelling-level
exposure heuristic, which keys only on the flat label and defaults
has_exposed_floor=False (assuming the floor sits over another *heated*
dwelling). RdSAP 10 §3.12 (PDF p.25) is explicit:
"Otherwise the floor area of the flat ... is:
- an exposed floor if there is an open space below"
i.e. a flat cantilevered over a passageway IS a heat-loss floor on
Table 20. The per-BP `is_exposed_floor` lodgement is authoritative and
now overrides the dwelling-level suppression upward, mirroring the
existing "another dwelling below" party override (which suppresses
downward). The code-1↔"E To external air" enum is confirmed by the
paired API+Summary worksheet certs (0350, 3800).
Eval: 45.1% → 45.3% within 0.5 (909 computed); cert 3836 +6.79 → +0.77,
5717 +1.31 → -0.07 and 0997 +0.76 → +0.05 cross into <0.5. Two
already-failing under-rated certs (7636, 2241) shift further — both are
dominated by independent cost-side over-counts the exposed floor merely
unmasks (7636 walls = 8.98 W/K for 33.87 m² is the real defect).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Joins each computed cert's signed error (eval _results.csv) with a rich
feature set extracted from its RAW API JSON (not the mapped
EpcPropertyData), then ranks (feature, value) buckets by error carried
and by |mean signed| bias. Surfaces systematic API-path handling gaps —
a field the mapper silently drops still shows as an error-correlated
bucket. Companion to eval_api_sap_accuracy.py / decompose_api_cost_error.py.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
