5 certs in a 2026 API sample raised `KeyError: 'rr_common_wall_area_m2'`
and were blocked from computing. Root cause: `_part_geometry`'s early
return (taken when a building part lodges no sap_floor_dimensions —
e.g. a party-wall-only or RR-only extension as bp[0]) returned only 6 of
the 9 keys the full return exposes, omitting rr_common_wall_area_m2,
rr_gable_area_m2 and cantilever_floor_area_m2. The §3.9 RR contribution
block reads geom["rr_common_wall_area_m2"] / ["rr_gable_area_m2"] for
EVERY part, so the floorless part's truncated dict raised KeyError at
heat_transmission.py:974.
Fix: the early return now exposes all 9 keys, the three RR/cantilever
geometry values defaulting to 0.0 — correct, since a part with no floor
dimensions has no derivable RR shell or cantilever (no floor area).
Pure contract-completion bug; no spec/U-value change.
Regression test pins the invariant directly: a floorless part's
_part_geometry keys must equal a with-floors part's keys. Validated: all
5 certs now compute (4 within ~2 SAP of lodged; the 5th, 8536, has a
separate residual). §4 suite 2393 passed; heat_transmission.py pyright
unchanged at 12, test file at 71.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Slice 2b. Timber frame (wall_construction=5) takes internal wall insulation but
not external (not constructable — ADR-0019), so the generator offers IWI only.
Cascade pin: the IWI Option reproduces the re-lodged timber-frame after at
abs(diff) <= 1e-4 (general Table 6 insulation-thickness bucket, not the solid-
brick documentary path).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Slice 2a. New recommend_solid_wall emits one Main-wall Recommendation carrying
External + Internal wall-insulation Options for an uninsulated (wall_insulation
_type=4) solid-brick (wall_construction=3) main wall, each priced at the heat-
loss wall area. Cascade pin: the generator's EWI and IWI Options reproduce
their respective re-lodged afters at abs(diff) <= 1e-4.
Detection keys on wall_construction code, not description (ADR-0019 note
corrected): the Elmhurst ingestion path leaves walls[].description empty, so
the code is the only cross-path signal; codes 1-5 are consistent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Slice 1 of solid-wall insulation. BuildingPartOverlay gains a
wall_insulation_thickness field; the generic applicator already folds it onto
SapBuildingPart by name. With wall_insulation_type=1 (EWI) / 3 (IWI) + 100 mm,
the calculator derives the post-insulation U-value (§5.8 documentary path,
λ=0.04 default) — and for IWI also lowers the thermal-mass parameter. Two new
Elmhurst before/after cascade pins (solid-brick EWI + IWI, cert 001431)
reproduce the re-lodged after at abs(diff) <= 1e-4 across SAP/CO2/PE.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two certs fetched fresh from the GOV.UK EPB register, each with an
Elmhurst Summary PDF (input) and a dr87 worksheet PDF (the (1)..(286)
ground truth):
0340-2467-9260-2006-6521 (Summary_000922 / dr87-0001-000922)
5500-5070-0822-0201-3663 (Summary_000920 / dr87-0001-000920)
Both run through BOTH front-ends — from_api_response and
from_elmhurst_site_notes — and through the rating + demand cascades.
Cross-mapper parity holds: the two paths agree to <1e-4 on continuous
SAP, fuel cost, CO2 and PE. Both paths reproduce the worksheet exactly:
0340: (255) cost 776.4295, (272) CO2 2875.0498, (286) PE 16474.5616;
fabric (33) 171.6188, (37) 205.9358; SAP int 70 = lodged.
5500: (255) cost 751.8295, (272) CO2 2423.4547, (286) PE 14397.0118;
fabric (33) 141.1226, (37) 167.3696; SAP int 66 = lodged.
Pinned in two tables of test_golden_fixtures.py:
- _EXPECTATIONS / test_golden_cert_residual_matches_pin — SAP/PE/CO2
residual vs the integer-rounded lodged register (SAP resid +0 both).
- _WORKSHEET_PE_CO2 / test_golden_cert_pe_co2_matches_worksheet —
PE (286)/(4) and CO2 (272) vs the worksheet at +0.0000 (the
load-bearing 1e-4 check; lodged register is integer-rounded).
Dropped-field audit (raw JSON keys vs the schema-21.0.1 dataclass
fields consumed by from_dict) re-run on both fresh JSONs: no new
silently-dropped fields — only created_at metadata and the
shower_outlet_type/shower_wwhrs keys already handled by
_normalize_shower_outlets (mapper.py:2047). No calculator or mapper
change required; this is pure validation + regression-pinning.
Full §4 suite: 2392 passed, 1 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The bulk endpoint 302-redirects to a 15.7 GB S3 ZIP with one NDJSON member per
year; each line wraps the per-cert payload in a stringified 'document' that
parses to the same RdSAP-Schema-21.0.1 shape from_api_response already handles.
parse_bulk_line unwraps a record; is_sap_version filters to SAP 10.2; RangeFile
exposes the S3 object as a seekable file so zipfile streams a single year's
member (and a sampler stops early) without downloading the whole archive.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
format_report_csv emits one comma-safe row per property: the calculator-error
fields (lodged/calculated/Δ/flag), the Plan headline figures (baseline+post
SAP/band, measures, cost+contingency, bill & CO2 savings, valuation %), the
flattened measure triggers, and any captured error — sortable in a spreadsheet
for a large dump.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
format_report_markdown emits: (1) cohort parity stats + a per-property
lodged-vs-calculated table flagging |Δ| > 0.5 (errors shown inline),
(2) Plans + costings (SAP/band jump, cost + contingency, bill & CO2 savings,
valuation uplift), (3) each fired measure with the EPC attributes that
triggered it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
build_property_reports models a dump in order (errors captured per-cert);
parity_report_for aggregates the lodged-vs-calculated SAP across the cohort
into the existing ParityReport (MAE/RMSE/bias/worst-N), excluding certs that
couldn't be mapped or scored. Residual convention is the calculator's own
(predicted - actual), the negative of PropertyReport.sap_error.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Section 3 of the report: build_property_report now runs the Modelling stage
and, for every Plan Measure, records the EPC attribute(s) that caused its
generator to fire (MeasureTrigger) — wall_construction/insulation for cavity
fill, roof thickness for loft, floor thickness/construction for floors, the
absent mechanical kind for ventilation. Modelling raises are captured as
plan_error, independent of the calculator-error capture.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Section 1 of the property inspection report: PropertyReport compares the
cert's lodged energy_rating_current to Sap10Calculator's un-rounded SAP and
flags |Δ| > 0.5 (the ADR-0010/0013 shadow-validation design target). A
mapping/scoring raise is captured per-cert as calculator_error, never
propagated, so one bad cert can't abort the sweep.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Updated 2130's pin notes with the case-18 finding: our cascade reproduces
the worksheet PV split to the decimal (gen 2684.17 / onsite 970.77 / export
1713.40), so the Appendix M1 β-split is exact, not the suspected bug. With
the gas PE factor also exact (1.13) and the wall measurement now wired
(S0380.215), 2130's +2/-11.72 is the irreducible API-only lodged residual
(0240-like), not a closable calculator bug. Notes-only; pin unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The schema-21 SapBuildingPart never declared `wall_insulation_thickness_
measured`, so `from_dict` silently discarded it. When a cert lodges
`wall_insulation_thickness == "measured"` the actual value (mm) lives in
that dropped field, so the cascade fell back to the 50 mm "insulation
present, unknown thickness" default instead of the lodged measurement.
Cert 2130 Ext1 lodges solid brick band B + INTERNAL insulation
"measured"/100 mm. Per RdSAP 10 §5.7 Table 8 (insulated-wall U by age band
+ insulation thickness) the 100 mm row gives U=0.32; the unknown-thickness
fallback gave 0.55. New `_api_resolve_wall_insulation_thickness` substitutes
the measured value for the "measured" sentinel; the existing
`_insulation_bucket`/Table-8 path then computes the correct U. Field added
to schema 21.0.0/21.0.1 SapBuildingPart; domain field widened to
Union[str, int] to match `roof_insulation_thickness`. Isolated: 2130 Ext1
is the only bp lodging "measured" across all 47 fixtures.
This spec-correct fix EXPOSED an offsetting under-count it had been masking
(per the repo's no-special-handling rule — the pre-fix +1 was two bugs
cancelling): 2130 cont SAP 83.35 → 83.78 (resid +1 → +2), PE -7.56 →
-11.72, CO2 -0.045 → -0.095. The exposed -11.72 PE (~-746 kWh/yr) is the
deferred gas-combi-PE + PV-β-credit under-count from S0380.45/.49, now
un-masked — the next slice. Re-pinned 2130 with the cause documented.
Suite: 2391 passed, 1 skipped. Zero new pyright errors (mapper 32=32).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Apply the deep-research off-gas figures so oil/smokeless/wood sit on the
same NEP-Apr-2026 retail / DESNZ DUKES gross-CV basis as the new coal
proxy (fuel-input, not useful-heat): OIL 9.16 -> 12.11 (prior value was
materially low vs current kerosene), SMOKELESS 10.0 -> 8.69, WOOD_LOGS
8.83 -> 8.25, WOOD_PELLETS 7.99 -> 7.38. SEG (15.0, Solar Energy UK) and
LPG (17.61, bottled-propane) kept; gas/electricity (Ofgem cap) unchanged.
CV arithmetic recorded in the snapshot _assumptions. OIL pin updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Coal and heat networks have no national retail/cap rate, so the snapshot
left them null and BillDerivation raised UnpricedFuel — dropping those
certs from an offline cohort run. Add researched proxy rates (fuel-input
basis, sources + arithmetic in the JSON _note/_gaps): COAL 7.13 p/kWh
(NEP Nov 2025 coal uprated + DESNZ DUKES house-coal GCV) and HEAT_NETWORK
16.0 p/kWh + 69.4 p/day (Insite Energy operator sample; indicative, schemes
vary ~8-30). Both flagged proxy/indicative — sense-check estimates, not
market rates. Existing curated fuels are unchanged.
Replaces the unpriced-raises pin for these two with a positive rate pin;
off-peak stays unpriced pending the day/night accessor. Golden cohort now
runs 57/57 offline with zero errors.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
After three faithful-worksheet iterations (simulated cases 15/16/17), the
7536 +1 SAP residual is confirmed 0240-like — an Elmhurst register-rounding
residual not reproducible from the API-only JSON, not a calculator bug.
Case 17 is faithful on windows (Main 16.98 / Ext1 13.59 / Ext2 1.89) and
ground floors; every per-element value matches our cascade: walls
0.70/0.28/0.40, roofs 0.40/0.18/0.68 (S0380.214), window U-eff
2.4368/1.8519, ground floors 0.97/0.26/1.12. The only worksheet divergences
were manual-entry artifacts: case-16 inverted the floor order (put the
50.98 m² upper floor as ground), and case-17 auto-derived spurious "to
external air" exposed floors from the small-ground/big-upper geometry —
real 7536 lodges floor_heat_loss 2/7/3 (unheated-space / ground), none is
code 1 (exposed). Our spec-correct cont SAP is 68.924; lodged 68 carries
Elmhurst's own residual.
Notes-only; pin unchanged (resid +1, PE -6.1952, CO2 -0.1639). Suite green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CertResult now carries its Plan (with flat baseline/post-SAP/measures
properties), and `format_cohort_csv` renders one browsable row per cert
(SAP transition, band, measures, cost, bill saving, valuation %, error).
`scripts/run_modelling_cohort.py` is turnkey: no args runs the committed
golden cohort, prints a sense-check table for the first measure-bearing
certs (a capped preview so a large dump doesn't flood the terminal), the
summary, and writes modelling_cohort.csv (gitignored). Point it at the
EPC dump when it lands.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`harness.cohort.run_cohort(paths)` parses each API-shaped EPC JSON with
from_api_response and models it via run_modelling — no database, no
network — capturing per-cert errors instead of aborting the sweep, plus
`format_cohort_summary`. A thin `scripts/run_modelling_cohort.py` CLI
points it at a directory. Proven over the 57 golden API certs: 56 ran
offline, 15 produced measures, 1 errored (COAL has no Fuel Rates entry —
a BillDerivation coverage gap, not a harness one). Ready for the EPC dump.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two fixes that unblock offline, no-database inspection over an arbitrary
EPC dump:
- Complete the harness sample catalogue with loft_insulation and
solid_floor_insulation — the four fabric generators can emit five
Measure Types, but the catalogue priced only three, so an offline run
on a property with an uninsulated loft or solid floor raised mid-run.
A new test pins the catalogue to cover every generator Measure Type.
- Add `run_modelling(epc, ...)` — runs ONLY the Modelling stage (no
Ingestion / Baseline), so it needs no lodged recorded-performance / RHI
and inspects recommendations on any calculator-scorable EPC. `run_one`
(full pipeline) stays for when you want Baseline too.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Plan derives its Valuation Uplift (ADR-0018) from its baseline -> post
band jump and works+contingency cost, given one external input — the
Property's current market value (a Property Valuation, mostly absent).
`Plan.valuation` / `Plan.baseline_epc_rating` are derived like the other
headline figures; `PlanModel.from_domain` maps the £ forms to the live
plan.valuation_* columns (NULL when no value — the percentage is not
persisted on those columns). `Property.current_market_value` is the new
optional source; the orchestrator threads it onto the Plan. `run_one`
takes a `current_market_value` so the harness can value the uplift, and
the sense-check table shows the average % (always) plus the £ forms when
known.
Sourcing the current market value (upload / default) remains deferred
(ADR-0018); it is None throughout until that lands, so the columns stay
NULL at scale.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A "Pitched, sloping ceiling" roof (roof_construction code 8) lodged with
"As Built" insulation (no measured thickness → None) was wrongly routed to
RdSAP 10 Table 18 column (1) "insulation between joists or unknown". A
sloping ceiling has no joist void, so per RdSAP 10 §5.11 roof-input item
5-5 ("Sloping ceiling insulation … unknown / as built → Table 18") and
Table 18 note (b) ("Applies also to roof with sloping ceiling") it takes
column (3) — band F = 0.68, band L = 0.18 (vs col 1 0.40 / 0.16).
Discriminator is the code-8 "sloping ceiling" string only: code-5 vaulted
ceilings stay on column (1) per the 33 cohort-2 "ND" vaulted certs
(S0380.211), and the "NI"/"ND" unknown case is untouched. New
`is_pitched_sloping_ceiling` flag threaded from heat_transmission to
`u_roof`; pre-1950 bands already reach the same col (3) value (2.30) via
the mapper's thickness=0 → Table 16 row-0 override, so the new branch
carries the post-1950 bands where col 1 ≠ col 3.
Worksheet-validated by simulated case 15 (the 7536 replica): our cascade
on its Summary matches the P960 worksheet exactly — roof HLC 29.17 W/K,
cont SAP 65.04 vs 65. Re-pins golden cert 7536: roof 26.77 → 29.17, cont
SAP 69.071 → 68.924, PE -7.0776 → -6.1952, CO2 -0.1875 → -0.1639 (SAP
integer 68, resid +1 unchanged — the remaining +0.92 is a diffuse demand
under-count needing a fully-faithful worksheet). Blast radius: 7536 only.
Suite: 2388 passed, 1 skipped (main); sap10_ml 233 passed + 2 pre-existing
stone-formula failures (out of scope). Zero new pyright errors.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The financial-uplift model per ADR-0018. `estimate_valuation_uplift(
current_band, target_band, current_value=None, total_cost=None)` returns
a `ValuationUplift`: band-transition uplift compounded from four broker
tables (MoneySupermarket / Lloyds per-step, Knight Frank / Rightmove
whole-jump), taking min/max/mean across the covering sources. Always a
percentage; absolute £ forms (increase at each bound + post-retrofit
value) only when a current market value is supplied; the 2x ROI cap
rescales the percentages and can only bite once a value is known. A
non-improving jump is a clean 0% no-op.
Pure function, no external dependency. Persisting it (where the value
lands) and sourcing the current market value stay deferred (ADR-0018).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Expand half of the recommendation_materials retirement (ADR-0017). A
Plan Measure installs a single Product, so thread its catalogue id end to
end — Product.id -> MeasureOption.material_id -> PlanMeasure.material_id
-> recommendation.material_id — replacing the per-material BOM child
table with one nullable column on the row. ProductPostgresRepository
reads the id from MaterialRow; the four fabric generators set it on their
Option; the orchestrator carries it onto the Plan Measure; the mirror
declares + maps the column. Optional throughout (the JSON stopgap
catalogue carries no ids -> NULL).
The multi-measure integration test now pins each persisted measure's
material_id to its seeded MaterialRow id. Migration spec (live column
must be added before this deploys; contraction is the owner's next step)
in docs/migrations/recommendation-material-id.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Slice 3. `harness.console.run_one(epc, goal_band=...)` wires the full
AraFirstRunPipeline against in-memory fakes — no Postgres, no network —
runs one property, prints the sense-check table, and returns the Plan
for interactive poking from a REPL at the worktree root. Defaults to the
committed harness sample catalogue.
Refactors the slice-1 integration test to delegate to run_one (dropping
~70 lines of duplicated wiring + the now-unused test catalogue fixture),
so it exercises the shipped entrypoint rather than a parallel copy. The
new console test covers run_one's print/return contract.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Slice 2. `harness.plan_table.format_plan_table(plan)` renders a Plan as a
plain-text table — one package summary line (baseline SAP/band -> post
SAP/band, CO2 saved, cost of works + contingency, bill saved) and one
line per Plan Measure (signed SAP points, cost, delivered kWh + £
savings). Pure presentation: reads the Plan, computes nothing. The
DB-less First Run test now prints it (visible under `pytest -s`) so the
modelled package can be eyeballed and debugged by hand.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Slice 1 of the DB-less inspection harness. Complete the in-memory
FakeUnitOfWork so the ModellingOrchestrator runs with no Postgres:
add FakeScenarioRepository + FakePlanRepository (idempotent, keyed by
(property_id, scenario_id)), expose scenario/product/plan on the fake
unit, and grow FakePropertyRepo to compose the effective EPC from the
EPC repo at read time — mirroring PropertyPostgresRepository, so the
EPC Ingestion persists is visible to Baseline + Modelling (the
through-repos hand-off, in memory).
The new integration test drives the full AraFirstRunPipeline
(Ingestion -> Baseline -> Modelling) against the FakeUnitOfWork — no
Session ever opened — on the uninsulated 000490 fixture with its lodged
recorded-performance filled in (it already carries the RHI block, so
Baseline can run) and asserts a multi-measure Plan is produced. The
committed product catalogue prices the wall/floor/ventilation measures
it fires.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Cert 9390 (community mains-gas boiler, API main_fuel_type=20) drew £0
standing charge → fuel cost under-counted → SAP read +4 high (71 vs 67).
Root cause: the standing-charge logic (`additional_standing_charges_gbp`)
only knows the GAS branch (`_is_gas_code`) and the off-peak-electric branch.
A heat-network community fuel is not a Table-32 gas code — EPC 20 = "mains
gas (community)" normalises to Table-32 code 20 (biomass), so
`_is_gas_code(20)` is False and the standing came out £0. The Summary path
masks this because it lodges community gas as Table-32 code 1 (ordinary
mains gas), which IS gas-recognised and already draws the £120 gas standing
— so the CH1-6 corpus was unaffected while the API path lost the charge.
Spec basis (verified against SAP 10.2 spec PDF):
- Table 12 (p.191) "Heat networks" row standing charge = £120/yr, note (k).
- Note (l): "Include half this value if only DHW is provided by a heat
network."
- §C3.2 (p.58): the full charge applies when the space heating is also a
heat network.
Worksheet-validated: simulated case 14 (community boilers + mains gas,
space + water) → worksheet (351) Additional standing charges = £120.
Fix: new `_heat_network_standing_charge_gbp(epc, main)` returns the
heat-network standing (£120 full when the space main is a heat network;
£60 when only DHW is on the network) or None otherwise. Applied at both
fuel-cost call sites, REPLACING the fuel-based `additional_standing_charges
_gbp` for heat-network mains (NOT additive) so a Summary-path community-gas
main — already £120 via the gas branch — is not double-counted to £240. The
CH1-6 community corpus stays exactly £120 (59 corpus tests pass).
9390 SAP +4 → -2 (cont 65.39 vs lodged 67): the spec-correct £120 standing
EXPOSES a separate ~7% demand over-count (also visible as PE 220 vs lodged
205) — a heat-source-efficiency-default / fabric residual, follow-up scope.
9390 is unpinned (retired P2.2 per ADR-0010 §10); helper locked by 2 unit
tests. Full suite 2386 passed, 1 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Cert 9390-2722-3520 (community mains-gas boiler scheme, sap_main_heating_
code=301, main_fuel_type=20) emitted CO2 0.44 t vs lodged 2.8 t — 6.4x low.
Root cause: the EPC `main_fuel_type` enum and the SAP Table 12 / Table 32
fuel-code numbering COLLIDE in the 18-25 range. Per
`datatypes/epc/domain/epc_codes.csv` (RdSAP-Schema-17.0) EPC fuel
20 = "mains gas (community)", but Table 12/32 code 20 is a solid biomass
fuel (CO2 0.028, PE 1.046, wood-logs price). The factor lookups
(`co2_factor_kg_per_kwh` / `primary_energy_factor` / `unit_price_p_per_kwh`)
check the Table-12/32 dict FIRST, so the EPC community fuel 20 silently
returned the biomass factor instead of translating 20 -> Table 12 code 51
(community mains gas: CO2 0.210, PE 1.130, mains-gas price).
Fix: new `_heat_network_factor_fuel_code(main)` translates the EPC community
fuel to its Table-12 code via `API_FUEL_TO_TABLE_12`, but ONLY for
heat-network mains (`_is_heat_network_main`) — a genuine biomass boiler
(non-community) keeps its raw Table-12 factor. Applied at the five
heat-network factor sites: space-heating CO2 / PE / unit-price and
water-heating (WHC 901) CO2 / PE. The Summary path is unaffected (it maps
"Mains gas - community" to code 1, no collision), so the community-heating
corpus (CH1-6) is untouched.
Worksheet-validated against simulated case 14 (community boilers + mains
gas, SAP code 301): worksheet (367) CO2 factor 0.2100, (467) PE factor
1.1300 — exactly the Table-12 code-51 values the translator now reaches.
9390 CO2 0.44 -> 3.03 t (lodged 2.8; spec-correct factors over the API-only
register residual per [[feedback-worksheet-not-api-reference]]), PE 204 ->
220 (the spec-correct 1.13 factor; the prior 204≈205 match was the
collision coinciding with the register residual). 9390 is unpinned (retired
at P2.2 per ADR-0010 §10); the translator is locked by two unit tests.
REMAINING (separate follow-up): 9390 SAP +4 is a cost-side gap — the
heat-network cost path does not apply the 1/heat_source_eff (1/0.80)
scaling that the CO2/PE paths do, so community fuel cost under-counts.
Suite: 2616 passed, 1 skipped (community corpus green); the 2
test_rdsap_uvalues stone-formula failures are pre-existing (HEAD 58ff7d88).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes the Ext1 vaulted-roof over-count that S0380.209 exposed on golden
cert 0240-0200-5706. BP2 lodges roof_construction=5 (vaulted ceiling),
roof_insulation_thickness="NI" (parsed to 0), description "Pitched,
insulated (assumed)", band J. The cascade returned U=0.68 — the RdSAP 10
§5.11.4 (p.44) retrofit-50 mm "insulation at joists" row. A vaulted /
sloping ceiling has no ceiling-joist void, so that row does not apply; per
RdSAP 10 §5.11 Table 18 (p.45) it takes the column (1) age-band default
(band J = 0.16).
The arbiter is the cohort, not the spec text alone: 33 cohort-2 certs
lodge "ND" (thickness None) vaulted roofs (roof_construction=5, band D)
that already pin to their dr87 worksheets at U=0.40 = Table 18 col (1) by
falling through the age-band default. 0240's only difference is the "NI"
sentinel (insulation present, unknown thickness) which uniquely hit the
0.68 override. (The S0380.209 note's predicted "cont ≈ 72.31" assumed a
col-3 0.25 value; the cohort's ND vaulted roofs disprove that — they use
col (1), so 0240 lands at cont 72.4617.)
Implementation: new `u_roof(is_sloping_ceiling=...)` flag, threaded from
heat_transmission for roof_construction_type containing "sloping ceiling"
(code 8) or "vaulted" (code 5). It fires only for the NI case
(thickness 0 + "insulated (assumed)"), routing to the col (1) age-band
default; the "ND"/None path is untouched (already col 1) and a NORMAL
pitched-with-loft roof still takes the §5.11.4 50 mm row (flag defaults
False). roof 76.93 → ~68 W/K → 0240 PE +5.5044 → +1.5181, CO2 +0.2757 →
+0.0728 (SAP integer 72 unchanged — the true value; lodged 73 needs the
unpreserved 2013+ pump).
Also corrects test_u_wall_cavity_as_built_partial_insulation_routes_to_
filled_cavity_row → ..._routes_to_as_built_row: a missed S0380.210
follow-up. That test (in domain/sap10_ml/tests/, which the AGENT_GUIDE §4
suite command does not run) asserted the pre-S0380.210 "partial insulation
→ filled" behavior on legacy-map parity, not worksheet evidence; S0380.210
corrected it to the as-built row per RdSAP 10 Table 6 + golden cert 0390's
four-metric closure.
Suite: 2614 passed, 1 skipped; the 2 remaining failures in
test_rdsap_uvalues.py (stone §5.6 thin-wall formula vs Table-6 1.7 cap)
are pre-existing (fail at HEAD 58ff7d88, before this branch's work).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move the scenario and installed_measure tables into
infrastructure/postgres/modelling/ as full-parity SQLModel definitions
(ScenarioModel, InstalledMeasureModel + MeasureType), completing the cluster
consolidation. backend/app/db/models/recommendations.py is now a pure
re-export shim.
ScenarioModel.goal is the PortfolioGoal enum (legacy planning branches on it),
sourced from domain/modelling/portfolio_goal.py; the repo's to_domain maps it to
its value string, so domain Scenario.goal is now the value ("Increasing EPC")
consistent with the orchestrator's check — fixing the latent name-vs-value
inconsistency the old str column masked (the scenario repo test stored the enum
*name*). Parity columns are nullable (mirror convention; live NOT-NULLs owned by
Drizzle).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Standardise the modelling persistence classes on the …Model suffix (PlanModel,
RecommendationModel, RecommendationMaterialModel) — matching the epc_property
precedent and the legacy names the rest of backend/ already imports, so the
shim's plan re-export becomes literal (no alias) and the eventual shim deletion
needs zero renames. The …Row→…Model sweep for the non-cluster tables
(Property/Task/Material/…) waits until their live legacy …Model counterparts
are retired, to avoid reintroducing dual-definition collisions. No behaviour
change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Golden cert 0390-2954-3640 (detached, TFA 360, age F) carried a +7 SAP /
-28 kWh/m² PE residual the audit attributed to a demand-side fabric gap.
Walking the §3 cascade localised it to the Main wall: lodged
wall_construction=4 (cavity), wall_insulation_type=4 (as-built / assumed),
description "Cavity wall, as built, partial insulation (assumed)". The
cascade mis-routed it to the Table 6 "Filled cavity" row (band F = 0.40)
because `_described_as_insulated` matches the "partial insulation"
substring.
RdSAP 10 Specification (10-06-2025) Table 6 — Wall U-values, England
distinguishes two cavity rows:
"Cavity as built" A-E 1.5, F 1.0, G 0.60, H 0.60, I 0.45, J 0.35, ...
"Filled cavity" A-E 0.7, F 0.40, G 0.35, H 0.35, I 0.45†, J 0.35†, ...
An "as built ... partial insulation (assumed)" cavity is the as-built
partial fill of the age band, NOT a retrofit cavity fill (a genuine fill
lodges the distinct "Cavity wall, filled cavity", wall_insulation_type=2).
It therefore routes to "Cavity as built" (band F = 1.0), mirroring the
worksheet-validated solid-brick rule in S0380.209 (cases 9/10: "as built,
insulated (assumed)" → as-built age-band row, not retrofit).
New `_cavity_described_as_filled` predicate is used only in u_wall's
cavity filled-row branch; it excludes the "partial insulation" substring
while keeping "insulated (assumed)" → filled (the unrelated, separately
asserted test_cavity_as_built_insulated_assumed_uses_filled_cavity_row is
unchanged). The shared `_described_as_insulated` (also consumed by the
roof/floor paths) is left untouched.
Wall HLC +53.6 W/K (U 0.40 → 1.0 over ~268 m²) lifts all four metrics
together — the signature of a real fabric bug, not a tuned offset:
SAP +7 → +0
PE -27.9745 → +0.5281 kWh/m²
CO2 -2.7134 → -0.1189 t/yr
Bands I-M are unaffected (the two rows coincide per the † footnote), so
golden certs 0535 (band M) / 7536 (band L) with "insulated (assumed)"
cavities continue to pin at 0. Full suite 2384 passed, 1 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move the live plan, recommendation, recommendation_materials and (retiring)
plan_recommendations tables into a new infrastructure/postgres/modelling/
subpackage as single SQLModel definitions (the epc_property pattern), absorbing
the rebuild's partial PlanRow/RecommendationRow mirrors and carrying full
legacy column parity plus recommendation.plan_id. Out-of-cluster references are
plain indexed ints (mirror convention); the live FKs are owned by the Drizzle
schema. backend/app/db/models/recommendations.py becomes a re-export shim
(ScenarioModel/InstalledMeasure stay for a later slice).
Fix the export conftest to create SQLModel-first (so Base funding_package's FK
to the now-SQLModel plan resolves) and skip the redundant drop_all on its
function-scoped throwaway DB (the epc enum type is now shared across both
metadatas). Resolves the pre-existing dual-definition collision: the rebuild
and legacy export suites are now co-runnable. No behaviour change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The EPC renders a recent-band as-built wall as "<material>, as built,
insulated (assumed)". The API mapper populates epc.walls with that string,
and heat_transmission's wall_ins_present gate keyed off the "insulated"
substring → routed the wall to the RdSAP 50 mm "insulation of unknown
thickness" bucket (e.g. sandstone band J U=0.25) instead of the as-built
age-band row (U=0.35).
Per RdSAP 10 Table 8/9 footnote the 50 mm row applies ONLY when insulation
is "known to have been increased subsequently (otherwise 'as built'
applies)". An "as built ... (assumed)" description is the EPC's age-band
assumption — it only renders on RECENT bands (an old band renders "no
insulation (assumed)"), so the as-built row applies. Genuine retrofit is
signalled by wall_insulation_type (External/Internal/Filled), which the
gate still checks independently.
Worksheet-validated by two new Elmhurst worksheets, both As Built band J:
- simulated case 9: sandstone → (29a) U 0.35
- simulated case 10: solid brick → (29a) U 0.35
both the as-built row, NOT 50 mm (0.25).
Fix: restrict the description-based gate to genuine retrofit via the new
local `_described_as_retrofit_insulated` (excludes "as built"/"(assumed)").
The cavity filled-row routing inside `u_wall` (which uses
`_described_as_insulated` directly) is untouched — the 3 cavity API certs
(0390/0535/7536) are unaffected.
test_heat_transmission: the old `..._uses_50mm_row` test asserted 50 mm via
an IMPOSSIBLE band-B + "insulated (assumed)" combination; corrected to a
valid recent-band (J) scenario asserting the as-built row (35 W/K).
Golden 0240: walls 24.45 → 34.23 W/K (U 0.25 → 0.35). SAP integer 72
unchanged; PE residual re-pinned +1.8687 → +5.5044, CO2 +0.0907 → +0.2757.
This spec-correct fix REMOVED the wall under-count that was masking the
Ext1 vaulted-roof over-count (cascade U 0.68 via the same "insulated
(assumed)" description vs case-9 sloping-ceiling 0.25) — that roof
over-count is the next slice; fixing both lands SAP cont ≈ 72.31 (=
Elmhurst case 9).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`_plan_for` now scores the baseline + every cumulative prefix once
(`cascade_scores`, best-practice order) and reuses those Scores for both the
role-3 marginal attribution and a per-measure bill cascade: bill each prefix at
one Fuel Rates snapshot and take consecutive Bill deltas as each measure's
marginal delivered-kWh and £ saving. Saving is signed (ventilation is
negative) and telescopes exactly to the Plan headline savings, because the
Plan's baseline/post Bills are now the same cascade endpoints (`bills[0]` /
`bills[-1]`) — which also drops the redundant standalone baseline `calculate`.
`recommendation.kwh_savings` / `energy_cost_savings` are filled from these.
Adds `Bill.total_consumption_kwh` (shared by Plan + the orchestrator). Pinned
end-to-end on the real calculator: Σ per-measure savings == the Plan totals
(ADR-0014 amendment).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`PlanMeasure` grows optional `kwh_savings` (delivered energy) and
`energy_cost_savings` (£) — its slice of the telescoping bill cascade, signed
so positive is a saving and `None` until billing runs. `RecommendationRow`
declares the matching live `recommendation.kwh_savings` /
`energy_cost_savings` columns and maps them in `from_domain` (None → NULL).
The vestigial `recommendation.energy_savings` stays undeclared (legacy = 0).
No FE migration — the columns already exist on the live table (ADR-0014 / 0017).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds simulated case 7: case 6 (P960-0001-001431) with the heating swapped
to a CONDENSING OIL COMBI (SAP code 130, Table 4b 82/73) and the cylinder
removed — combi instantaneous DHW (WHC 901), Table 3a keep-hot combi loss
(61) = 600 kWh/yr, no primary/storage loss, boiler interlock PRESENT (no
−5pp). This is the heating archetype golden cert 0240-0200-5706-2365-8010
uses, which case 6 (SAP code 127, a *regular* condensing oil boiler +
cylinder) never exercised.
The cascade reproduces the case-7 worksheet EXACTLY at abs=1e-4 on every
top-level SapResult output with ZERO calculator changes:
(211) 7865.4304 (213) 7556.9821 (219) 3496.8121 (98c) 12646.3783
(255) 1123.3372 (257) 1.9631 (272) 5738.9315 (258) 73
This validates the SAP 10.2 Appendix D Eq D1 combi efficiency blend +
Table 3a keep-hot combi loss + Table 4b code 130 (82/73) path, and
exonerates the combi mechanism as the source of 0240's API-path residual
— which therefore lives in 0240's fabric/demand or the API mapper.
Test-only slice (no impl change). New fixture file: 0 pyright errors.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pull the cumulative-prefix scoring out of `marginal_impacts` into a reusable
`cascade_scores(scorer, baseline, overlays) -> list[Score]` (index 0 the
baseline, one calculator run per prefix) plus a pure `marginals_from_scores`.
Each Score carries its SapResult, so the next slice's telescoping per-measure
bill cascade can re-bill the same prefixes the role-3 attribution already
scores — no extra `calculate` calls (ADR-0014 / ADR-0016). `marginal_impacts`
now delegates; behaviour unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ModellingOrchestrator gains a constructor-injected FuelRatesRepository (mirrors
Baseline): run() resolves get_current() once and reuses one BillDerivation across
the batch. _plan_for prices the baseline and post-package end-states from the
SapResults already on their Scores (no extra calculate) and passes the Bills to
Plan. PlanRow mirror + from_domain gain the four live columns post_energy_bill /
energy_bill_savings / post_energy_consumption / energy_consumption_savings.
Pipeline/handler wire the fuel-rates repo. Integration tests assert the columns
persist: the multi-measure (fallback) plan shows positive bill+consumption
savings; the already-at-target zero-measure plan shows the current bill with
exactly zero savings. Fuel-switch measures price at the new fuel for free (we
bill the simulated end-state). 183 modelling/billing/orchestration/repo tests
pass, pyright strict clean. Plan-level only; per-measure savings next.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Plan gains optional baseline_bill / post_bill (the Bills derived for the
unmodified and post-package end-states at one Fuel Rates snapshot) and derives
the four plan-level columns: post_energy_bill (post total), energy_bill_savings
(baseline - post), post_energy_consumption (Σ post section kWh), and
energy_consumption_savings (baseline - post delivered kWh). All return None until
billing runs (persisted as NULL), so existing Plan construction and the
not-yet-wired orchestrator stay green. Plan-level only; per-measure savings are a
later slice (ADR-0014 amendment).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Score gains sap_result: Optional[SapResult], populated by PackageScorer with the
calculator output its headline figures came from. This lets the Modelling stage
price the post-package (and baseline) end-state via Bill Derivation reusing a
SapResult already computed by the optimiser's re-score / the orchestrator's
baseline score — no second calculate (ADR-0014 amendment). The optimiser reads
only sap_continuous, so it stays domain-agnostic and the stub scorers (which omit
sap_result) are unaffected — all optimiser tests pass unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bill / EnergyBreakdown / BillDerivation / sap_fuel were under
domain/property_baseline/ only because Baseline was built first. The Modelling
stage now needs them too, so move them (and their tests) to a neutral
domain/billing/ — Fuel/FuelRates already live in the shared domain/fuel_rates/.
Avoids a modelling -> property_baseline cross-stage import and a package name
that wrongly implies ownership (ADR-0011, ADR-0014 amendment). Pure git mv +
import rewrite across 10 files; 40 billing/baseline/repo tests pass, pyright
strict clean. CONTEXT.md Bill Derivation location updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The orchestrator already threads budget/target_sap/dependencies into
optimise_package, so no orchestrator change was needed. Add an integration test
proving the new objective end-to-end on the real calculator: a band-D property
(~57.4) with a goal of band D — already met — yields a Plan with NO measures and
zero cost (the old max-gain objective would have recommended wall+floor+vent,
improving within the band it is already in). Clarified that the existing
multi-measure test now exercises the max-gain fallback (goal C unreachable from
D, tops out ~61). Narrowed Optional sap_points/estimated_cost through locals to
keep pyright strict-clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The warm-start (and max-gain fallback) now price each forced Measure Dependency
the candidate triggers, not just inject it afterwards: optimise/optimise_min_cost
fold dependencies into each candidate's cost+gain via _augmented_cost_gain, and
optimise_package scores each dependency's true role-1 signal (_with_role1_signals)
instead of the 0.0 placeholder. This stops the min-cost objective (i) ignoring the
~£900 a wall drags in (a wall-free package reaching target can be cheaper) and
(ii) picking a small-gain wall whose mandatory ventilation (down to -5 SAP) makes
it net-negative, which repair cannot un-pick.
Budget is now a hard envelope: the constraint applies to the augmented (measure +
its ventilation) cost, so a wall that fits alone but whose ventilation would bust
the budget is DROPPED rather than forced over budget. This reverses the earlier
'forced regardless of budget' call (which made sense when selection was
ventilation-blind). Safety invariant intact — presence still injected on every
path; we just never recommend a wall we can't afford to ventilate. ADR-0016
amendment updated. 94 modelling+orchestration tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
With S0380.201-206 closing every line ref, the detached dual-oil case 6
(Main 1 radiators 51% / Main 2 underfloor 49%, different parts, no boiler
interlock, 6 roof-of-room rooflights) now matches its P960-0001-001431
worksheet to 1e-4 on the whole SapResult. Registered in
`test_e2e_elmhurst_sap_score.py::_FIXTURE_PINS` (11 pins):
SAP 72 / cont 71.6597, ECF 2.0316, cost 1162.5374, CO2 5953.6679,
space heat (98c) 11991.9611, main fuel (211)+(213) 14736.9564,
HW (219) 4902.8601, lighting (232) 357.6571, pumps (231) 356.0.
This was the validation target the S0380.200 handover set. Updated the
fixture docstring's stale "§3-windows-only" scope note.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SAP 10.2 Appendix D §D2.1(2) Equation D1 blends the monthly water-heater
efficiency by the ratio of the boiler's space-heating load to its water
load. On a dual-main cert the DHW boiler does only its OWN share of space
heating ((204) for Main 1, (205) for Main 2), but the cascade fed Eq D1
the dwelling total ((202) = 1 − secondary). That over-weighted η_winter
and under-stated HW fuel — simulated case 6 (Main 1 serves DHW + 51% of
space heat) was HW −78 kWh vs the worksheet.
New `_water_heating_main_space_fraction` returns the DHW main's total-
space share via `_water_heating_main` (WHC-901 → Main 1 (204); WHC-914 →
Main 2 (205)); single-main / WHC-901 single systems get (202) = 1 −
(201), so they are unchanged. Case 6 (219) HW now 4902.8601 EXACT.
With S0380.205 (demand exact), case 6 now closes to 1e-4 on EVERY metric:
SAP cont 71.6597, ECF 2.0316, cost 1162.5374, (211)+(213) 14736.9564,
(219) 4902.8601, (231) 356, (232) 357.6571, CO2 5953.6679 (rating) /
4895.2137 (demand).
Re-pin: 0240 (dual combi, WHC 901, Main 1 51%) HW rises slightly → PE
+1.6893 → +1.8687, CO2 +0.0815 → +0.0907 (SAP 72 unchanged). Single-main
certs unchanged (2360 pass + 0 fail).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When two main heating systems heat different parts of a dwelling, SAP
10.2 §7 (PDF p.186) adapts the mean-internal-temperature calculation:
- Table 9b weighted responsiveness: R = (1−(203))·R_sys1 + (203)·R_sys2.
- Rest-of-dwelling temperature (90)m = weighted average of T2 computed
under EACH system's control schedule, weights (203)/[1−(91)] for sys2
and [1−(203)−(91)]/[1−(91)] for sys1 (or sys2's control alone when
(203) ≥ 1−(91)).
The cascade used Main 1's control + R=1.0 for the whole dwelling,
over-stating MIT by +0.037 °C on simulated case 6 (Main 1 radiators/2106
type 2 living + Main 2 underfloor/2110 type 3 elsewhere, R 1.0/0.75). That
inflated (97) heat loss by ~11 W → demand +61 kWh/yr.
`mean_internal_temperature_monthly` gains `main_2_control_type`,
`main_2_fraction`, `main_2_responsiveness`; cert_to_inputs derives them
from the second main detail (gated on main_heating_fraction > 0, so
single-main / DHW-only second mains pass the defaults → unchanged).
Case 6: (87) living, (90) elsewhere, (98c) demand 11991.96 and per-system
fuel (211)=7741.6458 / (213)=6995.3106 all match the worksheet to 1e-4.
Re-pin: golden 0240 (same 2106/2110 archetype, API-only) — PE +2.1519 →
+1.6893, CO2 +0.1051 → +0.0815 (both closer to zero; SAP 72 unchanged).
Single-main certs unchanged (2360 pass + 0 fail).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
