The gap-fill is wired end-to-end (slices 5a-5e) behind seams; this note is what's
left to switch it on in production: (1) implement the PredictionTargetAttributesReader
stub over property_overrides — with the override-value → API-code mapping
select_comparables needs; (2) run the epc_property.source Drizzle migration; (3)
pass the three optional collaborators at the IngestionOrchestrator composition
root. Plus the open Validation-Cohort exclusion (no code path exists yet — exclude
on source_path == "predicted" when one is built) and the anomaly dual-use pointer.
No code change: the validation exclusion has no consumer to attach to today, and
the structural signal (source_path == "predicted") already exists from slice-5a.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a `source` discriminator (lodged | predicted) to the EPC store so a Property
holds a lodged EPC and a predicted one (EPC Prediction gap-fill) at once
(ADR-0031). EpcRepository.save gains source="lodged"; idempotent delete is now
per-source (a predicted save no longer wipes lodged, and vice versa);
get_for_property/get_for_properties filter lodged; new get_predicted_for_property
/ get_predicted_for_properties read predicted. PropertyPostgresRepository.get +
get_many hydrate Property.predicted_epc, so the predicted picture reaches the
modelling read (both load via get_many). FakeEpcRepo mirrors the dual slot.
EpcPropertyModel gains `source` (default "lodged"); the test DB builds from the
SQLModel mirror so this is exercised without the prod migration. The matching
Drizzle change (column + per-(property_id,source) uniqueness) is the team's to
action before merge — docs/MIGRATION_NOTE_predicted_epc_source.md.
3 store tests (coexist, idempotent predicted re-save leaves lodged, lodged-only
has no predicted) + property-repo wiring; 85 pass across affected suites; new
code pyright-clean (2 pre-existing wwhrs errors in epc_property_table untouched).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve the slice-5 design tree (grill-with-docs): estimation runs in Ingestion
(refines ADR-0029 dec-3; drops the #1227 "shift to Modelling" — no surviving
rationale, and stages communicate only via persisted state); predicted EPC is
persisted in a DISTINCT slot (EPC table + source discriminator) so lodged +
predicted coexist (enables EPC Anomaly Flags); provenance is structural (the
slot), not a field on EpcPropertyData; effective_epc/source_path gain a
"predicted" branch; slice-5 is gap-fill only; property_type is a REQUIRED input
(hard cohort filter) from Landlord Overrides, and Properties with unknown type
are gated out (no national defaults). OS postcode_search as a broader type
source is a noted follow-on. CONTEXT EPC Prediction entry gains the gating rule.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Tier-2 (full national bulk streaming) is deferred. The near-term scale
validation is a Tier-1.5: a few-thousand-cert anonymised corpus stored in
S3 (too large to commit, far more stable than the 36-target gate fixture),
pulled to a temp dir and run through the same load_corpus +
evaluate_component_accuracy. Reuses the committed-fixture machinery wholesale
— only the data source differs. One scorer, three data sources (committed
fixture / S3 corpus / bulk stream).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Records the grilling-session decisions amending ADR-0029's validation:
- Source cohort keeps all cert vintages (components are agnostic of the SAP
methodology that rated them); only the held-out validation TARGET is
restricted to SAP 10.2. Amends ADR-0029 decision 5 ("pre-SAP10 dropped").
- Component Accuracy (predicted vs API actual components) is the primary,
calculator-independent signal. calc(predicted) vs calc(actual) rejected
(circular ground truth, hides calculator error); neighbour-mean-lodged-SAP
baseline rejected (mixes SAP versions). calc(predicted) vs API-lodged
SAP/carbon/PE kept as a secondary, calculator-floored guard.
- Two tiers: committed anonymized fixture (ratcheting CI gate) + bulk-export
national battle-test on harness/epc_bulk.py + harness/cohort.py, emitting
accuracy + a failure taxonomy, re-baselining the gate floors.
CONTEXT.md: Comparable Properties corrected to all-vintage source; new
Component Accuracy term. ADR-0029 Validation section marked superseded.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Records the four load-bearing design decisions from the grill-with-docs session:
standalone co-selectable rec; eligibility = lodged-only (no effectiveness gate,
electric-storage §A.2.2 no-op is the Optimiser's call); dedicated clearing
SecondaryHeatingOverlay; flat per-dwelling cost (a lodged secondary is fixed per
RdSAP, so a real decommission job, not room-scaled).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Documents the inherit-and-validate decision for 18.0: reuse 20.0.0's 0.148 +
band multipliers (the corpus can't self-fit — 958/1000 band-1 with no measured
band-1 windows), validated against 18.0's own band-4 rich certs (0.223 obs vs
0.148 x 1.51 pred). References ADR-0027 one-way (keeps the accepted ADR immutable).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the flat placeholder scalars (boiler £3000; tune-up £500/£900) with a
per-dwelling composite cost, mirroring the ASHP architecture (ADR-0025): a
`HeatingRates` table (data, `heating_rates.json`), typed `BoilerCostInputs` /
`TuneUpCostInputs`, pure `Products.boiler_bundle_cost` / `tune_up_cost`, and
modelling-layer interpreters that read the dwelling into those inputs.
The cost mirrors the Simulation Overlay component-for-component, sharing the
controls + cylinder pricing across both options:
- tune-up (standard) = standard controls + cylinder fixes
- tune-up (zone) = zone controls + cylinder fixes
- boiler upgrade = £3200 all-in + standard controls (only when the upgrade
fired a controls change) + cylinder fixes
Standard controls are priced INCREMENTALLY — only the parts missing to reach
SAP 2106 (programmer £120 / room thermostat £150 / TRV £35×radiators), read
from a Table 4e Group-1 feature map so a dwelling that already has a room
thermostat + TRVs is only charged the programmer. Zone controls are a full
smart kit (hub £205 + smart TRV £50×radiators) — the smart TRV is itself the
room sensor, so there is no separate per-room sensor line. Cylinder fixes:
jacket £50 (when under-insulated) + thermostat £150 (when absent). The boiler
is a like-for-like wet swap (no radiators/flue/pipework — eligibility already
requires an existing wet boiler), so those dead-code extras are not modelled.
Figures are research-validated 2025/26 UK installed costs (legacy Costs.py
lineage); fully-loaded totals with one contingency on top (Model B, not the
legacy VAT/preliminaries engine). Contingency: boiler 0.26; tune-ups 0.10
(was a 0.15 placeholder). ADR-0027 records the design; CONTEXT.md's Heating
Eligibility entry updated to cover the partial boiler/tune-up family + composed
cost. Products cost pins (delta<=1e-9) + interpreter tests + generator
composite-cost assertions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sharpen the glossary to decouple deterministic old-schema re-mapping from
neighbour-prediction gap-fill (a separate, unimplemented ML path), and add
the Reduced-Field Synthesis term. ADR-0027 records the pre-SAP10 20.0.0
mapper's best-attempt synthesis (corpus-fit glazing 0.148xTFAxband, 4-way
orientation) and its trade-offs. Grill resume doc captures every resolved branch.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Session 9 ran five independent data-driven audits (profiler, dropped-field scan,
CO2/PE reconciliation, cross-provider LIG parity, HW-demand reconciliation) — all
converged on diffuse remaining gap — and shipped glazing Table-24 (+16 certs) +
HW-only heat-network DLF, taking 54.90% -> 56.8% within-0.5. The data-driven seam
is exhausted; session 10 switches to worksheet-level ground truth via the
summary-report-based per-cert audit. New agent prompt at HANDOVER_SUMMARY_AUDIT.md
with method, starter candidate certs, ruled-out list, and conventions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Record the 4-agent audit, the shipped Tier-1 fuel-code strict-raise (7878a969),
and the un-actioned Tier 2/3 candidates (glazing codes 4-12, window orientation,
age-band swallows) for a future robustness slice.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Profile-driven re-sweep at HEAD da094feb. Every biased/error-carrying bucket
chased field-by-field resolved to proxy / already-deproven / already-fixed:
roof code 5/8 (same u_roof as code 4), per-bp age mapping (correct),
'(same dwelling above)' roofs (5 certs, 4 fine), index-less MEV gas
(centred by e6dda705 to signed +0.09), wit=4 cavity -0.25 (tail-driven),
community whc=903 HW (= deproven meter_type=3). The +32 outlier 2958 is
per-cert (twin 3420 is +0.18). Residual is a broad per-cert fabric+HW tail.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Records the fuel-type-boundary canonicalisation, the goldens-caught
constraint (code 33 is also the electricity-10h tariff code), and the
deferred dual-fuel/community/fabric follow-ups.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds the flat-roof slice (52.1 → 53.1%) and records the unifying principle
("unknown insulation → as-built age default, not uninsulated") plus the
cross-element review confirming all element types now conform.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds the cavity wall-U slice to the SESSION-5 block + headline table;
records the by-age-band re-split method that surfaced the G/H spike.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
47.6% → 48.6% within 0.5; immersion code mapping corrected (1=dual,
2=single); next robust leads are under-rating flat/party-fabric scatter.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Updates the headline (45.1 → 47.6%), records the four shipped fixes + the
roof-8 false-lead closure, documents the two methods that worked
(description-vs-code audit + outlier-robust categorical sweep by net skew +
median), and lists the open robust leads (whc=903 immersion HW, cat-7 storage,
dual immersion) with the scatter buckets to avoid.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pulls in 42 commits of calculator/mapper accuracy fixes from the per-cert
mapper-validation and floor/roof/heating fronts.
Conflict resolutions:
- mapper `_is_elmhurst_roof_window`: main dropped the branch's "wall location →
vertical" guard (it broke cert 000516's rooflight), but that re-broke cert
001431's two External-wall U>3.0 windows (which must stay vertical). The two
certs lodge a BYTE-IDENTICAL §11 row, so neither location nor U separates
them — the real discriminator is the room-in-roof context. Replaced the
unconditional U>3.0 backstop with one gated on the BP having a room-in-roof
(`_elmhurst_bp_has_room_in_roof`): 000516's Main BP has a "Room in roof type
1" (→ rooflight), 001431's does not (→ vertical). Validated against BOTH —
full Elmhurst worksheet suite 1038 pass + the 001431 window-extraction pin.
- property_postgres_repository: kept main's `ids_by_uprn` method + the branch's
`_restrictions_of` helper.
- sap_fuel.py: the branch relocated it to domain/billing/ (already carrying
main's to_table_32_code normalization), so kept the old path deleted.
Fallout from main's fabric fixes (validated by the boiler-3 real-cert pin which
still reproduces at delta 0):
- re-pinned the boiler-1 + boiler-instant-hw ASHP snapshot scores;
- main's §14.2 gas-boiler main-fuel derivation resolved the BGB/102 baseline
gap, so `test_gas_boiler_instant_hw_before_baselines` is now a passing test
(was an xfail tripwire).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Records the session-4 audit: walls/heating/controls clean; roof_construction=8
("Pitched, insulated", no measured thickness) computing U=2.30 is CORRECT, not a
bug — confirmed by user worksheet sim-case-29 (band C → Elmhurst SAP 55 ≡ our
56.75; lodged 80 is data-fidelity artifact). Lesson: "insulated (assumed)" = the
age-band default insulation level, not "well insulated". DO NOT re-chase roof-8.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Designed via grill-with-docs. API-first (Google Solar -> typed SolarPotential):
one Solar PV Recommendation, up to 5 conservatively-sized configs x battery
on/off; new PV overlay surface (per-roof-segment arrays, diverter conditional on
cylinder, export-capable ensured); overshading generation-calibrated from the
API's expected generation; eligibility offers conservation areas (only listed/
heritage block); composite costing from EA rates (contingency 0.15). CONTEXT
gains Solar Potential, Solar PV Recommendation, Solar PV Eligibility.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Code 3 = "(other premises below)" = above partially heated space (§3.12 →
U=0.7), confirmed 9/9 on single-BP certs (the diagnostic that dodged the
lossy-floors[] contamination). Records the 7536 re-pin and the lesson that
"irreducible residual" golden notes can mask a real mapper bug.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Record that profiler lead #1 (floor_codes=3) is not a clean single cause
(bimodal + confounded), that the paired worksheet certs confirm only
codes 1/6/7 (code 3 unmapped → needs a worksheet for 0380-2087), and that
immersion_type=2 / main_control=2107 / roof_codes=1 are scatter, not
dispatch bugs. The exposed-floor-on-flats fix (§3.12) shipped at b40e0f67.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Slice 7 of ADR-0025 costing: the modelling-layer interpretation half of the
split. ashp_cost_inputs derives existing system (mains_gas/fuel/SAP-code),
size band (floor area <= 75 m2), design heat loss (floor_area x 0.05 -- the
chosen proxy over HLC, ADR updated), radiator count (habitable + 3, floor-area
fallback) and reusable-wet-system flag. Catalogue math (Products) stays
EPC-free. ADR-0025 updated to record the floor-area pump-sizing choice.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Designs the heating Recommendation Generator via /grill-with-docs. Load-bearing
decisions: HW + controls + fuel + meter fold into each competing whole-system
bundle (the legacy heating-vs-HW split double-counted); each bundle is a fixed,
real, contractor-installable end-state (ASHP via PCDB index, HHR storage via
sap_main_heating_code=409), Product stays cost-only; eligibility encodes only
physical/planning installability since the Optimiser owns the economics (the
legacy ASHP built-form / 120 m² rule is dropped — research found no
authoritative basis); the Simulation Overlay is the deepest surface yet,
spanning main_heating_details[0] + sap_heating + top-level EpcPropertyData +
sap_energy_source. Build order HHRSH -> ASHP -> boiler (deferred).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolved in a grill-with-docs pass. recommend_lighting converts ALL non-LED
bulbs (incandescent + CFL + low-energy-unknown) to LED — all the way to LED, not
the legacy "fill to low energy", because SAP §12-1 rates LED efficacy (100)
above LEL (80) / CFL (55). A free Optimiser candidate (it improves SAP), unlike
ventilation's forced dependency. Its overlay is the first whole-dwelling,
top-level surface: a LightingOverlay carrying the four bulb-count fields by
their exact EPC names, folded directly onto EpcPropertyData (led=total, others
0). Priced per-bulb x non-LED count, contingency 0.26, measure_type
low_energy_lighting (MEASURE_MAP-aligned; "LED" in the description). Validation:
real before/after cascade pins (zero-existing-LEDs + some-existing-LEDs) at 1e-4,
clean (no fabric coupling). Ground-truth confirmed: 20 incandescent -> 20 LED
drops lighting (232) 783.7 -> 232.7 kWh/yr.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
PR feedback (dancafc): the SQLModel column was Optional[str], but the
domain `SapBuildingPart.wall_insulation_thickness` is Optional[Union[str,
int]] — `_api_resolve_wall_insulation_thickness` returns an int mm when the
API lodges `wall_insulation_thickness == "measured"` (SAP 10.2 §5.7 /
Table 8). The plain str column round-trips that int back as the string
"100", corrupting the Table 8 insulated-wall U-value lookup.
This column was missed in the round-trip-fidelity §1 JSONB sweep
(#1129) — its `Union[str, int]` sibling `roof_insulation_thickness` was
converted, but `wall_insulation_thickness` was not, and no 21.0.0/21.0.1
fixture lodges "measured" so the gap stayed latent. Convert to JSONB
(matching `roof_insulation_thickness` / `flat_roof_insulation_thickness`),
align the column type to Optional[Union[str, int]] (also removes a pyright
type-mismatch), record it in the migration doc §1, and add a round-trip
guard test asserting an int survives as an int (fails as '100' == 100 on
the old str column).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Grill-with-docs outcome for the glazing generator. Single planning-picked
Measure (double when unrestricted, secondary for conservation/listed/heritage),
upgrading all single-glazed windows together. The overlay writes lodged U-value
+ solar-g directly into WindowTransmissionDetails (our calculator consumes those
as inputs — it does NOT derive them from glazing_type, unlike Elmhurst) plus
glazing_type for the §5 daylight factor; EpcSimulation gains a per-window
`windows` surface. Priced flat average-per-window × single-glazed count.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Grill-with-docs outcome for the roof-insulation generator. One dispatching
recommend_roof_insulation, one Measure per roof by type (loft 300mm incl.
thatch / sloping-ceiling 100mm / flat-roof 200mm; no-access → none),
MAIN-only, room-in-roof deferred. Detection keys on the roof_construction_type
string (populated on both paths; the calculator already dispatches on it) with
sloping→flat→no-access→loft ordering; the roof_construction-int cross-mapper
parity is the follow-up Hestia-Homes/Model#1178. Thatch is not excluded — it
takes loft insulation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
