Grill-with-docs outcome for the roof-insulation generator. One dispatching
recommend_roof_insulation, one Measure per roof by type (loft 300mm incl.
thatch / sloping-ceiling 100mm / flat-roof 200mm; no-access → none),
MAIN-only, room-in-roof deferred. Detection keys on the roof_construction_type
string (populated on both paths; the calculator already dispatches on it) with
sloping→flat→no-access→loft ordering; the roof_construction-int cross-mapper
parity is the follow-up Hestia-Homes/Model#1178. Thatch is not excluded — it
takes loft insulation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reference the calculator-side wall_construction=6 disambiguation issue from the
strict-xfail tripwire and ADR-0019, so the blocker is traceable both ways.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pin the resolution reached in the grill: planning status persists as a
per-UPRN write-through cache in the existing `property_details_spatial` table
(not FE-property columns), read back off the Property in Modelling; unknown
UPRN defaults to unrestricted, matching legacy `empty_spatial_df` (superseding
the earlier "conservative stance" note).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
System-built (precast/no-fines concrete) takes both solid-wall Options like
solid brick (ADR-0019), keyed on `wall_construction == 6` (WALL_SYSTEM_BUILT,
Elmhurst `SY`). A basement-suitability guard (`main_wall_is_basement`) is added
since a below-ground basement wall is never EWI/IWI-suitable.
This is currently inert: `B Basement wall` also maps to 6 (mapper.py:2100) and
`main_wall_is_basement` is derived as `wall_construction == 6`, so every code-6
wall reads as basement and is guarded out — the live cohort is unchanged. The
system-built EWI/IWI cascade pin is committed as a strict-xfail tripwire that
flips green the moment the calculator disambiguates system-built from basement
(MAIN wall_construction==6 with main_wall_is_basement False). `wall_construction
== 8` is Park home, not system-built — not keyed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Slice 2a. New recommend_solid_wall emits one Main-wall Recommendation carrying
External + Internal wall-insulation Options for an uninsulated (wall_insulation
_type=4) solid-brick (wall_construction=3) main wall, each priced at the heat-
loss wall area. Cascade pin: the generator's EWI and IWI Options reproduce
their respective re-lodged afters at abs(diff) <= 1e-4.
Detection keys on wall_construction code, not description (ADR-0019 note
corrected): the Elmhurst ingestion path leaves walls[].description empty, so
the code is the only cross-path signal; codes 1-5 are consistent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Captures the grill-with-docs session for solid-wall insulation: CONTEXT.md
gains EWI/IWI Measure Types + the Wall Insulation Eligibility rule (+ a
flagged-ambiguity that the three planning flags stay distinct, never recollapsed
to legacy restricted_measures). ADR-0019 records the eligibility policy (cavity
-> cavity only; brick/system -> IWI+EWI; timber -> IWI only; cob/stone -> none;
conservation/flat block EWI, listed/heritage block both). ADR-0020 records
conservation/listed/heritage as three distinct Property attributes sourced by
extending the geospatial S3 repo (flags co-located with lat/long).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Next-phase handover: fetch live EPCs via EpcClientService, run the
offline Modelling harness, and save a per-property report covering
(1) lodged-vs-calculated SAP divergence (>0.5), (2) plans + costings,
(3) recommended measures + the EPC attributes that triggered them. Maps
the EPC API client (the user's blocker), the calculator-error ingredients
(parity_report scaffolding), and each generator's exact trigger fields.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Expand half of the recommendation_materials retirement (ADR-0017). A
Plan Measure installs a single Product, so thread its catalogue id end to
end — Product.id -> MeasureOption.material_id -> PlanMeasure.material_id
-> recommendation.material_id — replacing the per-material BOM child
table with one nullable column on the row. ProductPostgresRepository
reads the id from MaterialRow; the four fabric generators set it on their
Option; the orchestrator carries it onto the Plan Measure; the mirror
declares + maps the column. Optional throughout (the JSON stopgap
catalogue carries no ids -> NULL).
The multi-measure integration test now pins each persisted measure's
material_id to its seeded MaterialRow id. Migration spec (live column
must be added before this deploys; contraction is the owner's next step)
in docs/migrations/recommendation-material-id.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
From the grill-with-docs pass on the depth+scale phase. Splits the
overloaded "valuation" into two glossary terms — Property Valuation
(current market value, a Baseline attribute, mostly missing) and
Valuation Uplift (plan-conditional, percentage-primary; absolute £ only
when a Property Valuation exists, 2x ROI cap on the £ form). ADR-0018
records the percentage-primary decision and why (the EPC scale corpus
has no market values, so a value-primary model produces nothing), plus
the deferred sourcing / per-measure / rental-yield items.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Capture the next phase (close persisted-field gaps + financial uplift, plus a
large-scale e2e run of a SAP 10.2 EPC dump and console manual testing; measure
coverage deferred) and a self-contained handover prompt for a fresh agent to
pick up via a grilling session.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rewrite the migration spec into the full expand/contract sequence (add plan_id
→ backfill → dual-write → cut reads → drop) with the two load-bearing rules:
backfill before any read cuts over, and dual-write the m2m until all reads are
off it (the Drizzle FE reads the tables directly, so the repos can't deploy
atomically). Amend ADR-0017 from "m2m retired for new writes" to "m2m dropped +
one SQLModel definition per table under infrastructure/postgres/modelling/".
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Records the /grill-with-docs design for the Modelling Bill-Derivation slice:
Bill Derivation is cross-stage (relocate Bill/EnergyBreakdown/BillDerivation/
sap_fuel to a neutral domain/billing/); Modelling bills the fully-overlaid
post-package SapResult (so fuel-switch measures price at the new fuel for free),
diffing against the baseline at the same FuelRates snapshot; the post-package
and baseline SapResults are captured from scores the optimiser/orchestrator
already compute (Score.sap_result), so no second calculate; FuelRatesRepository
is constructor-injected into ModellingOrchestrator mirroring Baseline; plan-level
columns this slice, per-measure telescoping bill cascade next (energy_savings is
vestigial, left NULL).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The warm-start (and max-gain fallback) now price each forced Measure Dependency
the candidate triggers, not just inject it afterwards: optimise/optimise_min_cost
fold dependencies into each candidate's cost+gain via _augmented_cost_gain, and
optimise_package scores each dependency's true role-1 signal (_with_role1_signals)
instead of the 0.0 placeholder. This stops the min-cost objective (i) ignoring the
~£900 a wall drags in (a wall-free package reaching target can be cheaper) and
(ii) picking a small-gain wall whose mandatory ventilation (down to -5 SAP) makes
it net-negative, which repair cannot un-pick.
Budget is now a hard envelope: the constraint applies to the augmented (measure +
its ventilation) cost, so a wall that fits alone but whose ventilation would bust
the budget is DROPPED rather than forced over budget. This reverses the earlier
'forced regardless of budget' call (which made sense when selection was
ventilation-blind). Safety invariant intact — presence still injected on every
path; we just never recommend a wall we can't afford to ventilate. ADR-0016
amendment updated. 94 modelling+orchestration tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The original ADR-0016 mis-specified the warm-start objective as maximise-gain-
subject-to-budget (with the target a repair floor); the rebuild faithfully
implemented that wrong objective. The intended behaviour is the legacy
StrategicOptimiser Case 1: minimise cost subject to (true) SAP gain >= target and
cost <= budget, falling back to max-gain-within-budget only when the target is
unreachable. For Increasing EPC this is least-cost-to-target: cheapest package
reaching the band, stops at the target (no overshoot into a higher band), surplus
budget unspent.
Also records: target predicate sap_continuous >= band floor (conservative, no
legacy slack — re-score+repair supersede it); ventilation-aware selection (the
forced dependency, -1 to -5 SAP, is folded into candidate evaluation with a real
negative role-1 signal, not just injected afterwards); presence-vs-awareness
enforcement; warm-start+re-score+repair structure and scalability rationale kept.
Sharpened the CONTEXT.md Optimised Package definition to match.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Brings HANDOVER_MODELLING.md fully current: #1157 (Plan persistence) and
#1160 (Optimiser) closed this session; records the locked design
decisions (multi-phase deferred, Plan Measure term, reuse-live-tables
via SQLModel mirrors, pure-Python knapsack not mip), the gotchas (mip/CBC
broken on aarch64, moto missing, drive-Modelling-directly for fixtures
without lodged perf, seed materials per fired measure type), and the
remaining work (#1161 ventilation Measure Dependency + deferred fronts).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Outcome of the /grill-with-docs session scoping #1157.
- CONTEXT.md: add **Plan Measure** (the persisted selected Option +
role-3 attribution + cost); Recommendation stays the candidate.
Remove Scenario Phase / Plan Phase / Rolled-over Options — multi-phase
is deferred. Reshape Scenario + Plan to single-phase; fix relationships,
dialogue, and the "phase" ambiguity note.
- ADR-0005: rewritten to Deferred (multi-phase was speculative
prospective-client work; single-phase now; future plan_phase back-fill
path preserved). Stray phase refs cleaned in ADR-0016 / ADR-0009.
- ADR-0017 (new): Plan persistence — reuse the live plan/recommendation
tables via SQLModel mirrors + a PlanRepository on the UoW; add
recommendation.plan_id, retire the plan_recommendations m2m; flat
post-retrofit on plan; idempotent replace; CO2 in tonnes. Unselected
alternatives + bills noted as deferred directions.
- docs/migrations/recommendation-plan-id.md: the FE-owned Drizzle change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Records that the Elmhurst recommendation Summaries parse via the
extractor chain (not parse_site_notes_pdf), so the "parser gate" never
blocked the cascade pins. All four pins close at delta 0; loft 270→300
and the suspended-floor insulation-type field were the two gaps fixed.
Remaining: #1157 (HITL schema review) + ProductJsonRepository.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Captures issue status (#1153-#1161), the built compute spine, key
facts/gotchas (hand-built 000490 fixture, calculator entry, worktree-vs-main
import trap, test/commit conventions), and the two gates (parser fix -> wire
Elmhurst cascade pins; #1157 persist-Plan HITL schema review). For picking
the work back up in a fresh session.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reframe Recommendation as a target surface (partitions the EpcPropertyData
surface, so selected overlays never collide); add Measure Option,
Simulation Overlay (EpcSimulation), Product, Cost, Contingency, and
Measure Dependency. ADR-0016 fixes the scoring/optimisation approach
(warm-start grouped-knapsack MILP -> deterministic package re-score ->
greedy repair, with a final-package marginal cascade for display
attribution), resolving the open question in ADR-0005 §14.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The PropertyBaselineOrchestrator now reads the current Fuel Rates snapshot
once per batch, builds a BillDerivation, and prices each scored property's
SapResult -> EnergyBreakdown into a Bill carried on PropertyBaselinePerformance
(None only on the stub no-calculator path). The Bill is flattened onto nullable
bill_* flat columns (per-section kwh+cost, standing charges, SEG credit, total)
on the postgres table, with bill_total_annual_bill_gbp as the not-null
discriminator on read-back. Section absent from the bill stays None, not 0.
Updated all four orchestrator construction sites to inject the FuelRatesRepository
port (handler + three test sites), and the FE migration doc to reflect the
prefixed columns and that they are now populated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The SapResult -> EnergyBreakdown adapter (ADR-0014), a classmethod on the
target mirroring Performance.from_sap_result. Folds each positive per-end-use
delivered kWh into a billable EnergyLine: main/main-2/secondary heating and
hot water at their resolved fuel (sap_code_to_fuel); lighting/pumps-fans/
appliances/cooking/cooling as electricity. PV export carries to exported_kwh
for the SEG credit. Zero-kWh end uses emit no line; a positive kWh with no
fuel code raises rather than billing at a default (strict, mirrors the
calculator).
Adds BillSection.COOLING (electricity, from space_cooling_fuel_kwh_per_yr).
BillDerivation already prices any section it is given, so no change there.
Also corrects the ADR-0014 amendment: SapResult carries the calculator's own
fuel codes (raw API or Table-32 per mapper, ADR-0015); sap_fuel normalizes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Captures a /grill-with-docs session resolving how BillDerivation gets the
fuel each end use burns, and what Rebaselining actually is.
- ADR-0014 amendment: per-end-use fuel is a calculator OUTPUT (resolved
Table-32 codes on SapResult: main-1/main-2/secondary/HW + pv_exported_kwh);
the adapter is a pure SapResult->EnergyBreakdown map. Corrects stale §3
(is_gas_code... -> sap_fuel.sap_code_to_fuel). Adds COOLING section.
Interim, pending ADR-0015.
- ADR-0013 amendment: the calculator is the SCORING ENGINE within
Rebaselining (assemble the Effective EPC picture, then score), not the
whole of it; the Rebaseliner exposes its SapResult so the orchestrator
composes Effective Performance AND the Bill from one scoring.
- ADR-0015 (new): mappers own cert normalization; EpcPropertyData becomes a
strict type. Explains why fuel resolution sits in the calculator today.
- CONTEXT.md: Effective EPC = the assembled picture; Rebaselining = assemble
(overrides / neighbour-estimation / old-schema remap) then score.
- EpcPropertyData docstring points at ADR-0015.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Slice 5b: update the FE-owned migration spec so the other repo can create the
bill columns in parallel.
- Bill block: per-section delivered kWh + cost (heating, hot water, lighting,
appliances, cooking, pumps/fans, cooling) + standing_charges_gbp,
seg_credit_gbp, total_annual_bill_gbp, fuel_rates_period.
- space_heating_kwh / water_heating_kwh (RHI recorded demand) marked SUPERSEDED
by heating_kwh / hot_water_kwh (calculator delivered fuel); kept until the bill
populates, then dropped.
- Cooling section kept (mostly 0 but affects the bill, cheap to store).
- Records the calculator-load-bearing posture (effective_* may differ from
lodged_* for pre-10.2) and that columns are defined now / populated when the
SapResult->EnergyBreakdown adapter + BillDerivation wiring land.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Slice 3 of Bill Derivation. sap_code_to_fuel(code) maps a SAP 10.2 / Table 32
fuel code to the canonical billing Fuel — bounded to the ~47 Table 32 codes (the
carrier, orthogonal to the PCDB product index, so all PCDB heat pumps share one
electricity code). Mains gas / LPG / oil+bioliquids / coal / smokeless / wood /
electricity (standard + off-peak) / heat-network groupings; an unmapped code
(dual fuel, grid-export) raises UnmappedSapCode rather than guessing.
Also: ADR-0014 deferred/TODO section records the stubbed appliances+cooking
(pending the SapResult fields), the off-peak day/night split, the heat-network
rate gap, and regional rates / ETL.
The SapResult -> EnergyBreakdown adapter (next slice) is gated on the
appliances/cooking fields landing on SapResult.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pin the bills design from a /grill-with-docs session:
- ADR-0014: whole-home annual bill from SAP10 Calculation's delivered kWh per
end use, re-priced at real Fuel Rates (NOT the calculator's SAP-notional
total_fuel_cost_gbp, which is RdSAP Table 32 standardised prices ~half real
electricity). Fuel enum + FuelRates + FuelRatesRepository static snapshot;
per-section + total flat columns; raise on unpriced fuel (house coal /
heat network are the named gaps).
- ADR-0013 amendment: the shadow stepping-stone is collapsed — the calculator
is load-bearing now. effective=calculated for sap_version<10.2 (StubRebaseliner
floor 10.0->10.2); >=10.2 keeps lodged + logs divergence; a strict-raise
aborts the batch (load-bearing for bills regardless of version).
- CONTEXT: EPC Energy Derivation -> Bill Derivation (no "service" suffix);
Baseline Performance energy block = per-end-use kWh + per-section bill + total;
Fuel Rates = committed static snapshot; Rebaselining trigger threshold 10.2.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Wire Sap10Calculator into PropertyBaselineOrchestrator as a non-load-bearing
shadow runner. For each property it scores the Effective EPC beside the
load-bearing Lodged/Effective write, catches any strict-raise -> log.error
(never aborts the batch), and on success log.warning's divergence from Lodged:
SAP |continuous - lodged| > 0.5; PEUI/CO2 > 1% relative (CO2 after kg->tonnes).
Every line is tagged with sap_version so SAP-10.2 signal separates from
older-spec drift (ADR-0010 Validation Cohort).
Per ADR-0013, Calculated SAP10 Performance is not a persisted third value-set:
effective = calculated in every baselining scenario, so the calculator IS the
mechanism that produces Effective Performance (the Rebaseliner). It runs in
shadow only while being hardened; when overrides/estimation land it is promoted
to drive Effective and the failure posture flips to abort (ADR-0012, calculator
now load-bearing). No table change.
- ADR-0013 + CONTEXT (Calculated SAP10 Performance / Effective Performance /
Rebaselining) record the decision.
- CalculatorShadow port + LoggingCalculatorShadow + Calculator protocol.
- FakeCalculatorShadow for orchestrator unit tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Factual staleness fix flagged in the handover; the calculator lives in
domain/sap10_calculator/calculator.py. Glossary term 'Baseline Performance'
deliberately left unchanged (concept vs PropertyBaselinePerformance class).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make the stored units explicit on the property_baseline_performance columns:
- `*_co2_emissions` → `*_co2_emissions_t_per_yr` (tonnes CO₂/yr, whole dwelling)
- `*_primary_energy_intensity` → `*_primary_energy_intensity_kwh_per_m2_yr`
Column names only; the domain `Performance` VO stays unit-suffix-free (units are
a storage concern, mapped in from_domain/to_domain). Migration doc updated.
Round-trip stays green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replaces the handler's whole-pipeline Session (one transaction across all
three stages, connection pinned during Ingestion's external IO) with a
Unit-of-Work per stage (ADR-0012, added here). Each stage runs its batch in
one unit and commits once; any property raising aborts the batch and the
subtask fails noisily.
- BaselineOrchestrator(unit_of_work, rebaseliner): one unit for the batch,
commit once. Raise on a pre-SAP10 property leaves the unit uncommitted.
- IngestionOrchestrator(unit_of_work, epc_fetcher, geospatial_repo,
solar_fetcher): fetch/write split — phase 1 fetches the whole batch (EPC /
coords / solar) with NO unit open; phase 2 writes in one unit and commits.
The connection is never held during external IO. Geospatial S3 repo stays
injected (reference data, not transactional).
- Handler: module-scoped engine (pool reused across warm invocations) + a UoW
factory; whole-pipeline `with Session` gone. `build_first_run_pipeline`
composes on the factory. Source clients still behind the raising seam.
- ADR-0012 records the decision (per-stage boundary, all-or-nothing batch,
idempotent re-run, fetch/write split, module-scoped engine). Modelling stub
left untouched (no-op, no DB) per the ADR.
Tests: orchestrators on a shared FakeUnitOfWork (assert persisted batch +
exactly-once commit + no-commit-on-raise). New real-DB E2E integration test:
real PostgresUnitOfWork, Ingestion writes the EPC → Baseline reads it back
through the repo → re-run replaces, not duplicates (1 EPC row, 1 baseline row
after two runs). 121 pass in tests/; pyright strict clean; AAA.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Stage 2 of First Run. Establishes each Property's Baseline Performance
from persisted source data and writes it back — reads only from repos,
never a Fetcher or HTTP (ADR-0003), so it is byte-identical whether
Ingestion ran milliseconds ago or last week.
Domain (`domain/baseline/`):
- `Performance` VO — the four rated quantities: SAP / EPC Band / CO2 /
Primary Energy Intensity. `lodged_performance(epc)` reads them off the
EPC's recorded fields (PEUI = `energy_consumption_current`).
- `BaselinePerformance` (ADR-0004) — the paired `lodged` + `effective`
Performance + `rebaseline_reason`, plus the no-derivation part of the
energy block (`space_heating_kwh` / `water_heating_kwh`, off the RHI,
deterministic per ADR-0006). Both halves always populated.
- `Rebaseliner` port + `StubRebaseliner`: the re-score-on-override seam
(ADR-0011). SAP10 certs pass through (effective == lodged, reason
"none"); a pre-SAP10 cert raises `RebaselineNotImplemented` rather
than fabricating a plausible-but-wrong "none" — ML rebaselining is not
wired yet. Mirrors the repo's strict-raise culture.
Persistence: new `BaselineRepository` port + `BaselinePostgresRepository`
+ flat-column `baseline_performance` SQLModel (one row per Property). Per
ADR-0004's amendment this is a standalone table, NOT columns on the
retiring `property_details_epc`. Production migration is FE-owned
(Drizzle) — docs/migrations/baseline-performance-table.md.
Docs (grill-with-docs): corrected CONTEXT.md Lodged/Effective Performance
to Primary Energy Intensity (the term collided with its own _Avoid_ entry
under "heat demand") + fixed stale RHI field names; amended ADR-0004
Consequences for the standalone-table decision.
Fuel split + bills (rest of EPC Energy Derivation) deferred to a
follow-up — they need a Fuel Rates source (Ofgem-cap ETL) that does not
exist yet.
TDD, one test -> one impl: 7 tests (lodged read, rebaseliner pass-through
+ raise, orchestrator establish-and-persist + pre-SAP10 raise, Postgres
round-trip + absent). pyright strict clean; AAA layout.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add epc_renewable_heat_incentive table (space_heating_kwh, water_heating_kwh +
the three insulation-impact kWh fields), wired into EpcPostgresRepository
save/get. This is the P0 gap: RenewableHeatIncentive carries the baseline
space-heating/hot-water kWh that EPC Energy Derivation consumes.
The round-trip test now asserts full deep-equality (dropped the
renewable_heat_incentive exclusion) and passes for RdSAP 21.0.0 + 21.0.1.
DB migration for the new table documented in
docs/migrations/epc-property-round-trip-fidelity.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Relocate EpcPropertyModel + child tables from the dying backend/ tree to
infrastructure/postgres/epc_property_table.py (re-export shim keeps
documents_parser working). Add EpcRepository port + EpcPostgresRepository with
a full reverse mapper (epc_property tables -> EpcPropertyData).
Round-trip test surfaced two fidelity gaps:
1. Union[int,str] SAP code fields were str()-coerced on save, losing the int
(API) vs str (Site Notes) distinction. Now stored as JSONB (type-preserving).
2. The schema was a partial projection. Closed the cheap gaps on the model
(heating shower/bath counts, roof_construction_type, curtain_wall_age,
addendum, mechanical_vent_duct_insulation_level, SAP 10.2 §2 ventilation
fields + a ventilation_present flag). Structural gaps tracked as follow-ups;
renewable_heat_incentive (P0, #1137) excluded from the assertion until landed.
Round-trip passes for RdSAP-Schema-21.0.0 and 21.0.1; pyright strict clean.
Migration inventory for the DB: docs/migrations/epc-property-round-trip-fidelity.md
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Records the grill-with-docs outcomes for the ara_first_run rebuild: three
composable stage orchestrators (Ingestion/Baseline/Modelling), one lambda per
use case chaining them through repos (not in-memory), and the Fetcher-vs-Repo
data-source taxonomy. Amends ADR-0003's chaining rule to generalise beyond
RefreshOrchestrator. Adds the pipeline-composition + First Run vocabulary to
CONTEXT.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Orientation for the next chat picking up the two open fronts after the
ara_first_run rebuild shipped:
- where things stand (merged to main via per-cert; branch/worktree layout;
PRs into per-cert), authoritative ADRs/CONTEXT to read,
- current architecture + key files (post baseline→property_baseline /
FirstRun→AraFirstRun rename),
- conventions + gotchas (TDD, ephemeral PG, FakeUnitOfWork, pyright noise to
ignore, gh-credential push workaround),
- Task 1: wire Sap10Calculator into PropertyBaselineOrchestrator (Calculated
SAP10 Performance as a third value-set; failure-posture decision),
- Task 2: Modelling (stubs to build out; MaterialsRepository naming open;
needs a UoW when writing Plans),
- the raising/no-op seams not to mistake for done,
- known doc drift flagged (CONTEXT term vs PropertyBaselinePerformance class;
stale domain/sap/ path → domain/sap10_calculator).
Also banners ara_backend_design.md as superseded (architecture) by ADR-0011/0012.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make the stored units explicit on the property_baseline_performance columns:
- `*_co2_emissions` → `*_co2_emissions_t_per_yr` (tonnes CO₂/yr, whole dwelling)
- `*_primary_energy_intensity` → `*_primary_energy_intensity_kwh_per_m2_yr`
Column names only; the domain `Performance` VO stays unit-suffix-free (units are
a storage concern, mapped in from_domain/to_domain). Migration doc updated.
Round-trip stays green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replaces the handler's whole-pipeline Session (one transaction across all
three stages, connection pinned during Ingestion's external IO) with a
Unit-of-Work per stage (ADR-0012, added here). Each stage runs its batch in
one unit and commits once; any property raising aborts the batch and the
subtask fails noisily.
- BaselineOrchestrator(unit_of_work, rebaseliner): one unit for the batch,
commit once. Raise on a pre-SAP10 property leaves the unit uncommitted.
- IngestionOrchestrator(unit_of_work, epc_fetcher, geospatial_repo,
solar_fetcher): fetch/write split — phase 1 fetches the whole batch (EPC /
coords / solar) with NO unit open; phase 2 writes in one unit and commits.
The connection is never held during external IO. Geospatial S3 repo stays
injected (reference data, not transactional).
- Handler: module-scoped engine (pool reused across warm invocations) + a UoW
factory; whole-pipeline `with Session` gone. `build_first_run_pipeline`
composes on the factory. Source clients still behind the raising seam.
- ADR-0012 records the decision (per-stage boundary, all-or-nothing batch,
idempotent re-run, fetch/write split, module-scoped engine). Modelling stub
left untouched (no-op, no DB) per the ADR.
Tests: orchestrators on a shared FakeUnitOfWork (assert persisted batch +
exactly-once commit + no-commit-on-raise). New real-DB E2E integration test:
real PostgresUnitOfWork, Ingestion writes the EPC → Baseline reads it back
through the repo → re-run replaces, not duplicates (1 EPC row, 1 baseline row
after two runs). 121 pass in tests/; pyright strict clean; AAA.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Stage 2 of First Run. Establishes each Property's Baseline Performance
from persisted source data and writes it back — reads only from repos,
never a Fetcher or HTTP (ADR-0003), so it is byte-identical whether
Ingestion ran milliseconds ago or last week.
Domain (`domain/baseline/`):
- `Performance` VO — the four rated quantities: SAP / EPC Band / CO2 /
Primary Energy Intensity. `lodged_performance(epc)` reads them off the
EPC's recorded fields (PEUI = `energy_consumption_current`).
- `BaselinePerformance` (ADR-0004) — the paired `lodged` + `effective`
Performance + `rebaseline_reason`, plus the no-derivation part of the
energy block (`space_heating_kwh` / `water_heating_kwh`, off the RHI,
deterministic per ADR-0006). Both halves always populated.
- `Rebaseliner` port + `StubRebaseliner`: the re-score-on-override seam
(ADR-0011). SAP10 certs pass through (effective == lodged, reason
"none"); a pre-SAP10 cert raises `RebaselineNotImplemented` rather
than fabricating a plausible-but-wrong "none" — ML rebaselining is not
wired yet. Mirrors the repo's strict-raise culture.
Persistence: new `BaselineRepository` port + `BaselinePostgresRepository`
+ flat-column `baseline_performance` SQLModel (one row per Property). Per
ADR-0004's amendment this is a standalone table, NOT columns on the
retiring `property_details_epc`. Production migration is FE-owned
(Drizzle) — docs/migrations/baseline-performance-table.md.
Docs (grill-with-docs): corrected CONTEXT.md Lodged/Effective Performance
to Primary Energy Intensity (the term collided with its own _Avoid_ entry
under "heat demand") + fixed stale RHI field names; amended ADR-0004
Consequences for the standalone-table decision.
Fuel split + bills (rest of EPC Energy Derivation) deferred to a
follow-up — they need a Fuel Rates source (Ofgem-cap ETL) that does not
exist yet.
TDD, one test -> one impl: 7 tests (lodged read, rebaseliner pass-through
+ raise, orchestrator establish-and-persist + pre-SAP10 raise, Postgres
round-trip + absent). pyright strict clean; AAA layout.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add epc_renewable_heat_incentive table (space_heating_kwh, water_heating_kwh +
the three insulation-impact kWh fields), wired into EpcPostgresRepository
save/get. This is the P0 gap: RenewableHeatIncentive carries the baseline
space-heating/hot-water kWh that EPC Energy Derivation consumes.
The round-trip test now asserts full deep-equality (dropped the
renewable_heat_incentive exclusion) and passes for RdSAP 21.0.0 + 21.0.1.
DB migration for the new table documented in
docs/migrations/epc-property-round-trip-fidelity.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
