PR feedback (dancafc): the SQLModel column was Optional[str], but the
domain `SapBuildingPart.wall_insulation_thickness` is Optional[Union[str,
int]] — `_api_resolve_wall_insulation_thickness` returns an int mm when the
API lodges `wall_insulation_thickness == "measured"` (SAP 10.2 §5.7 /
Table 8). The plain str column round-trips that int back as the string
"100", corrupting the Table 8 insulated-wall U-value lookup.
This column was missed in the round-trip-fidelity §1 JSONB sweep
(#1129) — its `Union[str, int]` sibling `roof_insulation_thickness` was
converted, but `wall_insulation_thickness` was not, and no 21.0.0/21.0.1
fixture lodges "measured" so the gap stayed latent. Convert to JSONB
(matching `roof_insulation_thickness` / `flat_roof_insulation_thickness`),
align the column type to Optional[Union[str, int]] (also removes a pyright
type-mismatch), record it in the migration doc §1, and add a round-trip
guard test asserting an int survives as an int (fails as '100' == 100 on
the old str column).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Slice 1 of Bill Derivation — the reference-data foundation that later slices
price the calculator's per-end-use kWh against:
- Fuel enum (canonical billing fuels; the join key between the calculator's
SAP-code fuels and the rates snapshot). COAL + HEAT_NETWORK are members with
no national rate.
- FuelRates value object: unit_rate_p_per_kwh / standing_charge_p_per_day /
seg_export_p_per_kwh; raises UnpricedFuel on a fuel it has no rate for rather
than billing at a wrong default.
- FuelRatesRepository port (ADR-0011 Repo-reads-stored-reference-data) +
StaticFileFuelRatesRepository reading a committed JSON snapshot.
- Snapshot fuel_rates_2026_q2.json: GB national, Apr-Jun 2026 Ofgem cap
(gas/electricity) + DESNZ/NEP May 2026 (off-gas). Carries the full researched
data; the value object exposes single-rate fuels this slice. Off-peak
(day/night), house coal and heat network raise UnpricedFuel until later slices.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Final slice of ADR-0012: collapse the per-property read round-trips a batch
made (Baseline hydrated ~8 queries x 30 properties one at a time) into a
handful of per-table IN queries.
- EpcPostgresRepository: extracted a shared `_compose(rows)` from `get` (the
windows + floor-dim fetches are now passed in, not fetched inline), so both
`get` and the new `get_for_properties(property_ids)` build EpcPropertyData
from pre-fetched rows. `get_for_properties` fetches each child table once
(`WHERE epc_property_id IN ...`), groups in memory, and composes — load-whole
per ADR-0002.
- PropertyRepository.get_many(property_ids) -> Properties: one query for the
property rows + one bulk EPC hydration, composed in input order.
- BaselineOrchestrator / IngestionOrchestrator read the batch via get_many
instead of N x get.
- Ports + fakes gain the bulk methods.
The #1129 round-trip fidelity test stays green (the compose extraction is
behaviour-preserving). New tests: bulk hydration correctness + round-trips are
constant w.r.t. batch size (one-per-table, proven by query count). 123 pass;
pyright strict clean; AAA.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Re-runs of a First Run batch re-save a property's data; that must replace,
not duplicate (ADR-0012 idempotent batch writes).
- `EpcPostgresRepository.save` deletes the property's existing EPC graph
(parent + all child tables, floor-dims via their building parts) before
inserting, when a `property_id` is given. Anonymous saves still insert.
- `BaselinePostgresRepository.save` deletes the existing row for the
`property_id` before inserting — no more unique-constraint violation on
re-save; also what the re-score-on-override path needs.
- Solar already upserts, so it's unchanged.
The #1129 round-trip fidelity test stays green (delete-first is a no-op on
a first save). 2 new tests (re-save replaces, not duplicates). pyright
strict clean; AAA.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
First slice of the per-stage batch-transaction refactor (ADR-0012). A
UnitOfWork is the single transaction a stage runs its batch in: a context
manager exposing the DB repos bound to one session, committing once on
`commit()` and rolling back on exception or exit-without-commit
(all-or-nothing per batch, fail noisily).
- `UnitOfWork` (port): `property` / `epc` / `solar` / `baseline` repos +
`commit()` / `rollback()`; `__exit__` rolls back uncommitted work.
- `PostgresUnitOfWork(session_factory)`: opens a Session from an injected
factory (a module-scoped engine + sessionmaker in prod, so the pool is
reused across warm invocations), binds the Postgres repos to it, closes
on exit.
Not yet wired into any orchestrator — that lands in the Baseline /
Ingestion refactor slices. 3 tests against ephemeral PG (commit durable
across units; exception rolls back; no-commit persists nothing). pyright
strict clean; AAA.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Stage 2 of First Run. Establishes each Property's Baseline Performance
from persisted source data and writes it back — reads only from repos,
never a Fetcher or HTTP (ADR-0003), so it is byte-identical whether
Ingestion ran milliseconds ago or last week.
Domain (`domain/baseline/`):
- `Performance` VO — the four rated quantities: SAP / EPC Band / CO2 /
Primary Energy Intensity. `lodged_performance(epc)` reads them off the
EPC's recorded fields (PEUI = `energy_consumption_current`).
- `BaselinePerformance` (ADR-0004) — the paired `lodged` + `effective`
Performance + `rebaseline_reason`, plus the no-derivation part of the
energy block (`space_heating_kwh` / `water_heating_kwh`, off the RHI,
deterministic per ADR-0006). Both halves always populated.
- `Rebaseliner` port + `StubRebaseliner`: the re-score-on-override seam
(ADR-0011). SAP10 certs pass through (effective == lodged, reason
"none"); a pre-SAP10 cert raises `RebaselineNotImplemented` rather
than fabricating a plausible-but-wrong "none" — ML rebaselining is not
wired yet. Mirrors the repo's strict-raise culture.
Persistence: new `BaselineRepository` port + `BaselinePostgresRepository`
+ flat-column `baseline_performance` SQLModel (one row per Property). Per
ADR-0004's amendment this is a standalone table, NOT columns on the
retiring `property_details_epc`. Production migration is FE-owned
(Drizzle) — docs/migrations/baseline-performance-table.md.
Docs (grill-with-docs): corrected CONTEXT.md Lodged/Effective Performance
to Primary Energy Intensity (the term collided with its own _Avoid_ entry
under "heat demand") + fixed stale RHI field names; amended ADR-0004
Consequences for the standalone-table decision.
Fuel split + bills (rest of EPC Energy Derivation) deferred to a
follow-up — they need a Fuel Rates source (Ofgem-cap ETL) that does not
exist yet.
TDD, one test -> one impl: 7 tests (lodged read, rebaseliner pass-through
+ raise, orchestrator establish-and-persist + pre-SAP10 raise, Postgres
round-trip + absent). pyright strict clean; AAA layout.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add Coordinates value object + GeospatialRepository port + GeospatialS3Repository
adapter. Resolves a Property's lon/lat from the partitioned Ordnance Survey
Open-UPRN parquet (filename_meta -> partition -> UPRN row). A Repo, not a
Fetcher (ADR-0011): no live OS API call. The parquet reader is injected, so it's
unit-tested against fixture parquets with no S3/network; returns None when the
UPRN is uncovered or absent. pyright strict clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move the EpcClientService package (client + _retry + exceptions + tests) from
the dying backend/ tree to infrastructure/epc_client/ as the New-EPC-API Fetcher;
update the two callers (address2UPRN, a script). All 14 client tests pass.
Add SolarRepository port + SolarPostgresRepository persisting Google Solar
building insights as JSONB (solar_building_insights table), one row per Property.
The EPC repo half of this slice already landed in #1129. pyright strict clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add the Ara modelling aggregate root (ADR-0002): domain/property/ with
PropertyIdentity, SiteNotes, Property, Properties. Property.source_path
implements the two disjoint source paths + Recency Tie-Break (ADR-0001;
survey wins on an equal date); effective_epc resolves to the surveyed data
(Site Notes path) or the public EPC (epc_with_overlay path — Landlord
Overrides overlay is a later slice). Pure dataclasses, no infrastructure imports.
PropertyRepository port + PropertyPostgresRepository hydrate the aggregate
whole from a defensive view of the FE-owned 'property' table (identity columns)
plus the EPC slice via EpcRepository.get_for_property. Reads only from repos
(ADR-0003). 8 domain + 1 hydration test; pyright strict clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add epc_renewable_heat_incentive table (space_heating_kwh, water_heating_kwh +
the three insulation-impact kWh fields), wired into EpcPostgresRepository
save/get. This is the P0 gap: RenewableHeatIncentive carries the baseline
space-heating/hot-water kWh that EPC Energy Derivation consumes.
The round-trip test now asserts full deep-equality (dropped the
renewable_heat_incentive exclusion) and passes for RdSAP 21.0.0 + 21.0.1.
DB migration for the new table documented in
docs/migrations/epc-property-round-trip-fidelity.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Relocate EpcPropertyModel + child tables from the dying backend/ tree to
infrastructure/postgres/epc_property_table.py (re-export shim keeps
documents_parser working). Add EpcRepository port + EpcPostgresRepository with
a full reverse mapper (epc_property tables -> EpcPropertyData).
Round-trip test surfaced two fidelity gaps:
1. Union[int,str] SAP code fields were str()-coerced on save, losing the int
(API) vs str (Site Notes) distinction. Now stored as JSONB (type-preserving).
2. The schema was a partial projection. Closed the cheap gaps on the model
(heating shower/bath counts, roof_construction_type, curtain_wall_age,
addendum, mechanical_vent_duct_insulation_level, SAP 10.2 §2 ventilation
fields + a ventilation_present flag). Structural gaps tracked as follow-ups;
renewable_heat_incentive (P0, #1137) excluded from the assertion until landed.
Round-trip passes for RdSAP-Schema-21.0.0 and 21.0.1; pyright strict clean.
Migration inventory for the DB: docs/migrations/epc-property-round-trip-fidelity.md
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Final slice of ADR-0012: collapse the per-property read round-trips a batch
made (Baseline hydrated ~8 queries x 30 properties one at a time) into a
handful of per-table IN queries.
- EpcPostgresRepository: extracted a shared `_compose(rows)` from `get` (the
windows + floor-dim fetches are now passed in, not fetched inline), so both
`get` and the new `get_for_properties(property_ids)` build EpcPropertyData
from pre-fetched rows. `get_for_properties` fetches each child table once
(`WHERE epc_property_id IN ...`), groups in memory, and composes — load-whole
per ADR-0002.
- PropertyRepository.get_many(property_ids) -> Properties: one query for the
property rows + one bulk EPC hydration, composed in input order.
- BaselineOrchestrator / IngestionOrchestrator read the batch via get_many
instead of N x get.
- Ports + fakes gain the bulk methods.
The #1129 round-trip fidelity test stays green (the compose extraction is
behaviour-preserving). New tests: bulk hydration correctness + round-trips are
constant w.r.t. batch size (one-per-table, proven by query count). 123 pass;
pyright strict clean; AAA.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Re-runs of a First Run batch re-save a property's data; that must replace,
not duplicate (ADR-0012 idempotent batch writes).
- `EpcPostgresRepository.save` deletes the property's existing EPC graph
(parent + all child tables, floor-dims via their building parts) before
inserting, when a `property_id` is given. Anonymous saves still insert.
- `BaselinePostgresRepository.save` deletes the existing row for the
`property_id` before inserting — no more unique-constraint violation on
re-save; also what the re-score-on-override path needs.
- Solar already upserts, so it's unchanged.
The #1129 round-trip fidelity test stays green (delete-first is a no-op on
a first save). 2 new tests (re-save replaces, not duplicates). pyright
strict clean; AAA.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
First slice of the per-stage batch-transaction refactor (ADR-0012). A
UnitOfWork is the single transaction a stage runs its batch in: a context
manager exposing the DB repos bound to one session, committing once on
`commit()` and rolling back on exception or exit-without-commit
(all-or-nothing per batch, fail noisily).
- `UnitOfWork` (port): `property` / `epc` / `solar` / `baseline` repos +
`commit()` / `rollback()`; `__exit__` rolls back uncommitted work.
- `PostgresUnitOfWork(session_factory)`: opens a Session from an injected
factory (a module-scoped engine + sessionmaker in prod, so the pool is
reused across warm invocations), binds the Postgres repos to it, closes
on exit.
Not yet wired into any orchestrator — that lands in the Baseline /
Ingestion refactor slices. 3 tests against ephemeral PG (commit durable
across units; exception rolls back; no-commit persists nothing). pyright
strict clean; AAA.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Stage 2 of First Run. Establishes each Property's Baseline Performance
from persisted source data and writes it back — reads only from repos,
never a Fetcher or HTTP (ADR-0003), so it is byte-identical whether
Ingestion ran milliseconds ago or last week.
Domain (`domain/baseline/`):
- `Performance` VO — the four rated quantities: SAP / EPC Band / CO2 /
Primary Energy Intensity. `lodged_performance(epc)` reads them off the
EPC's recorded fields (PEUI = `energy_consumption_current`).
- `BaselinePerformance` (ADR-0004) — the paired `lodged` + `effective`
Performance + `rebaseline_reason`, plus the no-derivation part of the
energy block (`space_heating_kwh` / `water_heating_kwh`, off the RHI,
deterministic per ADR-0006). Both halves always populated.
- `Rebaseliner` port + `StubRebaseliner`: the re-score-on-override seam
(ADR-0011). SAP10 certs pass through (effective == lodged, reason
"none"); a pre-SAP10 cert raises `RebaselineNotImplemented` rather
than fabricating a plausible-but-wrong "none" — ML rebaselining is not
wired yet. Mirrors the repo's strict-raise culture.
Persistence: new `BaselineRepository` port + `BaselinePostgresRepository`
+ flat-column `baseline_performance` SQLModel (one row per Property). Per
ADR-0004's amendment this is a standalone table, NOT columns on the
retiring `property_details_epc`. Production migration is FE-owned
(Drizzle) — docs/migrations/baseline-performance-table.md.
Docs (grill-with-docs): corrected CONTEXT.md Lodged/Effective Performance
to Primary Energy Intensity (the term collided with its own _Avoid_ entry
under "heat demand") + fixed stale RHI field names; amended ADR-0004
Consequences for the standalone-table decision.
Fuel split + bills (rest of EPC Energy Derivation) deferred to a
follow-up — they need a Fuel Rates source (Ofgem-cap ETL) that does not
exist yet.
TDD, one test -> one impl: 7 tests (lodged read, rebaseliner pass-through
+ raise, orchestrator establish-and-persist + pre-SAP10 raise, Postgres
round-trip + absent). pyright strict clean; AAA layout.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add Coordinates value object + GeospatialRepository port + GeospatialS3Repository
adapter. Resolves a Property's lon/lat from the partitioned Ordnance Survey
Open-UPRN parquet (filename_meta -> partition -> UPRN row). A Repo, not a
Fetcher (ADR-0011): no live OS API call. The parquet reader is injected, so it's
unit-tested against fixture parquets with no S3/network; returns None when the
UPRN is uncovered or absent. pyright strict clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move the EpcClientService package (client + _retry + exceptions + tests) from
the dying backend/ tree to infrastructure/epc_client/ as the New-EPC-API Fetcher;
update the two callers (address2UPRN, a script). All 14 client tests pass.
Add SolarRepository port + SolarPostgresRepository persisting Google Solar
building insights as JSONB (solar_building_insights table), one row per Property.
The EPC repo half of this slice already landed in #1129. pyright strict clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add the Ara modelling aggregate root (ADR-0002): domain/property/ with
PropertyIdentity, SiteNotes, Property, Properties. Property.source_path
implements the two disjoint source paths + Recency Tie-Break (ADR-0001;
survey wins on an equal date); effective_epc resolves to the surveyed data
(Site Notes path) or the public EPC (epc_with_overlay path — Landlord
Overrides overlay is a later slice). Pure dataclasses, no infrastructure imports.
PropertyRepository port + PropertyPostgresRepository hydrate the aggregate
whole from a defensive view of the FE-owned 'property' table (identity columns)
plus the EPC slice via EpcRepository.get_for_property. Reads only from repos
(ADR-0003). 8 domain + 1 hydration test; pyright strict clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add epc_renewable_heat_incentive table (space_heating_kwh, water_heating_kwh +
the three insulation-impact kWh fields), wired into EpcPostgresRepository
save/get. This is the P0 gap: RenewableHeatIncentive carries the baseline
space-heating/hot-water kWh that EPC Energy Derivation consumes.
The round-trip test now asserts full deep-equality (dropped the
renewable_heat_incentive exclusion) and passes for RdSAP 21.0.0 + 21.0.1.
DB migration for the new table documented in
docs/migrations/epc-property-round-trip-fidelity.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Relocate EpcPropertyModel + child tables from the dying backend/ tree to
infrastructure/postgres/epc_property_table.py (re-export shim keeps
documents_parser working). Add EpcRepository port + EpcPostgresRepository with
a full reverse mapper (epc_property tables -> EpcPropertyData).
Round-trip test surfaced two fidelity gaps:
1. Union[int,str] SAP code fields were str()-coerced on save, losing the int
(API) vs str (Site Notes) distinction. Now stored as JSONB (type-preserving).
2. The schema was a partial projection. Closed the cheap gaps on the model
(heating shower/bath counts, roof_construction_type, curtain_wall_age,
addendum, mechanical_vent_duct_insulation_level, SAP 10.2 §2 ventilation
fields + a ventilation_present flag). Structural gaps tracked as follow-ups;
renewable_heat_incentive (P0, #1137) excluded from the assertion until landed.
Round-trip passes for RdSAP-Schema-21.0.0 and 21.0.1; pyright strict clean.
Migration inventory for the DB: docs/migrations/epc-property-round-trip-fidelity.md
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
