Commit graph

5812 commits

Author SHA1 Message Date
Daniel Roth
49e7b7fea6 Wire service to get_evidence_files_by_job_id; retire get_core_evidence_files_by_job_id 🟪 2026-06-04 15:40:25 +00:00
Daniel Roth
662f6de0ab get_evidence_files_by_job_id downloads other files when include_other=True 🟩 2026-06-04 15:40:25 +00:00
Daniel Roth
c4ffaaa069 get_evidence_files_by_job_id downloads other files when include_other=True 🟥 2026-06-04 15:40:25 +00:00
Daniel Roth
f95b6bdd7d get_evidence_files_by_job_id returns DownloadedFiles with empty other when include_other=False 🟩 2026-06-04 15:40:25 +00:00
Daniel Roth
665dc69ad5 get_evidence_files_by_job_id returns DownloadedFiles with empty other when include_other=False 🟥 2026-06-04 15:40:25 +00:00
Daniel Roth
e7c679e0db Group evidence into core and other via _group_into_core_and_other_files 🟪 2026-06-04 15:40:25 +00:00
Daniel Roth
99229844b5 _select_other_files returns non-core evidence files 🟩 2026-06-04 15:40:25 +00:00
Daniel Roth
db796747d9 _select_other_files returns non-core evidence files 🟥 2026-06-04 15:40:25 +00:00
Daniel Roth
6cb6c8c756 allow for missing deal stage column when triggering sqs from file 2026-06-04 15:40:25 +00:00
Daniel Roth
790e430aff rename local handler trigger script 2026-06-04 15:40:25 +00:00
Jun-te Kim
a4670f8bc0 deploy lambda 2026-06-04 15:32:51 +00:00
Jun-te Kim
261fae2e79 reformatted to be DDD structure 2026-06-04 14:50:04 +00:00
Jun-te Kim
dfd05ba28b tests files 2026-06-04 11:47:42 +00:00
Jun-te Kim
c614ff6388 save local changes 2026-06-03 12:41:56 +00:00
Jun-te Kim
bf166e7f46 test suite to pick it up 2026-06-02 15:33:46 +00:00
Jun-te Kim
6c8fe86cf9 ddd tests 2026-06-02 15:31:42 +00:00
Jun-te Kim
25ba1427b1 seperate ddd tests 2026-06-02 15:30:58 +00:00
Jun-te Kim
4e02eb7c77 more tests to ensure we don't deploy something that is brokern 2026-06-02 15:03:20 +00:00
Jun-te Kim
a3c80b6691
Merge pull request #1147 from Hestia-Homes/feature/landlord_data
address2uprn was missing a dependency
2026-06-02 11:50:42 +01:00
Jun-te Kim
144233a5f3 backend was missing a dependency 2026-06-02 10:46:29 +00:00
Jun-te Kim
feb3bc08f0
Merge pull request #1144 from Hestia-Homes/feature/landlord_data
if you change the descript it destories and make a new one instead of…
2026-06-02 10:39:17 +01:00
Jun-te Kim
f3ad339cf5 if you change the descript it destories and make a new one instead of edit 2026-06-02 09:36:31 +00:00
Jun-te Kim
8accb51383
Merge pull request #1142 from Hestia-Homes/feature/landlord_data
iam permissions for my lambda to import location
2026-06-02 09:43:32 +01:00
Jun-te Kim
04dc1b20fe iam permissions 2026-06-01 21:08:19 +00:00
Jun-te Kim
0a123c6723
Merge pull request #1122 from Hestia-Homes/feature/landlord_data
Feature/landlord data
2026-06-01 20:25:20 +01:00
Jun-te Kim
616744a606 Merge remote-tracking branch 'origin/main' into feature/landlord_data
# Conflicts:
#	datatypes/epc/schema/rdsap_schema_21_0_0.py
#	datatypes/epc/schema/rdsap_schema_21_0_1.py
2026-06-01 17:02:20 +00:00
Jun-te Kim
bf3b689f15 Remove EPC and asset_list changes unrelated to SAL handler
This branch's objective is the SAL ingestion handler
(applications/SAL/handler.py) and its dependency tree. Drop work
that crept in but is unreferenced by it:

- EPC feature: domain/epc, infrastructure/epc (gov_uk + historical
  clients), tests/infrastructure/epc
- datatypes/epc edits (instantaneous_wwhrs Optional) reverted to main
- asset_list/app.py local data-file/column tweak reverted to main

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-01 16:39:09 +00:00
Jun-te Kim
bdf703ea00 updated rdsap option; seperated s3 location in infrastrucutre; added open ai api 2026-06-01 16:33:14 +00:00
Jun-te Kim
754e6609fd standardist Address 2026-06-01 16:32:48 +00:00
Khalim Conn-Kowlessar
1ea71a3acb refactor(ara): rename FirstRunPipeline → AraFirstRunPipeline (PR #1139 review)
Aligns the composition with its entry point (the `ara_first_run` lambda +
`AraFirstRunTriggerBody`): clearer what the file does.

- orchestration/first_run_pipeline.py → ara_first_run_pipeline.py
- FirstRunPipeline → AraFirstRunPipeline; FirstRunCommand → AraFirstRunCommand
- test files renamed to match

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 16:28:48 +00:00
Khalim Conn-Kowlessar
d89983d44f refactor(property): PropertyRow.id non-Optional (PR #1139 review)
`property` is an FE-owned table the backend only ever reads — every row read
carries an id — so the autoincrement-PK `Optional[int]` idiom doesn't apply
here. Make it `int` and drop the now-redundant None guard in get_many.

(Contrast: solar_table keeps Optional id — the backend DOES insert those, so
id is genuinely None pre-flush.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 16:28:48 +00:00
Khalim Conn-Kowlessar
50914e8aae refactor(property-baseline): units on co2 / PEUI columns (PR #1139 review)
Make the stored units explicit on the property_baseline_performance columns:
- `*_co2_emissions` → `*_co2_emissions_t_per_yr` (tonnes CO₂/yr, whole dwelling)
- `*_primary_energy_intensity` → `*_primary_energy_intensity_kwh_per_m2_yr`

Column names only; the domain `Performance` VO stays unit-suffix-free (units are
a storage concern, mapped in from_domain/to_domain). Migration doc updated.
Round-trip stays green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 16:28:48 +00:00
Khalim Conn-Kowlessar
457d959b1f refactor(property-baseline): rename baseline → property_baseline aggregate (PR #1139 review)
Wholesale rename of the Baseline aggregate to PropertyBaseline for clarity /
to disambiguate from baselines that appear elsewhere in Modelling. Scoped to
this aggregate only — the distinct Rebaselining term (rebaseline_reason,
StubRebaseliner, RebaselineNotImplemented) is deliberately untouched.

- domain/baseline → domain/property_baseline; BaselinePerformance →
  PropertyBaselinePerformance.
- repositories/baseline → repositories/property_baseline; BaselineRepository
  / BaselinePostgresRepository → PropertyBaseline*.
- orchestration/baseline_orchestrator.py → property_baseline_orchestrator.py;
  BaselineOrchestrator → PropertyBaselineOrchestrator. BaselineStage →
  PropertyBaselineStage.
- infrastructure/postgres: baseline_performance_table.py →
  property_baseline_performance_table.py; table `baseline_performance` →
  `property_baseline_performance`; Model renamed.
- UnitOfWork attribute `.baseline` → `.property_baseline`.
- Docs: ADR-0004 references + migration doc (renamed to
  property-baseline-performance-table.md) updated.

CONTEXT.md glossary term ("Baseline Performance") left as-is pending a
ubiquitous-language call (raised on the PR). 123 tests pass; pyright strict
clean (only the unrelated pre-existing moto import errors remain).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 16:28:48 +00:00
Khalim Conn-Kowlessar
d2d008f5c5 perf(repos): bulk get_many / get_for_properties — batch reads, not N round-trips (#1138)
Final slice of ADR-0012: collapse the per-property read round-trips a batch
made (Baseline hydrated ~8 queries x 30 properties one at a time) into a
handful of per-table IN queries.

- EpcPostgresRepository: extracted a shared `_compose(rows)` from `get` (the
  windows + floor-dim fetches are now passed in, not fetched inline), so both
  `get` and the new `get_for_properties(property_ids)` build EpcPropertyData
  from pre-fetched rows. `get_for_properties` fetches each child table once
  (`WHERE epc_property_id IN ...`), groups in memory, and composes — load-whole
  per ADR-0002.
- PropertyRepository.get_many(property_ids) -> Properties: one query for the
  property rows + one bulk EPC hydration, composed in input order.
- BaselineOrchestrator / IngestionOrchestrator read the batch via get_many
  instead of N x get.
- Ports + fakes gain the bulk methods.

The #1129 round-trip fidelity test stays green (the compose extraction is
behaviour-preserving). New tests: bulk hydration correctness + round-trips are
constant w.r.t. batch size (one-per-table, proven by query count). 123 pass;
pyright strict clean; AAA.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 16:28:48 +00:00
Khalim Conn-Kowlessar
7275850c9e refactor(orchestration): wire stages onto the UnitOfWork; per-stage commit (#1138)
Replaces the handler's whole-pipeline Session (one transaction across all
three stages, connection pinned during Ingestion's external IO) with a
Unit-of-Work per stage (ADR-0012, added here). Each stage runs its batch in
one unit and commits once; any property raising aborts the batch and the
subtask fails noisily.

- BaselineOrchestrator(unit_of_work, rebaseliner): one unit for the batch,
  commit once. Raise on a pre-SAP10 property leaves the unit uncommitted.
- IngestionOrchestrator(unit_of_work, epc_fetcher, geospatial_repo,
  solar_fetcher): fetch/write split — phase 1 fetches the whole batch (EPC /
  coords / solar) with NO unit open; phase 2 writes in one unit and commits.
  The connection is never held during external IO. Geospatial S3 repo stays
  injected (reference data, not transactional).
- Handler: module-scoped engine (pool reused across warm invocations) + a UoW
  factory; whole-pipeline `with Session` gone. `build_first_run_pipeline`
  composes on the factory. Source clients still behind the raising seam.
- ADR-0012 records the decision (per-stage boundary, all-or-nothing batch,
  idempotent re-run, fetch/write split, module-scoped engine). Modelling stub
  left untouched (no-op, no DB) per the ADR.

Tests: orchestrators on a shared FakeUnitOfWork (assert persisted batch +
exactly-once commit + no-commit-on-raise). New real-DB E2E integration test:
real PostgresUnitOfWork, Ingestion writes the EPC → Baseline reads it back
through the repo → re-run replaces, not duplicates (1 EPC row, 1 baseline row
after two runs). 121 pass in tests/; pyright strict clean; AAA.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 16:28:48 +00:00
Khalim Conn-Kowlessar
fa5224b6ed feat(repos): idempotent EPC + Baseline writes (replace by property_id) (#1138)
Re-runs of a First Run batch re-save a property's data; that must replace,
not duplicate (ADR-0012 idempotent batch writes).

- `EpcPostgresRepository.save` deletes the property's existing EPC graph
  (parent + all child tables, floor-dims via their building parts) before
  inserting, when a `property_id` is given. Anonymous saves still insert.
- `BaselinePostgresRepository.save` deletes the existing row for the
  `property_id` before inserting — no more unique-constraint violation on
  re-save; also what the re-score-on-override path needs.
- Solar already upserts, so it's unchanged.

The #1129 round-trip fidelity test stays green (delete-first is a no-op on
a first save). 2 new tests (re-save replaces, not duplicates). pyright
strict clean; AAA.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 16:28:48 +00:00
Khalim Conn-Kowlessar
5524385984 feat(uow): UnitOfWork port + PostgresUnitOfWork adapter (#1138)
First slice of the per-stage batch-transaction refactor (ADR-0012). A
UnitOfWork is the single transaction a stage runs its batch in: a context
manager exposing the DB repos bound to one session, committing once on
`commit()` and rolling back on exception or exit-without-commit
(all-or-nothing per batch, fail noisily).

- `UnitOfWork` (port): `property` / `epc` / `solar` / `baseline` repos +
  `commit()` / `rollback()`; `__exit__` rolls back uncommitted work.
- `PostgresUnitOfWork(session_factory)`: opens a Session from an injected
  factory (a module-scoped engine + sessionmaker in prod, so the pool is
  reused across warm invocations), binds the Postgres repos to it, closes
  on exit.

Not yet wired into any orchestrator — that lands in the Baseline /
Ingestion refactor slices. 3 tests against ephemeral PG (commit durable
across units; exception rolls back; no-commit persists nothing). pyright
strict clean; AAA.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 16:28:48 +00:00
Khalim Conn-Kowlessar
61846665b1 feat(first-run): FirstRunPipeline E2E — Ingestion → Baseline → Modelling (#1136)
Completes the First Run spine. Replaces the #1130 stub FirstRunPipeline
with the real three-stage composition and wires it into the handler.

- `FirstRunPipeline.run(command)` sequences Ingestion → Baseline →
  Modelling, threading **only** `property_ids` between stages (and
  `scenario_ids` into Modelling, off the command — never a prior stage's
  output). Stages are injected behind thin `IngestionStage` /
  `BaselineStage` / `ModellingStage` Protocols (the EpcFetcher/SolarFetcher
  idiom), so the handler owns wiring and tests substitute fakes (ADR-0011).
- `ModellingOrchestrator` stub + `ScenarioRepository` / `MaterialsRepository`
  seam ports — `run(property_ids, scenario_ids)` reads through repos, does
  no scoring yet. Method shapes deferred to the Modelling per-service grills
  (Scenario / Scenario Phase / Snapshot / Optimised Package / Plans are rich
  — not pre-empted here).
- Handler delegates to the real pipeline via `build_first_run_pipeline`
  (Postgres-backed repos off the session). The Ingestion source clients
  (EPC API / Google Solar / geospatial S3) are isolated behind one
  `_source_clients_from_env` seam that raises until the deploy/Terraform
  config settles — out of scope for this slice. Subtask complete/failed +
  CloudWatch URL still come from `@subtask_handler`.

Integration test (the criterion's centrepiece): wires REAL Ingestion +
REAL Baseline + stub Modelling through a shared fake EPC repo, with a
repo-backed PropertyRepo composing the Property from that slice. Proves
Baseline reads the very EPC Ingestion persisted — the through-repos
hand-off, no in-memory coupling. Plus a composition test pinning stage
order + only-property_ids threading.

TDD, one test → one impl. pyright strict clean; AAA layout. 116 pass in
the tests/ tree, no regressions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 16:28:48 +00:00
Khalim Conn-Kowlessar
9f22b0aae8 feat(baseline): BaselineOrchestrator + BaselinePerformance aggregate (#1135)
Stage 2 of First Run. Establishes each Property's Baseline Performance
from persisted source data and writes it back — reads only from repos,
never a Fetcher or HTTP (ADR-0003), so it is byte-identical whether
Ingestion ran milliseconds ago or last week.

Domain (`domain/baseline/`):
- `Performance` VO — the four rated quantities: SAP / EPC Band / CO2 /
  Primary Energy Intensity. `lodged_performance(epc)` reads them off the
  EPC's recorded fields (PEUI = `energy_consumption_current`).
- `BaselinePerformance` (ADR-0004) — the paired `lodged` + `effective`
  Performance + `rebaseline_reason`, plus the no-derivation part of the
  energy block (`space_heating_kwh` / `water_heating_kwh`, off the RHI,
  deterministic per ADR-0006). Both halves always populated.
- `Rebaseliner` port + `StubRebaseliner`: the re-score-on-override seam
  (ADR-0011). SAP10 certs pass through (effective == lodged, reason
  "none"); a pre-SAP10 cert raises `RebaselineNotImplemented` rather
  than fabricating a plausible-but-wrong "none" — ML rebaselining is not
  wired yet. Mirrors the repo's strict-raise culture.

Persistence: new `BaselineRepository` port + `BaselinePostgresRepository`
+ flat-column `baseline_performance` SQLModel (one row per Property). Per
ADR-0004's amendment this is a standalone table, NOT columns on the
retiring `property_details_epc`. Production migration is FE-owned
(Drizzle) — docs/migrations/baseline-performance-table.md.

Docs (grill-with-docs): corrected CONTEXT.md Lodged/Effective Performance
to Primary Energy Intensity (the term collided with its own _Avoid_ entry
under "heat demand") + fixed stale RHI field names; amended ADR-0004
Consequences for the standalone-table decision.

Fuel split + bills (rest of EPC Energy Derivation) deferred to a
follow-up — they need a Fuel Rates source (Ofgem-cap ETL) that does not
exist yet.

TDD, one test -> one impl: 7 tests (lodged read, rebaseliner pass-through
+ raise, orchestrator establish-and-persist + pre-SAP10 raise, Postgres
round-trip + absent). pyright strict clean; AAA layout.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 16:28:48 +00:00
Khalim Conn-Kowlessar
a910ce9855 feat(ara): AraFirstRunTriggerBody + ara_first_run lambda skeleton (#1130)
Stage-2 entry point for the First Run use case. Adds the
`ara_first_run` Lambda package mirroring the `postcode_splitter`
template, its typed trigger contract, and a stub `FirstRunPipeline`.

- `AraFirstRunTriggerBody`: thin command of five fields — `task_id`,
  `sub_task_id` (UUID, lifecycle), `portfolio_id`, `property_ids`,
  `scenario_ids` (int business IDs). No `model_config` override, so
  Pydantic's default `extra="ignore"` lets the FastAPI backend add
  fields without breaking deployed lambdas. UPRNs / Scenario defs are
  deliberately off the event — read from source-of-truth tables.
- Thin `handler.py`: validate-and-delegate only, via a named
  `dispatch_first_run` seam (testable without the Lambda runtime).
  Subtask status (in-progress/complete/failed) + CloudWatch log URL
  come for free from the existing `@subtask_handler()` decorator.
- `FirstRunPipeline` (orchestration/) stub: `run(command)` receives the
  validated command. Declares a structural `FirstRunCommand` Protocol
  (the three business fields) that `AraFirstRunTriggerBody` satisfies,
  so orchestration needs no application-layer import — rhymes with the
  `EpcFetcher`/`SolarFetcher` Protocols on IngestionOrchestrator
  (ADR-0011). Full Ingestion→Baseline→Modelling composition lands in
  #1136.
- Dockerfile / requirements.txt / local_handler/ mirror postcode_splitter.

TDD: 7 new tests (trigger-body validation incl. forward-compat +
id-types, pipeline seam, handler delegation). pyright strict clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 16:28:48 +00:00
Khalim Conn-Kowlessar
454456bf22 feat(ingestion): IngestionOrchestrator end-to-end (#1134)
Stage 1 of the pipeline: per property, read its UPRN from the property row,
fetch its EPC, resolve coordinates from the Geospatial reference repo, thread
those into the Solar fetcher, and persist EPC + solar via repos. Fetchers never
call each other — the orchestrator threads the coordinate (ADR-0011). Coordinates
are reference data (deterministic from UPRN), resolved transiently to drive the
solar fetch rather than persisted per-property.

Depends on thin EpcFetcher/SolarFetcher Protocols (EpcClientService and
GoogleSolarApiClient satisfy them structurally). Unit-tested against fakes — no
DB, gov API, or network: persists EPC, threads coords into solar, skips
UPRN-less properties and skips solar when coordinates are absent. pyright clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 16:28:48 +00:00
Khalim Conn-Kowlessar
285e7f8824 feat(geospatial): GeospatialRepo — OS Open-UPRN coordinate lookup (#1131)
Add Coordinates value object + GeospatialRepository port + GeospatialS3Repository
adapter. Resolves a Property's lon/lat from the partitioned Ordnance Survey
Open-UPRN parquet (filename_meta -> partition -> UPRN row). A Repo, not a
Fetcher (ADR-0011): no live OS API call. The parquet reader is injected, so it's
unit-tested against fixture parquets with no S3/network; returns None when the
UPRN is uncovered or absent. pyright strict clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 16:28:48 +00:00
Khalim Conn-Kowlessar
5a3be9d672 feat(ingestion): relocate EpcClientService to infrastructure + SolarRepo (#1133)
Move the EpcClientService package (client + _retry + exceptions + tests) from
the dying backend/ tree to infrastructure/epc_client/ as the New-EPC-API Fetcher;
update the two callers (address2UPRN, a script). All 14 client tests pass.

Add SolarRepository port + SolarPostgresRepository persisting Google Solar
building insights as JSONB (solar_building_insights table), one row per Property.
The EPC repo half of this slice already landed in #1129. pyright strict clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 16:28:48 +00:00
Khalim Conn-Kowlessar
460970af39 feat(property): Property aggregate + PropertyRepository (#1132)
Add the Ara modelling aggregate root (ADR-0002): domain/property/ with
PropertyIdentity, SiteNotes, Property, Properties. Property.source_path
implements the two disjoint source paths + Recency Tie-Break (ADR-0001;
survey wins on an equal date); effective_epc resolves to the surveyed data
(Site Notes path) or the public EPC (epc_with_overlay path — Landlord
Overrides overlay is a later slice). Pure dataclasses, no infrastructure imports.

PropertyRepository port + PropertyPostgresRepository hydrate the aggregate
whole from a defensive view of the FE-owned 'property' table (identity columns)
plus the EPC slice via EpcRepository.get_for_property. Reads only from repos
(ADR-0003). 8 domain + 1 hydration test; pyright strict clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 16:28:48 +00:00
Khalim Conn-Kowlessar
3e1d3acfbf feat(epc): persist renewable_heat_incentive — full round-trip equality (#1137)
Add epc_renewable_heat_incentive table (space_heating_kwh, water_heating_kwh +
the three insulation-impact kWh fields), wired into EpcPostgresRepository
save/get. This is the P0 gap: RenewableHeatIncentive carries the baseline
space-heating/hot-water kWh that EPC Energy Derivation consumes.

The round-trip test now asserts full deep-equality (dropped the
renewable_heat_incentive exclusion) and passes for RdSAP 21.0.0 + 21.0.1.
DB migration for the new table documented in
docs/migrations/epc-property-round-trip-fidelity.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 16:28:48 +00:00
Khalim Conn-Kowlessar
559616d3bb feat(epc): EPC persistence round-trip fidelity + JSONB code columns (Slice 1 #1129)
Relocate EpcPropertyModel + child tables from the dying backend/ tree to
infrastructure/postgres/epc_property_table.py (re-export shim keeps
documents_parser working). Add EpcRepository port + EpcPostgresRepository with
a full reverse mapper (epc_property tables -> EpcPropertyData).

Round-trip test surfaced two fidelity gaps:
 1. Union[int,str] SAP code fields were str()-coerced on save, losing the int
    (API) vs str (Site Notes) distinction. Now stored as JSONB (type-preserving).
 2. The schema was a partial projection. Closed the cheap gaps on the model
    (heating shower/bath counts, roof_construction_type, curtain_wall_age,
    addendum, mechanical_vent_duct_insulation_level, SAP 10.2 §2 ventilation
    fields + a ventilation_present flag). Structural gaps tracked as follow-ups;
    renewable_heat_incentive (P0, #1137) excluded from the assertion until landed.

Round-trip passes for RdSAP-Schema-21.0.0 and 21.0.1; pyright strict clean.
Migration inventory for the DB: docs/migrations/epc-property-round-trip-fidelity.md

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 16:28:48 +00:00
Khalim Conn-Kowlessar
8291f29721 docs(ara): composable stage-orchestrator design (ADR-0011 + ADR-0003 amend + CONTEXT)
Records the grill-with-docs outcomes for the ara_first_run rebuild: three
composable stage orchestrators (Ingestion/Baseline/Modelling), one lambda per
use case chaining them through repos (not in-memory), and the Fetcher-vs-Repo
data-source taxonomy. Amends ADR-0003's chaining rule to generalise beyond
RefreshOrchestrator. Adds the pipeline-composition + First Run vocabulary to
CONTEXT.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 16:28:48 +00:00
Khalim Conn-Kowlessar
ca6a0efd70 Slice S0380.155: SAP 10.2 Table 4a — heat-pump water-efficiency column dispatch
SAP 10.2 Table 4a (PDF p.163-164) heat-pump rows split efficiency into
two columns — "space" and "water":

    Code  System                                            space  water
    211   Ground source HP with flow temp <= 35°C            230    170
    213   Water source HP with flow temp <= 35°C             230    170
    215   Gas-fired GSHP with flow temp <= 35°C              120     84
    216   Gas-fired WSHP with flow temp <= 35°C              120     84
    217   Gas-fired ASHP with flow temp <= 35°C              110     77
    521   Warm-air electric GSHP                             230    170
    523   Warm-air electric WSHP                             230    170
    525   Warm-air gas-fired GSHP                            120     84
    526   Warm-air gas-fired WSHP                            120     84
    527   Warm-air gas-fired ASHP                            110     77

The split reflects real physics: heat pumps lose efficiency raising
water to ~55°C DHW temperatures vs ~35°C space-heating flow. ASHP
"in other cases" (codes 214, 221, 223, 224) and the "other cases"
gas-fired rows (225-227) have space == water = 170 / 84 / 77 — no
distinct DHW column.

Pre-slice the cascade routed WHC ∈ {901, 902, 914} ("HW from main
heating") through `seasonal_efficiency(main_code)`, which only consults
the Space column. For SAP code 211 the cascade returned 2.30 (= space)
when the spec requires 1.70 (= water). HW fuel kWh undercounted by
26% on the heating-systems corpus gshp variant: cascade 841.47 kWh vs
worksheet 1138.46 kWh.

New `_TABLE_4A_HEAT_PUMP_WATER_EFFICIENCY` dict (10 codes where Space
≠ Water) consulted in `_water_efficiency_with_category_inherit` before
falling through to the existing `seasonal_efficiency` path. Codes
where Space == Water keep the legacy inheritance — no behaviour
change. Non-HP main heating (boilers, storage heaters) likewise
unchanged.

Closures (gshp variant — SAP code 211 + WHC=901 + cylinder):
  HW fuel kWh:  841.47 → 1138.45 (matches worksheet 1138.46)
  ΔSAP_c:       +0.9373 → -0.0178
  Δcost:        -£21.60 → +£0.41
  ΔCO2:         -34.98  → +7.06 kg/yr
  ΔPE:          -418.92 → +33.52 kWh/yr

No regressions on 40 other corpus variants — gshp is the only fixture
that lodges a heat-pump code with diverging Space/Water columns.

Cohort-1 ASHP closure (S0380.28 reciprocal interpolation) is unaffected
because that path runs through `heat_pump_record` PCDB Appendix N3
when a PCDB Table 362 record is lodged; this fix is the Table 4a
fallback for cases without a PCDB record.

Extended handover suite: 899 pass / 0 fail. Pyright net-zero (43 → 43).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-01 16:28:48 +00:00
Khalim Conn-Kowlessar
981aaadf73 Slice S0380.154: SAP 10.2 §12.4.4 — back-boiler summer-immersion HW split
SAP 10.2 §12.4.4 (PDF p.36-37):

  "Independent boilers that provide domestic hot water usually do so
   throughout the year. With open fire back boilers or closed room
   heaters with boilers, an alternative system (electric immersion)
   may be provided for heating water in summer. In that case water
   heating is provided by the boiler for months October to May and by
   the alternative system for months June to September."

Scope is verbatim Table 4a codes 156 (Open fire with back boiler to
radiators) and 158 (Closed room heater with boiler to radiators). Range
cooker boilers (160, 161), pellet stoves with boilers (159), and
independent solid-fuel boilers (151, 153, 155) are NOT covered.

Pre-slice, the cascade treated the back-boiler cohort identically to
year-round solid-fuel mains: (59)m primary loss applied Jun-Sep, HW
fuel kWh was billed entirely at the boiler's solid-fuel rate, the HW
CO2 / PE factors used the boiler fuel's annual factor, and the off-peak
electric standing charge (£40 for 18-hour tariff) was not added because
the cert's lodged water-heating fuel code was anthracite.

Implementation (4 wired pieces):

1. `_section_12_4_4_summer_immersion_applies(epc, main)` — predicate
   gate keyed on back-boiler SAP code (156, 158) + WHC ∈ {901, 902, 914}
   "HW from main heating" + cylinder present.

2. `_primary_loss_override` zeroes (59)m for Jun-Sep when the predicate
   fires — matches the Elmhurst P960 worksheet which has (59) Jun-Sep =
   0 for SF2 (vs ~42 kWh/month for SF3 range cooker).

3. `_section_12_4_4_hw_blend(...)` — returns the 5-tuple
   (annual_hw_fuel_kwh, blended_cost_gbp_per_kwh, blended_co2_factor,
   blended_pe_factor, extra_standing_charge_gbp). The blend is kWh-
   weighted across:
   - Winter Oct-May: boiler fuel at the boiler's Table 32 unit price /
     Table 12 annual CO2 / Table 12 annual PE factor
   - Summer Jun-Sep: standard electricity (Table 12d/12e monthly
     factors weighted by summer (62)m demand) priced at the tariff's
     off-peak low rate per Table 13 note 2 (the 6.8 - 0.036V × N -
     0.105V dual-immersion formula clamps to zero high-rate for
     normal V/N combos on tariffs with ≥18 hrs low rate; SF2 has
     V=110, N≈2 → 100% low-rate)
   - The Table 32 off-peak electric standing charge that fires when
     hot water uses off-peak electricity per Table 12 note (a). For
     EIGHTEEN_HOUR tariff this is Table 32 code 38 = £40.

4. Orchestrator (`cert_to_inputs`) resolves the blend once and overrides
   `hot_water_kwh_per_yr`, `hot_water_fuel_cost_gbp_per_kwh`,
   `hot_water_co2_factor_kg_per_kwh`, `hot_water_primary_factor`, and
   `standing_charges_gbp` when the predicate fires. Other certs fall
   back to the existing single-fuel HW helpers (no behaviour change).

Worksheet evidence (heating-systems corpus property 001431 SF2 — code
158 + WHC=901 + cylinder thermostat + 18-hour tariff):
  - (62) Oct-May = 2205.80 kWh, Jun-Sep = 684.55 kWh
  - (217)m = 65 winter / 100 summer, (219) = 3393.5 anthr + 684.55 elec
    = 4078.06 fuel kWh
  - (247) HW cost = 4078.06 × 4.27 p/kWh blended = £174.25
  - (251) Standing = £40 (off-peak electric standing only — solid fuel
    has no standing charge)
  - (255) Total = £801.13

Closures (SF2):
  ΔSAP_c   +1.86 → -0.0000  (EXACT)
  Δcost   -£42.84 → -£0.00  (EXACT)
  ΔCO2  +346.87  → -93.10 kg/yr (residual: Elmhurst CO2 blend uses a
                                  different summer-month weighting that
                                  the SAP 10.2 Table 12d cascade does
                                  not reproduce — spec-correct per
                                  Table 12d header).
  ΔPE   -605.76  → -1027.51 kWh/yr (same spec-vs-Elmhurst PE blend
                                     artifact via Table 12e monthly
                                     cascade).

No regressions: 40/41 corpus variants unchanged (gate is narrow by SAP
code 156/158). Extended handover suite 898 pass / 0 fail. Pyright net-
zero (43 → 43).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-01 16:28:48 +00:00
Khalim Conn-Kowlessar
0001c7d11f Slice S0380.153: SAP 10.2 Table 3 — not-separately-timed DHW for solid-fuel boilers
SAP 10.2 Table 3 (PDF p.160) provides three primary-loss rows keyed off
the DHW timing arrangement, the middle row giving winter h=5 / summer
h=3 for "Cylinder thermostat, water heating NOT separately timed".

Solid-fuel boiler systems (Table 4a codes 151-161 — independent boilers,
open-fire + back boilers, closed room heaters with boilers, range cooker
boilers, stoves with boilers) do not ship with dual programmers. Per
SAP 10.2 §9.2.4 (PDF p.27) these are "independent solid fuel boilers,
open fires with a back boiler and room heaters with a boiler" — the
appliance itself is the timer. DHW timing follows the burn schedule,
not a separate cylinder programmer, so the middle Table 3 row applies.

Pre-slice `_separately_timed_dhw` returned True for any cylinder +
non-electric HW fuel cert (the S0380.140 gate), routing solid-fuel
boilers through h=3 year-round (the third row, "Cylinder thermostat,
water heating separately timed"). That under-counted winter (59)m
by ~21 kWh/month × 8 winter months across the affected cohort, with
the under-counted water-heating gain propagating into MIT / SH / SAP.

New gate: `sap_main_heating_code in _TABLE_4A_SOLID_FUEL_BOILER_CODES`
(frozenset of {151, 153, 155, 156, 158, 159, 160, 161}) — added before
the existing cylinder-present fallback. The post-S0380.140 electric-
immersion / heat-pump / no-main branches are unchanged. Table 4b
liquid-fuel boilers (101-141) keep the True default — modern gas/oil
installations standardly include dual programmers and the worksheet
confirms `oil 1` / `oil pcdb 1..3` / `pcdb 1` are pinned exact at
h=3 year-round.

Worksheet evidence (heating-systems corpus property 001431):
  - solid fuel 3 (SAP code 160 range cooker boiler + WHC=901
    cylinder thermostat): worksheet (59)m winter = 64.58 (h=5, p=0)
    and summer = 41.92 / 43.31 (h=3, p=0). Cascade closes ΔSAP +0.30
    → −0.0000, Δcost −£6.84 → −0.00, ΔPE −214 → −0.00 (4-metric exact).
  - solid fuel 2 (SAP code 158 closed room heater + back boiler):
    same Table 3 fix narrows ΔSAP +2.06 → +1.86. Remaining ~1.86 SAP
    is the SAP 10.2 §12.4.4 immersion-in-summer rule for back-boilers
    (codes 156, 158) — the worksheet has summer (59)m = 0 because the
    Elmhurst P960 lodges `Summer Immersion: Yes` + the spec routes
    Jun-Sep HW through an electric immersion at η=100%. That's a
    bigger lift (monthly HW efficiency + fuel-split plumbing) and is
    a follow-up slice.

Other corpus variants: no impact (verified via cohort sweep). The
gate is narrow by SAP code so only the 2 affected variants move.

Extended handover suite: 897 pass / 0 fail (+1 from new AAA test).
Pyright net-zero (43 → 43, transient +1 fixed via `EpcPropertyData`
import on the new test's `_cylinder_epc_for` return annotation).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-01 16:28:48 +00:00