Commit graph

96 commits

Author SHA1 Message Date
Daniel Roth
beb4e5d0d9 Move SharePoint renamer logic from scripts/ into orchestrator and app-root handler 2026-06-15 11:01:51 +00:00
Jun-te Kim
a6123d762c Merge branch 'main' of https://github.com/Hestia-Homes/Model into feature/junte+khalim 2026-06-12 13:45:30 +00:00
Jun-te Kim
ff4a2e4242
Merge pull request #1198 from Hestia-Homes/feature/bill-derivation
Feature/bill derivation
2026-06-12 14:44:30 +01:00
Jun-te Kim
77c5f7da49 Merge branch 'feature/bill-derivation' of https://github.com/Hestia-Homes/Model into feature/junte+khalim 2026-06-12 12:52:40 +00:00
Jun-te Kim
32de7f6c3f 17.1 and 18 done by claude 2026-06-12 12:52:36 +00:00
Khalim Conn-Kowlessar
b211715750 feat(modelling): wire secondary-heating-removal into the pipeline (ADR-0028)
Orchestrator runs recommend_secondary_heating_removal; report._triggers_for
explains it via the lodged secondary_heating_type; harness catalogue + ARA seed
price it. Re-pins the golden/integration plans it shifts: it is a cheap (\£250)
SAP lever, so on gas-main certs lodging an electric secondary (691) it displaces
the \£12k ASHP (0330, 0036) or joins the all-beneficial-measures package (000490,
where its marginal SAP is 0 under the category-4 ASHP but the heater is still
physically removed). Consistent with the optimiser's existing kitchen-sink
package behaviour.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 16:04:07 +00:00
Daniel Roth
0edeeaefa6 populate_sheet writes to new Sero template column layout 🟩 2026-06-10 13:21:50 +00:00
Daniel Roth
236f33c25f move spreadsheet population logic to domain 2026-06-09 14:43:24 +00:00
Daniel Roth
608006d977 reapply conditional formatting to populated file 2026-06-09 13:29:11 +00:00
Daniel Roth
f8c955b2d3 local runner and correct template path 2026-06-09 13:03:05 +00:00
Daniel Roth
82156fac8f Audit generator populates XLSX, uploads to S3, and records UploadedFile row 🟪
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 12:05:55 +00:00
Daniel Roth
612d522b35 Audit generator populates XLSX, uploads to S3, and records UploadedFile row 🟩
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 12:01:08 +00:00
Daniel Roth
a1d09aa880 Audit generator populates XLSX, uploads to S3, and records UploadedFile row 🟥
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 11:59:09 +00:00
Daniel Roth
5178cd02c5 UploadedFile, FileTypeEnum, FileSourceEnum importable from infrastructure.postgres.uploaded_file_table 🟩
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 11:50:51 +00:00
Jun-te Kim
06cb4f7b6e Merge branch 'feature/bill-derivation' into feature/junte+khalim 2026-06-09 10:06:40 +00:00
Khalim Conn-Kowlessar
7942a8101a feat(modelling): considered_measures allowlist on the orchestrator
Add domain/modelling/considered_measures.py::restrict_to_considered_measures —
the pure allowlist that limits a run to a chosen set of MeasureType (mirroring
the legacy engine's `inclusions`). It filters at the Option level, so a
multi-option Recommendation (e.g. Heating & Hot Water competing HHRSH against
an ASHP bundle) is kept with only its allowed Options; a Recommendation left
with none is dropped. None = consider everything (unrestricted default).

Thread `considered_measures: frozenset[MeasureType] | None` through
ModellingOrchestrator.run -> _plan_for -> _scored_candidate_groups /
_candidate_recommendations (applies the filter) and _measure_dependencies
(suppresses a forced dependency whose required measure is outside the
allowlist, so a restricted run forces nothing it is not considering). The
local-run seam (harness.console.run_modelling) gains the same param.

The Optimiser still freely chooses among survivors — including none. Tests:
the pure filter (3 cases) + an orchestrator-seam test proving a
{solar_pv}-restricted run yields only solar_pv options. 257 pass + 3 xfail;
pyright clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 20:32:11 +00:00
Jun-te Kim
b48700e964 Merge branch 'main' into feature/junte+khalim 2026-06-08 16:56:15 +00:00
Daniel Roth
3136d5a0f8 use correct db session to write to db 2026-06-08 14:07:03 +00:00
Daniel Roth
aaeb339254 Logging plus invoke locally bypassing subtask handler 2026-06-08 13:32:59 +00:00
Khalim Conn-Kowlessar
24492aa4ba Merge origin/main into feature/bill-derivation (calculator + mapper fixes)
Pulls in 42 commits of calculator/mapper accuracy fixes from the per-cert
mapper-validation and floor/roof/heating fronts.

Conflict resolutions:
- mapper `_is_elmhurst_roof_window`: main dropped the branch's "wall location →
  vertical" guard (it broke cert 000516's rooflight), but that re-broke cert
  001431's two External-wall U>3.0 windows (which must stay vertical). The two
  certs lodge a BYTE-IDENTICAL §11 row, so neither location nor U separates
  them — the real discriminator is the room-in-roof context. Replaced the
  unconditional U>3.0 backstop with one gated on the BP having a room-in-roof
  (`_elmhurst_bp_has_room_in_roof`): 000516's Main BP has a "Room in roof type
  1" (→ rooflight), 001431's does not (→ vertical). Validated against BOTH —
  full Elmhurst worksheet suite 1038 pass + the 001431 window-extraction pin.
- property_postgres_repository: kept main's `ids_by_uprn` method + the branch's
  `_restrictions_of` helper.
- sap_fuel.py: the branch relocated it to domain/billing/ (already carrying
  main's to_table_32_code normalization), so kept the old path deleted.

Fallout from main's fabric fixes (validated by the boiler-3 real-cert pin which
still reproduces at delta 0):
- re-pinned the boiler-1 + boiler-instant-hw ASHP snapshot scores;
- main's §14.2 gas-boiler main-fuel derivation resolved the BGB/102 baseline
  gap, so `test_gas_boiler_instant_hw_before_baselines` is now a passing test
  (was an xfail tripwire).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 13:12:21 +00:00
Khalim Conn-Kowlessar
b249f69cb2 feat(modelling): thread SolarPotential into the orchestrator's solar Generator
Slice 8 of the Solar PV Recommendation Generator (ADR-0026). The
ModellingOrchestrator now reads each Property's persisted Google Solar
buildingInsights JSON (uow.solar), projects it once per Property into a typed
SolarPotential via `_solar_potential_for` (None for a missing or error
payload), and threads it into `recommend_solar` alongside planning_restrictions
— mirroring the ASHP wiring. Solar fires only when a feasible potential is
present, so dwellings without fetched solar data are unaffected.

FakeSolarRepo now returns None for an unseeded Property (was raising) and
supports `by_property` seeding, so the orchestrator's new solar read is exercised.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 12:22:56 +00:00
Daniel Roth
c22ee3821b Merge branch 'main' into feature/handle-new-magicplan-response-structure 2026-06-08 09:57:26 +00:00
Khalim Conn-Kowlessar
0f89845321 feat(modelling): wire the ASHP bundle into the candidate pool
recommend_heating now receives planning_restrictions in the orchestrator (the
ASHP planning gate); the ASHP bundle joins the free candidate pool for every
house/bungalow. Catalogue + contingency (legacy 0.25) gain air_source_heat_pump;
report.py _triggers_for explains the ASHP trigger; the harness forcing test
covers it. Integration tests seed an air_source_heat_pump MaterialRow (ASHP
fires on every house, the broadest trigger yet). NB the optimiser correctly does
NOT select ASHP for an EPC-band goal — gas->electric does not improve the SAP
cost-rating; ASHP is a CO2/PE measure, selectable once non-EPC goals land. ASHP
bundle COMPLETE (S5-S7). ADR-0024.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-06 17:12:07 +00:00
Khalim Conn-Kowlessar
b55ab3727f feat(modelling): wire the HHR storage bundle into the candidate pool
recommend_heating joins the free candidate pool in _candidate_recommendations;
the HHR storage bundle reaches the optimised package for an electric/off-gas
dwelling. Catalogue + contingency (legacy 0.10) gain
high_heat_retention_storage_heaters; report.py _triggers_for explains the
heating trigger (electric/off-gas main); the harness _GENERATOR_MEASURE_TYPES
forcing test covers it. ASHP + boiler bundles still to come. ADR-0024.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 19:22:50 +00:00
Jun-te Kim
8b9dcc73f2 fix 2026-06-05 17:24:17 +00:00
Daniel Roth
e84de954fb define MagicPlanConfig class to get environment variables 2026-06-05 15:46:32 +00:00
Daniel Roth
198d2afdb1 Merge branch 'main' into feature/handle-new-magicplan-response-structure 2026-06-05 14:35:56 +00:00
Daniel Roth
8e349704b1 move magic plan handler to applications/ 2026-06-05 14:33:26 +00:00
Daniel Roth
37b5a3a6e5 move domain code out of datatypes/domain 2026-06-05 14:07:28 +00:00
Khalim Conn-Kowlessar
f7863f986d feat(modelling): wire the lighting generator into the candidate pool
Slice 4 of the lighting generator (ADR-0023): run recommend_lighting in
_candidate_recommendations (no planning gate). Price low_energy_lighting in the
offline catalogue + contingency table (0.26, the legacy rate); the
_GENERATOR_MEASURE_TYPES forcing test enforces both. A run_modelling test pins
the wiring end-to-end (an incandescent-lit dwelling gets the LED upgrade in the
optimised package).

Downstream updates, all because lighting now fires on any cert with non-LED
bulbs: report.py gains the low_energy_lighting trigger (the non-LED counts); the
two golden-cert report tests and the multi-measure integration test now expect
low_energy_lighting alongside the fabric measures (the sample/golden EPCs lodge
low-energy-unknown bulbs); first-run integration seeds a low_energy_lighting
MaterialRow.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 12:39:54 +00:00
Jun-te Kim
b07db1ef6b property override 2026-06-05 12:18:13 +00:00
Khalim Conn-Kowlessar
2c36a8e1d6 Merge remote-tracking branch 'origin/main' into feature/bill-derivation
# Conflicts:
#	repositories/property/property_postgres_repository.py
#	tests/orchestration/fakes.py
2026-06-05 11:09:00 +00:00
Daniel Roth
ebd6f1623f Merge branch 'main' into feature/handle-new-magicplan-response-structure 2026-06-05 10:16:14 +00:00
Khalim Conn-Kowlessar
456a81df0a feat(modelling): wire glazing generator into the candidate pool
Slice 4 of the glazing generator (ADR-0022): run recommend_glazing in
_candidate_recommendations, threading the Property's PlanningRestrictions so a
protected dwelling is offered secondary glazing instead of double (mirrors
recommend_solid_wall). Price both Measure Types in the offline catalogue
(double £600/window, secondary £510 -- the legacy 0.85x scaling) and the
contingency table (0.15, the legacy windows_glazing rate); the
_GENERATOR_MEASURE_TYPES forcing test enforces both entries exist.

run_modelling tests pin the wiring end-to-end on an all-single-glazed dwelling:
double when unrestricted, secondary when listed. The first-run integration test
seeds a double_glazing Product because its lodged EPC has a single-glazed
window. _single_glazed_epc() deep-copies build_epc() (which shares its window
objects) so the mutation can't leak into other tests' baselines.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 09:29:09 +00:00
Khalim Conn-Kowlessar
7d40cddf3b feat(modelling): fold loft into the roof dispatcher; thatch routes to loft
Slice 2 (ADR-0021). `recommend_roof_insulation` now owns the loft branch as the
fallback — a plain pitched loft, a thatched roof (the covering doesn't block
insulating the loft floor), or an unlodged roof type all take loft (joist)
insulation at 300 mm when `roof_insulation_thickness == 0`. Sloping is tested
first; a no-access roof gets nothing. Retired the standalone
`recommend_loft_insulation`; the orchestrator and its tests now call the
dispatcher.

Pinned: thatch before→after (None→300) reproduces at 1e-4; the existing loft pin
still holds through the dispatcher. Behaviour-preserving on the golden cohort
(roof measure unchanged: none across all 57) — the dispatch is strictly more
precise (won't fire loft on a sloping/no-access roof).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 21:12:01 +00:00
Khalim Conn-Kowlessar
af5dfdf8e2 feat(ingestion): cache the spatial reference per UPRN
Slice 3c.4. Ingestion now resolves the whole spatial reference in one lookup
(`spatial_for`) — the coordinates drive the Solar fetch as before, and the
reference (coordinates + planning protections) is persisted per-UPRN via
`uow.spatial` in the same write batch, so Modelling can read the protections
back off the Property (ADR-0020). `_Fetched` carries the UPRN and the reference
into the write phase.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 17:24:03 +00:00
Khalim Conn-Kowlessar
c5182627ba feat(modelling): thread Property planning restrictions to the solid-wall gate
Slice 3b+3d (ADR-0019/0020). Property gains a planning_restrictions attribute
(default unrestricted); the ModellingOrchestrator threads it from the Property
through _plan_for -> _scored_candidate_groups -> _candidate_recommendations into
recommend_solid_wall, replacing the unrestricted default. run_modelling exposes
a planning_restrictions param so the offline harness can inspect restricted
properties. Integration test: a listed solid-brick dwelling that gets IWI when
unrestricted now yields no wall insulation. 145 tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 16:32:26 +00:00
Khalim Conn-Kowlessar
7648032d73 feat(modelling): wire solid-wall insulation into the candidate pool
Slice 2e. recommend_solid_wall joins the orchestrator's fabric generator pool
(restrictions default unrestricted until slice 3 sources them); the harness
catalogue + contingencies (26%) gain external_wall_insulation /
internal_wall_insulation. run_modelling on an uninsulated solid-brick dwelling
(baseline SAP 36.6) now selects internal wall insulation into the optimised
package; the catalogue-completeness guard covers both new measure types.
Golden cohort 57/57 still error-free; IWI now fires on a real cohort cert.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 16:15:56 +00:00
Khalim Conn-Kowlessar
899532a6d6 feat(baseline): CalculatorRebaseliner — calculator goes load-bearing (ADR-0013 amend)
Slice 5a: the promotion. Replaces StubRebaseliner in production and collapses the
shadow runner into the rebaseliner (ADR-0013 amendment).

- CalculatorRebaseliner runs Sap10Calculator on every Property:
  * sap_version < 10.2 -> Effective Performance IS the calculator output
    (band via Epc.from_sap_score, CO2 kg->t, PEUI rounded), reason "pre_sap10".
  * sap_version >= 10.2 -> Effective = lodged (API figures on-target), and the
    calculator only logs divergence (SAP>0.5, PEUI/CO2 1%) as a validation signal.
  * a calculator raise propagates -> batch aborts (ADR-0012); fix the cert at once.
- Rebaseliner.rebaseline gains property_id (for the divergence log).
- LoggingCalculatorShadow / the calculator_shadow seam removed from the
  orchestrator; its divergence-comparison logic now lives in the rebaseliner.
- StubRebaseliner kept (signature updated) for orchestrator/repo unit tests.

The SapResult->EnergyBreakdown adapter + BillDerivation wiring (to populate the
bill block) follow once the appliances/cooking SapResult fields land.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 15:40:26 +00:00
Khalim Conn-Kowlessar
741993270e feat(baseline): run Sap10Calculator in shadow on Property Baseline (ADR-0013)
Wire Sap10Calculator into PropertyBaselineOrchestrator as a non-load-bearing
shadow runner. For each property it scores the Effective EPC beside the
load-bearing Lodged/Effective write, catches any strict-raise -> log.error
(never aborts the batch), and on success log.warning's divergence from Lodged:
SAP |continuous - lodged| > 0.5; PEUI/CO2 > 1% relative (CO2 after kg->tonnes).
Every line is tagged with sap_version so SAP-10.2 signal separates from
older-spec drift (ADR-0010 Validation Cohort).

Per ADR-0013, Calculated SAP10 Performance is not a persisted third value-set:
effective = calculated in every baselining scenario, so the calculator IS the
mechanism that produces Effective Performance (the Rebaseliner). It runs in
shadow only while being hardened; when overrides/estimation land it is promoted
to drive Effective and the failure posture flips to abort (ADR-0012, calculator
now load-bearing). No table change.

- ADR-0013 + CONTEXT (Calculated SAP10 Performance / Effective Performance /
  Rebaselining) record the decision.
- CalculatorShadow port + LoggingCalculatorShadow + Calculator protocol.
- FakeCalculatorShadow for orchestrator unit tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 15:40:26 +00:00
Jun-te Kim
261fae2e79 reformatted to be DDD structure 2026-06-04 14:50:04 +00:00
Jun-te Kim
dfd05ba28b tests files 2026-06-04 11:47:42 +00:00
Khalim Conn-Kowlessar
b3f4609c2d feat(modelling): wire Valuation Uplift onto the Plan
The Plan derives its Valuation Uplift (ADR-0018) from its baseline -> post
band jump and works+contingency cost, given one external input — the
Property's current market value (a Property Valuation, mostly absent).
`Plan.valuation` / `Plan.baseline_epc_rating` are derived like the other
headline figures; `PlanModel.from_domain` maps the £ forms to the live
plan.valuation_* columns (NULL when no value — the percentage is not
persisted on those columns). `Property.current_market_value` is the new
optional source; the orchestrator threads it onto the Plan. `run_one`
takes a `current_market_value` so the harness can value the uplift, and
the sense-check table shows the average % (always) plus the £ forms when
known.

Sourcing the current market value (upload / default) remains deferred
(ADR-0018); it is None throughout until that lands, so the columns stay
NULL at scale.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 08:59:04 +00:00
Khalim Conn-Kowlessar
31da90f5eb feat(modelling): persist recommendation.material_id from the catalogue
Expand half of the recommendation_materials retirement (ADR-0017). A
Plan Measure installs a single Product, so thread its catalogue id end to
end — Product.id -> MeasureOption.material_id -> PlanMeasure.material_id
-> recommendation.material_id — replacing the per-material BOM child
table with one nullable column on the row. ProductPostgresRepository
reads the id from MaterialRow; the four fabric generators set it on their
Option; the orchestrator carries it onto the Plan Measure; the mirror
declares + maps the column. Optional throughout (the JSON stopgap
catalogue carries no ids -> NULL).

The multi-measure integration test now pins each persisted measure's
material_id to its seeded MaterialRow id. Migration spec (live column
must be added before this deploys; contraction is the owner's next step)
in docs/migrations/recommendation-material-id.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 08:26:58 +00:00
Daniel Roth
1c1642c1c5 add TODO comment 2026-06-04 08:19:16 +00:00
Khalim Conn-Kowlessar
b976c3abd2 feat(modelling): attribute per-measure bill savings via a telescoping cascade
`_plan_for` now scores the baseline + every cumulative prefix once
(`cascade_scores`, best-practice order) and reuses those Scores for both the
role-3 marginal attribution and a per-measure bill cascade: bill each prefix at
one Fuel Rates snapshot and take consecutive Bill deltas as each measure's
marginal delivered-kWh and £ saving. Saving is signed (ventilation is
negative) and telescopes exactly to the Plan headline savings, because the
Plan's baseline/post Bills are now the same cascade endpoints (`bills[0]` /
`bills[-1]`) — which also drops the redundant standalone baseline `calculate`.

`recommendation.kwh_savings` / `energy_cost_savings` are filled from these.
Adds `Bill.total_consumption_kwh` (shared by Plan + the orchestrator). Pinned
end-to-end on the real calculator: Σ per-measure savings == the Plan totals
(ADR-0014 amendment).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 18:01:11 +00:00
Khalim Conn-Kowlessar
198122d145 feat(modelling): derive + persist plan-level post-retrofit bills (#1152 follow-up)
ModellingOrchestrator gains a constructor-injected FuelRatesRepository (mirrors
Baseline): run() resolves get_current() once and reuses one BillDerivation across
the batch. _plan_for prices the baseline and post-package end-states from the
SapResults already on their Scores (no extra calculate) and passes the Bills to
Plan. PlanRow mirror + from_domain gain the four live columns post_energy_bill /
energy_bill_savings / post_energy_consumption / energy_consumption_savings.
Pipeline/handler wire the fuel-rates repo. Integration tests assert the columns
persist: the multi-measure (fallback) plan shows positive bill+consumption
savings; the already-at-target zero-measure plan shows the current bill with
exactly zero savings. Fuel-switch measures price at the new fuel for free (we
bill the simulated end-state). 183 modelling/billing/orchestration/repo tests
pass, pyright strict clean. Plan-level only; per-measure savings next.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 17:30:47 +00:00
Daniel Roth
174ef26075 refactor magicplan in ddd structure 2026-06-03 17:20:20 +00:00
Khalim Conn-Kowlessar
ced6287baa refactor(billing): relocate Bill Derivation to domain/billing/ (cross-stage)
Bill / EnergyBreakdown / BillDerivation / sap_fuel were under
domain/property_baseline/ only because Baseline was built first. The Modelling
stage now needs them too, so move them (and their tests) to a neutral
domain/billing/ — Fuel/FuelRates already live in the shared domain/fuel_rates/.
Avoids a modelling -> property_baseline cross-stage import and a package name
that wrongly implies ownership (ADR-0011, ADR-0014 amendment). Pure git mv +
import rewrite across 10 files; 40 billing/baseline/repo tests pass, pyright
strict clean. CONTEXT.md Bill Derivation location updated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 17:19:23 +00:00
Khalim Conn-Kowlessar
84ec6da032 refactor(modelling): group domain/modelling into generators/scoring/optimisation
domain/modelling/ had grown to 15 flat modules. Group the behavioural ones into
subpackages — generators/ (wall/roof/floor Recommendation Generators), scoring/
(overlay applicator, package scorer, role-1/3 scoring), optimisation/ (optimiser
+ measure dependency) — and leave the shared value-object vocabulary
(recommendation, plan, scenario, product, contingencies, simulation) flat at the
top, since it is imported everywhere. Pure move + import-path rewrite across 89
import sites; no behaviour change. 136 pass, pyright strict clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 13:48:36 +00:00