Commit graph

6527 commits

Author SHA1 Message Date
Daniel Roth
e7954ad83a
Merge pull request #1210 from Hestia-Homes/bug/audit-generator-deploy-fix
Update subtask_handler to work for handlers where TaskOrchestrator is optional
2026-06-10 14:04:20 +01:00
Daniel Roth
921dc5108b revert pytest.ini 2026-06-10 12:59:06 +00:00
Daniel Roth
44b4882841 Merge branch 'main' into bug/audit-generator-deploy-fix 2026-06-10 12:53:18 +00:00
Daniel Roth
8ff58bd645 tell subtask_handler whether to send TaskOrchestrator to handler, defaulting to True 2026-06-10 12:43:24 +00:00
Khalim Conn-Kowlessar
07f534ee11 feat(modelling): system tune-up options (standard + zone controls)
Add the system tune-up to the heating Recommendation: keep the existing wet
boiler but install better heating controls and fix the cylinder. Two competing
Options (the Optimiser picks <=1 across the whole heating rec) per the user's
two best control end-states:

- system_tune_up        — standard controls (programmer + room thermostat +
  TRVs, SAP 10.2 Table 4e code 2106)
- system_tune_up_zoned  — time-and-temperature zone control (code 2110, type 3):
  more SAP uplift for more cost

Both keep the boiler (no fuel / SAP code / flue change), set the control
ABSOLUTELY to their end-state, and apply the conditional cylinder fixes (an
80 mm jacket when under-insulated, a thermostat when absent — only when a
cylinder exists). Each control option is offered only when it genuinely improves
the existing control — standard is skipped when the control is already 2106 /
2110 / 2112, zone when already 2110 / 2112 — so neither is ever a downgrade or a
no-op.

Validated against the Elmhurst "system tune up" re-lodgements (cert 001431):
nine befores spanning controls 2101-2113 all converge to the two common afters,
proving the control overlay is absolute. The cascade pin is parametrised over
two starting controls (2101 "no control" + 2113 "room thermostat and TRVs") x
both afters, delta 0 (SAP/CO2/PE).

Wires the two MeasureTypes through contingencies (0.15), the offline catalogue
(500 / 900), the catalogue-coverage list, the report triggers, and the ARA
first-run seed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 10:20:46 +00:00
Daniel Roth
3ceafe8b49
Merge pull request #1208 from Hestia-Homes/bug/audit-generator-task-orchestrator
Pass unused TaskOrchestrator to audit generator handler to avoid TypeError
2026-06-10 10:36:18 +01:00
Daniel Roth
5c4999aade unused task orchestrator in handler signature 2026-06-10 09:30:38 +00:00
Daniel Roth
2c605f80ca pass task orchestrator to handler 2026-06-10 09:29:46 +00:00
Khalim Conn-Kowlessar
3d108a9d9e feat(harness): explain gas_boiler_upgrade in the report triggers
Add the `gas_boiler_upgrade` branch to `report._triggers_for`, mirroring the
generator's eligibility guard so a cohort report explains why the boiler upgrade
fired: the wet-boiler SAP code, the mains-gas connection that makes the gas
end-state installable, and the cylinder presence that shapes the bundle (combi
vs regular + cylinder fixes).

No golden API cert selects the boiler upgrade (it competes with — and on houses
loses to — the ASHP bundle within the one heating Recommendation), so the branch
is covered by a direct `_triggers_for` unit test, following the repo pattern for
testing internal helpers (cert_to_inputs).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 08:44:37 +00:00
Daniel Roth
bb1c8c88ce Two-parameter subtask handler completes without TypeError 🟩 2026-06-10 08:40:32 +00:00
Khalim Conn-Kowlessar
2413bc87da feat(modelling): solid-fuel(coal)->gas boiler upgrade + boiler_flue_type end-state
Pin the coal-boiler-with-cylinder upgrade and add the `boiler_flue_type`
end-state field. A solid-fuel (coal) boiler (fuel 11, SAP code 153) on a
mains-gas street converts to a gas condensing boiler (fuel 11->26, code 102) —
the non-gas->gas path for a solid-fuel system, eligible because code 153 is in
the wet-boiler solid-fuel range 151-161 and mains gas is present.

New `boiler_flue_type` HeatingOverlay field, routed to main_heating_details[0]
and set to 2 (room-sealed/balanced) on both boiler shapes: every relodged after
lodges flue type 2, but coal's before lodged none. The field is SAP-inert (the
cascade score is unchanged by it), so it is written purely for end-state
fidelity — the overlay now represents the installed condensing boiler's flue.
Validated via the overlay-equality unit tests.

The coal after predates the user-locked "always add a cylinder thermostat when
absent" rule, so it stale-lodged thermostat 'N'; the pin corrects it to the
rule's end-state 'Y' in-test (the gas with-cylinder after got the same
correction by re-lodging). The cylinder is already 80 mm insulated, so the
jacket is skipped and only the thermostat is added; controls (2106) are
unchanged. Cascade-pinned delta 0 (SAP/CO2/PE).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 08:27:07 +00:00
Daniel Roth
51cf545776 Two-parameter subtask handler completes without TypeError 🟥 2026-06-10 08:22:43 +00:00
Jun-te Kim
14dc4efeed save so i can continue 2026-06-09 17:07:55 +00:00
Khalim Conn-Kowlessar
7bc9797a26 test(modelling): pin non-gas->gas + already-insulated-cylinder boiler upgrades
Two more boiler-upgrade cascade pins, validating the existing generator across
fuels and cylinder states (no source change):

- oil combi: an oil boiler (fuel 28, code 130) on a mains-gas street converts to
  a gas condensing combi (fuel 28->26, code 104). Proves the non-gas -> gas
  conversion gated on a mains-gas connection (ADR-0024 revised).
- already-insulated cylinder: a gas boiler heating a pre-jacketed cylinder
  (type 2 / 80 mm, no thermostat) gets a new boiler + a thermostat, with the
  jacket NOT re-applied. Proves the cylinder path's skip-jacket branch against a
  real cert. (Sourced from an LPG re-lodgement whose fuel the Summary mapper
  reads as mains gas 26 — a separate LPG fuel-mapping gap, noted in the test.)

Both pin delta 0 (SAP/CO2/PE) against the relodged after.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 16:37:32 +00:00
Khalim Conn-Kowlessar
63dd69ff8b feat(modelling): gas combi boiler upgrade + controls-when-inadequate
Extend the gas-boiler-upgrade Option to combi (no-cylinder) dwellings and add
the controls upgrade shared by both boiler shapes. A dwelling has a cylinder or
it does not, so the one `gas_boiler_upgrade` Option is shaped per dwelling:

- no cylinder -> a gas condensing combi (Table 4b code 104), no cylinder fields
  touched;
- a cylinder  -> a regular boiler (code 102) heating it, with the conditional
  cylinder jacket/thermostat (slice 1).

Controls: bring an inadequate boiler control up to full programmer + room
thermostat + TRVs (SAP 10.2 Table 4e Group 1 code 2106). "Inadequate" = the
Group-1 codes with NO room thermostat (2101, 2102, 2107, 2108, 2109, 2111) —
these lack boiler interlock (Table 4c(2) / footnote c) p.171), so adding a room
thermostat genuinely improves SAP. Room-thermostatted (2103/2104/2105/2106/2113)
or better zone controls (2110/2112) are left unchanged — never downgraded, so
no phantom uplift. The with-cylinder cert (control 2106) is therefore untouched
and its pin still holds at delta 0.

Validated by the combi before/after re-lodgement (cert 001431, gas boiler
upgrade - no cylinder): control 2111 "TRVs and bypass" -> 2106, fan flue
False->True, SAP code 112 -> 104. Cascade-pinned delta 0 (SAP/CO2/PE). Removed
the slice-1 placeholder test asserting no boiler Option fires without a cylinder
(the combi Option now correctly fires there).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 16:28:00 +00:00
Khalim Conn-Kowlessar
31c74ab500 feat(modelling): gas-boiler-upgrade-with-cylinder option in the heating rec
Add the first boiler-upgrade option to the single "Heating & Hot Water"
Recommendation (ADR-0024 expansion): a dwelling whose existing wet boiler heats
a hot-water cylinder is offered a new gas condensing boiler, with the cylinder
jacketed when under-insulated and given a thermostat when absent. One competing
Option (the Optimiser picks <=1), folded into one composite Plan line.

The end-state is read from the Elmhurst before/after re-lodgements (cert 001431,
gas boiler upgrade - with cylinder), which REVISE ADR-0024:

- Target is always a gas condensing boiler, not fuel-preserving: every after
  lodges fuel 26. Gas->gas always; a non-gas wet boiler ->gas only with a
  mains-gas connection; electric boilers are left alone (electrification is the
  upgrade path). Eligibility = wet-boiler SAP code (Table 4a/4b 101-141 /
  151-161 / 191-196) + not an electric boiler + mains gas present.
- End-state is a Table 4b SAP code, not a PCDB index: code 102 (regular boiler
  + cylinder). The calculator derives the condensing seasonal efficiency from
  the code, so no efficiency input exists or is needed.
- A modern condensing boiler has a fanned flue: the after flips
  `fan_flue_present` False->True on every cert (SAP 10.2 Table 4f flue-fan +
  the Table 4b condensing-efficiency basis). Added as a new HeatingOverlay
  field, routed to main_heating_details[0].
- Cylinder thermostat is always added when absent (user-locked); the jacket is
  the 80 mm `cylinder_insulation_type=2` end-state, applied only when the
  cylinder is below 80 mm (never downgrading a better one). Both are conditional
  per-dwelling components, not a frozen overlay.

Cascade-pinned delta-0 (SAP/CO2/PE) against the relodged after via
`_assert_overlay_reproduces_after`. NB the absolute SAP on this dwelling is
subject to a separate Summary-path mapper roof-fidelity gap (we read the roof
better-insulated than Elmhurst, scoring ~75 vs the printed 56); the gap is
identical on before+after (the boiler measure never touches the roof) so it
cancels and the pin still proves the exact heating field-delta. Tracked on the
calculator branch.

Wires the new `gas_boiler_upgrade` MeasureType through contingencies (0.26),
the offline sample catalogue, the catalogue-coverage list, and the ARA
first-run integration seed (the option fires on any mains-gas boiler+cylinder
dwelling).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 16:16:46 +00:00
Jun-te Kim
3b7d26fe34 added test for a 1000 examples 2026-06-09 16:02:21 +00:00
Daniel Roth
f6a4b0f722
Merge pull request #1206 from Hestia-Homes/bug/audit-generator-deploy-fix
Add missing variable to lambda variables
2026-06-09 16:39:34 +01:00
Daniel Roth
c1dda41857 add missing variable to lambda variables 2026-06-09 15:33:35 +00:00
Daniel Roth
63b1fa08a8
Merge pull request #1204 from Hestia-Homes/feature/generate-ventilation-audit-from-magicplan
New application to generate ventilation audit file for a given hubspot deal ID
2026-06-09 16:25:12 +01:00
Daniel Roth
dcd5204b54 put db engine construction inside handler to avoid import errors in test 2026-06-09 15:18:42 +00:00
Daniel Roth
236f33c25f move spreadsheet population logic to domain 2026-06-09 14:43:24 +00:00
Daniel Roth
48a413a5e2 correct imports in postgres repo 2026-06-09 14:25:29 +00:00
Daniel Roth
94f8ef5458 Merge branch 'main' into feature/generate-ventilation-audit-from-magicplan 2026-06-09 14:22:57 +00:00
Daniel Roth
163703285f deployment 2026-06-09 14:15:52 +00:00
Daniel Roth
b9eb23f6df allow write to real s3 when running locally 2026-06-09 13:52:39 +00:00
Daniel Roth
608006d977 reapply conditional formatting to populated file 2026-06-09 13:29:11 +00:00
Daniel Roth
f8c955b2d3 local runner and correct template path 2026-06-09 13:03:05 +00:00
Daniel Roth
e010fa1b40 remove redundant files 2026-06-09 12:52:04 +00:00
Daniel Roth
e11ab85d9d
Merge pull request #1202 from Hestia-Homes/bug/magicplan-task-handler
Use correct task handler decorator for magicplan
2026-06-09 13:36:12 +01:00
Daniel Roth
ea899fe0ab actually revert changes 2026-06-09 12:27:57 +00:00
Daniel Roth
ade6891ff0 revert accidental changes 2026-06-09 12:24:12 +00:00
Daniel Roth
31c2e07a02 revert accidental changes to magicplan repo 2026-06-09 12:19:39 +00:00
Daniel Roth
dcf0b2ccb9 use task_handler rather subtask_handler for magicplan 2026-06-09 12:17:57 +00:00
Daniel Roth
82156fac8f Audit generator populates XLSX, uploads to S3, and records UploadedFile row 🟪
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 12:05:55 +00:00
Daniel Roth
612d522b35 Audit generator populates XLSX, uploads to S3, and records UploadedFile row 🟩
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 12:01:08 +00:00
Daniel Roth
a1d09aa880 Audit generator populates XLSX, uploads to S3, and records UploadedFile row 🟥
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 11:59:09 +00:00
Daniel Roth
f08a75e103 UploadedFilePostgresRepository returns latest uploaded file by deal ID and type 🟩
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 11:54:14 +00:00
Daniel Roth
53f0da8666 UploadedFilePostgresRepository returns latest uploaded file by deal ID and type 🟥
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 11:53:25 +00:00
Daniel Roth
5178cd02c5 UploadedFile, FileTypeEnum, FileSourceEnum importable from infrastructure.postgres.uploaded_file_table 🟩
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 11:50:51 +00:00
Daniel Roth
41b282042f UploadedFile, FileTypeEnum, FileSourceEnum importable from infrastructure.postgres.uploaded_file_table 🟥
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 11:42:53 +00:00
Jun-te Kim
06cb4f7b6e Merge branch 'feature/bill-derivation' into feature/junte+khalim 2026-06-09 10:06:40 +00:00
Khalim Conn-Kowlessar
4006753620 fix(scripts): authenticate the EPC client with OPEN_EPC_API_TOKEN
The new gov EPC API (api.get-energy-performance-data..., Bearer auth) returns
403 "Bad authentication header" with EPC_AUTH_TOKEN but 200 with
OPEN_EPC_API_TOKEN — the token name is misleading (it is the Bearer token for
the new API, not the open-data API). Verified live against
/api/domestic/search. Unblocks the live EPC fetch in run_modelling_e2e.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 20:58:22 +00:00
Khalim Conn-Kowlessar
0f6077a830 feat(scripts): DB-catalogue local run + optional --persist for run_modelling_e2e
Slice 5 (local run sources the DB, read-only) + slice 6 (optional persist),
landing together as one script rewrite (the persist path is interleaved with
the compute path).

The same local computation now runs whether or not the result is stored:
- Both modes price against the live `material` catalogue (read-only
  ProductPostgresRepository over one shared Session) and model against a real
  Scenario read from the DB (--scenario-id; its goal_value drives the band,
  rejected if null) — so the inspected recommendations are exactly what gets
  stored. The JSON sample catalogue is no longer used by this script.
- --measures restricts the run to a comma-separated considered_measures
  allowlist (e.g. high_heat_retention_storage_heaters,solar_pv).
- --persist writes the inputs (EPC + spatial + solar) and the *same* computed
  Plan via the production repos in one PostgresUnitOfWork, then commits
  (idempotent: PlanPostgresRepository replaces by (property_id, scenario_id)).
  Gated: --persist requires --scenario-id and --portfolio-id. Default is
  inspect-only — no DB writes.

harness.console.run_modelling gains `products` and `scenario` overrides (the
seam the script drives); defaults unchanged, so existing callers are
unaffected. Suite 257 pass + 3 xfail; pyright clean; --help/guard/measure
parsing verified. Not yet executed against the DB (awaiting property_ids +
write-confirm).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 20:45:50 +00:00
Khalim Conn-Kowlessar
62e1d4b813 fix(product): deterministic catalogue pick by ordering get() by id
ProductPostgresRepository.get took .first() with no ORDER BY, so when a
measure type has several active material rows (the live catalogue holds 74
solar_pv, 5 high_heat_retention_storage_heaters) the chosen row — hence the
cost and material_id — depended on the database's physical row order. Order by
id so a re-seed prices the same product every time.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 20:35:45 +00:00
Khalim Conn-Kowlessar
7942a8101a feat(modelling): considered_measures allowlist on the orchestrator
Add domain/modelling/considered_measures.py::restrict_to_considered_measures —
the pure allowlist that limits a run to a chosen set of MeasureType (mirroring
the legacy engine's `inclusions`). It filters at the Option level, so a
multi-option Recommendation (e.g. Heating & Hot Water competing HHRSH against
an ASHP bundle) is kept with only its allowed Options; a Recommendation left
with none is dropped. None = consider everything (unrestricted default).

Thread `considered_measures: frozenset[MeasureType] | None` through
ModellingOrchestrator.run -> _plan_for -> _scored_candidate_groups /
_candidate_recommendations (applies the filter) and _measure_dependencies
(suppresses a forced dependency whose required measure is outside the
allowlist, so a restricted run forces nothing it is not considering). The
local-run seam (harness.console.run_modelling) gains the same param.

The Optimiser still freely chooses among survivors — including none. Tests:
the pure filter (3 cases) + an orchestrator-seam test proving a
{solar_pv}-restricted run yields only solar_pv options. 257 pass + 3 xfail;
pyright clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 20:32:11 +00:00
Khalim Conn-Kowlessar
9ef97be958 refactor(modelling): type measure_type fields as MeasureType
Tighten the recommendation/plan vocabulary off generic str:
MeasureOption.measure_type and PlanMeasure.measure_type are now MeasureType
(also _GlazingTarget.measure_type, MeasureDependency.triggers ->
frozenset[MeasureType], and the optimiser's chosen/required-type locals).
Because MeasureType is a StrEnum the change is transparent to persistence
(the `recommendation` varchar column), the optimiser group-by key, and every
`== "solar_pv"`-style comparison — so pyright now enforces the enum at every
construction site with no runtime behaviour change.

The catalogue boundary stays str: ProductRepository.get(measure_type: str)
and Product.measure_type are unchanged (they map arbitrary DB/JSON rows), so
the fake product repos in tests need no edit. Test construction helpers coerce
their str arg via MeasureType(...); direct constructions use members.

Suite green: tests/domain/modelling + orchestration + harness 253 pass + 3
xfail; pyright clean on production + tests (pre-existing moto + property-
override-rowcount baselines untouched).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 20:13:31 +00:00
Khalim Conn-Kowlessar
d58ac60d29 feat(modelling): MeasureType StrEnum as the canonical measure vocabulary
Introduce domain/modelling/measure_type.py — a StrEnum with one member per
modelled measure (the 15 the generators emit). A StrEnum so each member *is*
its string value: it persists straight into the `recommendation` varchar
column, is the optimiser's group-by key, and compares equal to the catalogue /
EPC strings — so it replaces the per-generator string constants with no
persistence or optimiser change.

Repoint every generator's measure-type constant/literal to a MeasureType
member (wall, solid_wall, roof, floor, glazing, lighting, ventilation,
heating, solar). Field annotations stay `str` for now; tightening them to
MeasureType is the next slice.

This is the enum the historical engine deferred (engine.py:970
"TODO - formalise property measure types into an enum") and the vocabulary the
forthcoming `considered_measures` allowlist will speak (mirroring the legacy
`inclusions`).

Suite green: tests/domain/modelling + orchestration + harness 253 pass + 3
xfail; pyright clean on the enum + generators.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 19:54:04 +00:00
Jun-te Kim
b48700e964 Merge branch 'main' into feature/junte+khalim 2026-06-08 16:56:15 +00:00
Daniel Roth
e5e67c203b temp commit to backup prd in case laptop dies 2026-06-08 16:48:37 +00:00