Commit graph

7203 commits

Author SHA1 Message Date
Khalim Conn-Kowlessar
dfba20babf docs: session-8 handover — API mechanical_ventilation enum → §2 MV-kind dispatch
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 23:02:51 +00:00
Khalim Conn-Kowlessar
71b378b9e5 fix(ventilation): map API mechanical_ventilation enum to §2 MV-kind dispatch
The profiler flagged `mechanical_ventilation=2` as a clean systematic
over-rate: 20 certs, signed +1.90 SAP, only 5% within 0.5 (every one
positive). Root cause: the API path (`from_api_response`) dropped the
doc-level `mechanical_ventilation` field, so `sap_ventilation.
mechanical_ventilation_kind` was always None and the §2 cascade
defaulted to NATURAL — under-stating the ventilation air-change rate
(and hence heat loss) for every mechanical system. (Only the Elmhurst/
Summary path mapped it, via `_ELMHURST_MV_TYPE_TO_KIND`.)

RdSAP-Schema-21 `mechanical_ventilation` enum (epc_codes.csv) →
MechanicalVentilationKind picking the SAP 10.2 §2 (24a..d) effective-ach
formula:
  0 natural                      -> NATURAL (24d)
  1 MV (no heat recovery)        -> MV (24b)
  2 mechanical extract, dc (MEV) -> EXTRACT_OR_PIV_OUTSIDE (24c)
  3 mechanical extract, c (MEV)  -> EXTRACT_OR_PIV_OUTSIDE (24c)
  5 positive input from loft     -> NATURAL (loft-sourced PIV adds no
                                    system air change per RdSAP 10 §2.6)
  6 positive input from outside  -> EXTRACT_OR_PIV_OUTSIDE (24c)
Code 4 (MVHR, 24a) is DEFERRED — its formula needs the lodged
heat-recovery efficiency (PCDB Table 326) the API→cascade path doesn't
yet plumb; mapping it to MVHR with a null efficiency would mis-model it
as MV, so it stays NATURAL (3 scattered certs, accurate at the median).
Unmapped integers raise `UnmappedApiCode` (mirror of `_api_sheltered_
sides` / `_api_type_1_gable_kind`).

Eval: the extract cohort (mech_vent 2/3/6) moved +1.90 -> +0.9 median
(within-0.5 5% -> 35%); 20 improved / 3 regressed (offsetting). Headline
within-0.5 54.24% -> 55.01%, within-1.0 69.64% -> 70.08%, mean|err|
1.248 -> 1.233, 909 computed / 0 raises. The +0.9 residual on MEV is the
fan electricity (§2.6.4 SFP, PCDB Table 322) — a separate follow-up.
2 AAA tests; goldens + full calc/epc/parser regression green; pyright
net-zero.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 23:02:04 +00:00
Khalim Conn-Kowlessar
943f83ed01 docs: session-7 handover — sheltered alternative walls (RdSAP Table 4 R=0.5)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 22:50:55 +00:00
Khalim Conn-Kowlessar
3e05c95e65 fix(wall-U): apply RdSAP Table 4 "Sheltered" R=0.5 to alternative walls
Audit of the API-SAP error (53% within 0.5) localised the systematic
under-rate to ELECTRIC FLATS (houses sit at 60-66% within 0.5; electric
flats 13-19%). Decomposing the flat error showed it tracks space-heating
demand per m² — the worst certs reach 130-289 kWh/m² (accurate certs sit
at 14-110), i.e. a grossly over-stated fabric heat loss, amplified ~4x by
the electricity unit price and the steep low-band SAP log curve.

Root cause: the gov-EPC API lodges `sheltered_wall="Y"` on alternative
wall sub-areas (a sub-area adjacent to an unheated buffer — stair core,
adjoining structure), but the field was dropped by the schema + domain
dataclasses and the calculator billed the alt sub-area at its full
exposed U. RdSAP 10 Table 4 (PDF p.22) "Sheltered": such a wall carries
an added external surface resistance R=0.5 m²K/W → U_sheltered =
1/(1/U + 0.5) — the SAME adjustment the main wall already applies for
`gable_wall_type=2` (`gable_wall_sheltered`,
`_SHELTERED_GABLE_ADDED_RESISTANCE_M2K_W`). Cert 0340-2976 (band-A flat,
42 m² sheltered timber-frame alt) over-stated its wall channel by
~58 W/K → walls 128 -> 70 W/K.

Threads the field end-to-end: schema dataclasses (21.0.0/21.0.1) +
domain `SapAlternativeWall.is_sheltered` (default False — the Summary/
Elmhurst path leaves it False, sheltering rides through its lodged
U-value there, so goldens are untouched) + `from_api_response` mapping
`"Y"->True` + `_alt_wall_w_per_k` applying the 0.5 resistance on the
cascade path (lodged-U and basement alt-walls return before it).

140 certs (15% of the corpus) carry a sheltered alt-wall; they under-
rated at median -0.82 / mean signed -1.33 / 23% within 0.5. Eval: 102
improved, 38 regressed (offsetting-error cases — fix is spec-uniform per
[[feedback_software_no_special_handling]]); within-0.5 53.14% -> 54.24%,
within-1.0 67.99% -> 69.64%, within-2.0 81.85% -> 83.50%, mean|err|
1.312 -> 1.248, 909 computed / 0 raises. Goldens (6035, 000565) and full
calc/epc/parser regression green; pyright net-zero.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 22:49:42 +00:00
Khalim Conn-Kowlessar
e1adc8d3d5 docs: session-6 handover — community fuel collision (waste/biomass/biogas) fixed gated on heat-network
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 21:56:56 +00:00
Khalim Conn-Kowlessar
a7761ea83f fix(fuel): map gov-API community fuels 30/31/32 (waste/biomass/biogas) to Table-12 community rows, gated on heat-network context
The gov-API `main_fuel_type`/`water_heating_fuel` enum (epc_codes.csv)
codes 30="waste combustion (community)", 31="biomass (community)",
32="biogas (community)" collide in VALUE with the Table-32 electricity
codes 30 (standard rate), 31 (7-hour low) and 32 (7-hour high). All three
sit in `_ELECTRIC_FUEL_CODES`, so `is_electric_fuel_code` flagged a
community-scheme main as electric and `_is_electric_main` routed its cost
through the off-peak electricity branch — BYPASSING the heat-network rate
in `_heat_network_factor_fuel_code`. Cert 8536 (biomass community, SAP
code 301) was billing at 5.5 p/kWh grid electricity instead of the 4.24
p/kWh heat-network rate → -17.2 SAP.

Per RdSAP 10 §C / SAP 10.2 Table 12 (PDF p.191) the community
waste/biomass/biogas rows are codes 42/43/44 (the same rows the
backwards-compat enum codes 11/12/13 already map to). Add 30->42, 31->43,
32->44 to both API fuel-translation tables.

The remap CANNOT be global (`canonical_fuel_code`): the cascade uses the
bare Table-32 code 30 internally as `_STANDARD_ELECTRICITY_FUEL_CODE`
(the RdSAP no-water-heating immersion default writes
`water_heating_fuel=30`), so a blanket remap mis-prices genuine grid
electricity as community waste (cert 2211 regressed +16 SAP in a
prototype). Instead `_heat_network_community_fuel_code` translates only
when `_is_heat_network_main` is true, at the `_main_fuel_code` /
`_water_heating_fuel_code` fuel-TYPE boundary, where the community
meaning is unambiguous.

Per the strict-raise principle ([[reference-unmapped-sap-code]]), a
heat-network main lodging a colliding community fuel the table doesn't
cover raises `UnmappedSapCode` rather than silently falling through to
the same-numbered electricity code.

Eval (API SAP vs lodged): cert 8536 -17.25 -> -6.51, cert 5036 -6.29 ->
+1.36; mean|err| 1.329 -> 1.312, within-1.0 67.88% -> 67.99%,
within-2.0 81.74% -> 81.85%, within-0.5 held at 53.14%, 909 computed /
0 raises. No golden / calculator regressions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 21:55:48 +00:00
Khalim Conn-Kowlessar
4006753620 fix(scripts): authenticate the EPC client with OPEN_EPC_API_TOKEN
The new gov EPC API (api.get-energy-performance-data..., Bearer auth) returns
403 "Bad authentication header" with EPC_AUTH_TOKEN but 200 with
OPEN_EPC_API_TOKEN — the token name is misleading (it is the Bearer token for
the new API, not the open-data API). Verified live against
/api/domestic/search. Unblocks the live EPC fetch in run_modelling_e2e.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 20:58:22 +00:00
Khalim Conn-Kowlessar
0f6077a830 feat(scripts): DB-catalogue local run + optional --persist for run_modelling_e2e
Slice 5 (local run sources the DB, read-only) + slice 6 (optional persist),
landing together as one script rewrite (the persist path is interleaved with
the compute path).

The same local computation now runs whether or not the result is stored:
- Both modes price against the live `material` catalogue (read-only
  ProductPostgresRepository over one shared Session) and model against a real
  Scenario read from the DB (--scenario-id; its goal_value drives the band,
  rejected if null) — so the inspected recommendations are exactly what gets
  stored. The JSON sample catalogue is no longer used by this script.
- --measures restricts the run to a comma-separated considered_measures
  allowlist (e.g. high_heat_retention_storage_heaters,solar_pv).
- --persist writes the inputs (EPC + spatial + solar) and the *same* computed
  Plan via the production repos in one PostgresUnitOfWork, then commits
  (idempotent: PlanPostgresRepository replaces by (property_id, scenario_id)).
  Gated: --persist requires --scenario-id and --portfolio-id. Default is
  inspect-only — no DB writes.

harness.console.run_modelling gains `products` and `scenario` overrides (the
seam the script drives); defaults unchanged, so existing callers are
unaffected. Suite 257 pass + 3 xfail; pyright clean; --help/guard/measure
parsing verified. Not yet executed against the DB (awaiting property_ids +
write-confirm).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 20:45:50 +00:00
Khalim Conn-Kowlessar
62e1d4b813 fix(product): deterministic catalogue pick by ordering get() by id
ProductPostgresRepository.get took .first() with no ORDER BY, so when a
measure type has several active material rows (the live catalogue holds 74
solar_pv, 5 high_heat_retention_storage_heaters) the chosen row — hence the
cost and material_id — depended on the database's physical row order. Order by
id so a re-seed prices the same product every time.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 20:35:45 +00:00
Khalim Conn-Kowlessar
87485bbe3d docs: session-5 handover — fuel-code collision fix (anthracite/coal)
Records the fuel-type-boundary canonicalisation, the goldens-caught
constraint (code 33 is also the electricity-10h tariff code), and the
deferred dual-fuel/community/fabric follow-ups.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 20:32:32 +00:00
Khalim Conn-Kowlessar
7942a8101a feat(modelling): considered_measures allowlist on the orchestrator
Add domain/modelling/considered_measures.py::restrict_to_considered_measures —
the pure allowlist that limits a run to a chosen set of MeasureType (mirroring
the legacy engine's `inclusions`). It filters at the Option level, so a
multi-option Recommendation (e.g. Heating & Hot Water competing HHRSH against
an ASHP bundle) is kept with only its allowed Options; a Recommendation left
with none is dropped. None = consider everything (unrestricted default).

Thread `considered_measures: frozenset[MeasureType] | None` through
ModellingOrchestrator.run -> _plan_for -> _scored_candidate_groups /
_candidate_recommendations (applies the filter) and _measure_dependencies
(suppresses a forced dependency whose required measure is outside the
allowlist, so a restricted run forces nothing it is not considering). The
local-run seam (harness.console.run_modelling) gains the same param.

The Optimiser still freely chooses among survivors — including none. Tests:
the pure filter (3 cases) + an orchestrator-seam test proving a
{solar_pv}-restricted run yields only solar_pv options. 257 pass + 3 xfail;
pyright clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 20:32:11 +00:00
Khalim Conn-Kowlessar
19235d1144 fix(fuel): canonicalise colliding gov-API solid-fuel codes (anthracite/coal) at the fuel-type boundary
A coal main (gov-API main_fuel_type=33) was priced at the electricity
10-hour low rate (7.5 p) and anthracite (5) at the bulk-LPG rate
(12.19 p), because the shared price/CO2/PE lookups check Table-32/12-code
membership BEFORE translating the API enum — and codes 5/33 collide with
a different-fuel Table code. This drove the cohort's single worst cert
(2100 anthracite, -61 SAP). `is_electric_fuel_code(33)` also wrongly
classified the coal main as electric.

The gov-API fuel enum (confirmed by description-vs-code audit on
main_heating[].description): 5=anthracite, 33=coal, 9=dual-fuel,
20/25/31=community. The collision can't be resolved inside the shared
table functions — code 33 is ALSO the electricity-10h TARIFF code used by
the dual-rate CO2/PE split (golden 000565), so normalising there breaks
electricity certs. Instead `canonical_fuel_code` normalises the colliding
SOLID-fuel enums (5->15 anthracite, 33->11 house coal) at the fuel-TYPE
boundary in `_main_fuel_code` / `_water_heating_fuel_code`, where the code
is known to be a fuel type (never a tariff code).

Scoped to anthracite (5) + coal (33) — the unambiguous large mispricings.
Dual-fuel (9, 0.45 p delta) and community (20/25/31, heat-network path)
are deferred (noted in `_GOV_API_COLLISION_FUELS`).

API SAP eval: mean|err| 1.424 -> 1.329 (the -61 anthracite outlier 2100
-> -11, residual now fabric); within-0.5 53.1% (flat); 909 computed, 0
raises. Golden + Elmhurst regression green (the shared table functions
are unchanged, so the electricity-tariff CO2/PE path is untouched).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 20:31:43 +00:00
Khalim Conn-Kowlessar
9ef97be958 refactor(modelling): type measure_type fields as MeasureType
Tighten the recommendation/plan vocabulary off generic str:
MeasureOption.measure_type and PlanMeasure.measure_type are now MeasureType
(also _GlazingTarget.measure_type, MeasureDependency.triggers ->
frozenset[MeasureType], and the optimiser's chosen/required-type locals).
Because MeasureType is a StrEnum the change is transparent to persistence
(the `recommendation` varchar column), the optimiser group-by key, and every
`== "solar_pv"`-style comparison — so pyright now enforces the enum at every
construction site with no runtime behaviour change.

The catalogue boundary stays str: ProductRepository.get(measure_type: str)
and Product.measure_type are unchanged (they map arbitrary DB/JSON rows), so
the fake product repos in tests need no edit. Test construction helpers coerce
their str arg via MeasureType(...); direct constructions use members.

Suite green: tests/domain/modelling + orchestration + harness 253 pass + 3
xfail; pyright clean on production + tests (pre-existing moto + property-
override-rowcount baselines untouched).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 20:13:31 +00:00
Khalim Conn-Kowlessar
d58ac60d29 feat(modelling): MeasureType StrEnum as the canonical measure vocabulary
Introduce domain/modelling/measure_type.py — a StrEnum with one member per
modelled measure (the 15 the generators emit). A StrEnum so each member *is*
its string value: it persists straight into the `recommendation` varchar
column, is the optimiser's group-by key, and compares equal to the catalogue /
EPC strings — so it replaces the per-generator string constants with no
persistence or optimiser change.

Repoint every generator's measure-type constant/literal to a MeasureType
member (wall, solid_wall, roof, floor, glazing, lighting, ventilation,
heating, solar). Field annotations stay `str` for now; tightening them to
MeasureType is the next slice.

This is the enum the historical engine deferred (engine.py:970
"TODO - formalise property measure types into an enum") and the vocabulary the
forthcoming `considered_measures` allowlist will speak (mirroring the legacy
`inclusions`).

Suite green: tests/domain/modelling + orchestration + harness 253 pass + 3
xfail; pyright clean on the enum + generators.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 19:54:04 +00:00
Khalim Conn-Kowlessar
d90b6f5643 docs: session-5 handover — flat-roof fix + the unknown-insulation principle
Adds the flat-roof slice (52.1 → 53.1%) and records the unifying principle
("unknown insulation → as-built age default, not uninsulated") plus the
cross-element review confirming all element types now conform.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 19:30:52 +00:00
Khalim Conn-Kowlessar
58cff932e6 fix(roof-U): flat roof, undetermined thickness, "no/limited insulation" → Table 18 age default
A flat roof lodged "Flat, no insulation" / "Flat, limited insulation" with
an undetermined insulation thickness ('ND'/'AB' → parsed None) was given the
Table 16 row-0/12mm U (2.30 / 1.50) from the description marker, regardless
of age band. Per RdSAP 10 §5.11.4 (PDF p.44) "U-values in Table 18 are used
when thickness of insulation cannot be determined" — the column (3)
flat-roof age-band default applies. The "no/limited insulation" text is
RdSAP's as-built rendering: at old bands (A-D) the column (3) default IS
2.30 (so those certs are unchanged), but a newer-band flat roof carries the
age-band insulation as built (band H = 0.35, F = 0.68, not 2.30).

Confirmed by the description-vs-rating audit: cert 0390-2753 (top-floor
flat, band H, "Flat, no insulation", thickness 'ND') lodges roof
energy_efficiency_rating = 3 (moderate U), NOT the rating-1 that 2.30
implies — and drove a -31.78 SAP error (roof 202 W/K over 88 m²). Same
masked-at-old-bands structure as the cavity-U fix: accurate at A-D where the
default coincides with 2.30, catastrophic only where it diverges.

Pitched roofs are deliberately NOT rerouted (their "no insulation" text is
load-bearing — the broad 'ND'→Table-18 reroute was empirically net-negative
for pitched lofts).

API SAP eval: 52.1% -> 53.1% within 0.5; <1.0 67.2% -> 68.0%; median |err|
0.475 -> 0.467; mean|err| 1.497 -> 1.424; flat-roof bucket within-0.5
23% -> 35% (11 improved, 2 regressed).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 19:26:03 +00:00
Khalim Conn-Kowlessar
898dcfda18 docs: session-5 handover — as-built cavity-U fix (48.6 → 52.1%)
Adds the cavity wall-U slice to the SESSION-5 block + headline table;
records the by-age-band re-split method that surfaced the G/H spike.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 18:53:01 +00:00
Khalim Conn-Kowlessar
2e466ed1e6 fix(wall-U): as-built "insulated (assumed)" cavity uses Cavity-as-built row, not Filled cavity
An as-built cavity wall (wall_insulation_type=4) lodged "Cavity wall, as
built, insulated (assumed)" was routed to RdSAP 10 Table 6's "Filled
cavity" row. Per Table 6 (England, PDF p.41) the Filled-cavity row carries
the "†" footnote ("assumed as built") only at age bands I-M, where it is
numerically identical to "Cavity as built"; at bands A-H the Filled-cavity
row represents a GENUINE fill, not the as-built assumption. So an as-built
cavity must use the "Cavity as built" row at all bands (band G/H = 0.60,
not the filled 0.35).

This is the same latent A-H bug slice S0380.210 fixed for the "partial
insulation (assumed)" variant but left in place for "insulated (assumed)"
by a legacy production convention. The API SAP-accuracy cohort over-rated
"Cavity wall, as built, insulated (assumed)" certs at bands G/H by a clean
+1.38 / +1.61 SAP median (n=37 / n=18); bands I-M were unaffected (rows
coincide), confirming the spec mechanism per-band.

Retires the `_cavity_described_as_filled` description sniffer — as-built
cavities now always use the as-built row regardless of the rendered
insulation adjective; a genuine retrofit fill is still caught by the
explicit wall_insulation_type=2 branch.

API SAP eval: 48.6% -> 52.1% within 0.5; <1.0 63.8% -> 67.2%; median |err|
0.548 -> 0.475; mean|err| 1.561 -> 1.497; 909 computed, 0 raises.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 18:51:17 +00:00
Jun-te Kim
b48700e964 Merge branch 'main' into feature/junte+khalim 2026-06-08 16:56:15 +00:00
Daniel Roth
e5e67c203b temp commit to backup prd in case laptop dies 2026-06-08 16:48:37 +00:00
Daniel Roth
209a0c401c audit generator application framework 2026-06-08 16:20:05 +00:00
Khalim Conn-Kowlessar
152682d802 docs: session-5 handover — WHC-903 immersion off-peak HW (Table 13) closed
47.6% → 48.6% within 0.5; immersion code mapping corrected (1=dual,
2=single); next robust leads are under-rating flat/party-fabric scatter.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 15:48:25 +00:00
Khalim Conn-Kowlessar
43d4c67d12 fix(hw-cost): WHC-903 immersion off-peak HW bills at Table 13 high-rate fraction
Electric immersion water heating (WHC 903) on an off-peak tariff billed
100% at the low rate, under-costing the dwelling and over-rating it
(median +0.98 SAP across the off-peak WHC-903 API cohort, n=57).

SAP 10.2 Table 12a "Immersion water heater" row (PDF p.191) routes the
water-heating column to Table 13 (PDF p.197): the high-rate fraction is
a function of cylinder volume V, assumed occupancy N (Appendix J Table
1b) and single-/dual-immersion. The remainder bills at the low rate.
Table 13 Note 2 supplies exact equations equivalent to the rounded grid;
`electric_dhw_high_rate_fraction` evaluates them (validated against the
published 110 L grid cells). Per Note 1 the 10-hour equations cover any
tariff with >=10 hours/day low-rate (so 18-/24-hour use that column).

Immersion code mapping CONFIRMED 1=dual, 2=single via RdSAP 10 §10.5
(PDF p.54 — an immersion is "assumed dual" on a dual/off-peak meter)
cross-checked against the API cohort (code 1 sits 3.6:1 on dual meters;
code 2 on single meters). This INVERTS an earlier handover's unverified
"1=single, 2=dual" note — the dual code carries Table 13's small
fraction, matching the cohort over-rating direction; the single mapping
overshot in a prototype.

API SAP eval: 47.6% -> 48.6% within 0.5; <1.0 62.6% -> 63.8%;
mean|err| 1.586 -> 1.561; 909 computed, 0 raises.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 15:45:41 +00:00
Daniel Roth
7497865fb2 Retrieve plan by uploaded_file_id 🟪 2026-06-08 15:44:19 +00:00
Daniel Roth
f8fcf38886 get_plan_by_uploaded_file_id 🟩 2026-06-08 15:37:52 +00:00
Jun-te Kim
0ccb0e0bf9
Merge pull request #1196 from Hestia-Homes/feautre/additional_properties_for_tracking
Feautre/additional properties for tracking
2026-06-08 16:19:34 +01:00
Khalim Conn-Kowlessar
1b4806f8e4 feat(scripts): wire S3 geospatial + Google Solar into run_modelling_e2e
Per Property the inspection script now resolves the UPRN's spatial
reference from the Ordnance Survey Open-UPRN parquet in S3
(GeospatialS3Repository over a boto3 ParquetReader) and threads both
levers into run_modelling:

- planning_restrictions: the conservation/listed/heritage flags that gate
  the wall + solar measures (ADR-0019/0020).
- solar_insights: a live Google Solar buildingInsights fetch keyed on the
  reference coordinates, so the Solar PV Options can fire (ADR-0026).

Mirrors IngestionOrchestrator._fetch's coords->solar flow. Degrades
gracefully per Property: a UPRN S3 doesn't cover -> unrestricted/no-solar;
a point Google has no coverage for (BuildingInsightsNotFoundError) ->
no-solar; both still modelled. --no-solar skips the Google leg. A context
note (restrictions; solar) is printed and written to the md/csv summary.

Verified live: spatial_for + solar fetch round-trip on real UPRNs (S3 via
ambient ~/.aws creds, pyarrow reads parquet bytes). pyright clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 14:55:33 +00:00
Daniel Roth
26ba09a1c7
Merge pull request #1194 from Hestia-Homes/correct-path-to-magicplan-dockerfile
Correct magicplan dockerfile path in terraform
2026-06-08 15:54:47 +01:00
Daniel Roth
e1515b0c00 correct path in terraform 2026-06-08 14:49:41 +00:00
Khalim Conn-Kowlessar
d83c431c7d docs: session-4 handover — interlock + secondary fixes, robust-audit method, open leads
Updates the headline (45.1 → 47.6%), records the four shipped fixes + the
roof-8 false-lead closure, documents the two methods that worked
(description-vs-code audit + outlier-robust categorical sweep by net skew +
median), and lists the open robust leads (whc=903 immersion HW, cat-7 storage,
dual immersion) with the scatter buckets to avoid.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 14:47:31 +00:00
Daniel Roth
c40a7c0af9
Merge pull request #1192 from Hestia-Homes/bug/fix-magicplan-orchestrator-tests
Fix magicplan tests to reflect changes to the orchestrator
2026-06-08 15:44:01 +01:00
Jun-te Kim
a1b4bf4e98 added 4 deal proeprties 2026-06-08 14:42:06 +00:00
Daniel Roth
648e726ca2 fix orchestration tests 2026-06-08 14:40:14 +00:00
Khalim Conn-Kowlessar
faf29942ba fix(secondary): apply Table 11 secondary when lodged via description only (§A.2.2)
`_secondary_fraction` keyed "has a secondary" off the integer
`secondary_heating_type` code. The gov-API path surfaces the secondary as a
DESCRIPTION instead (`secondary_heating.description`, e.g. "Portable electric
heaters (assumed)") and leaves the integer code None. So a gas/oil boiler
main (not in the §A.2.2 forced-secondary set) with an assumed portable-electric
secondary dropped the secondary entirely (sec_kWh=0), under-costing the
dwelling and over-rating its SAP.

Per RdSAP §A.2.2 / SAP 10.2 Table 11, a lodged secondary is costed at its
Table 11 fraction (cat-2 boiler = 0.10, billed at standard-rate electricity per
the §A.2.2 assumed portable-electric default). New
`_has_lodged_secondary_description` treats a real `secondary_heating.description`
as a lodged secondary; passed to `_secondary_fraction` at both call sites. The
description is authoritative — same lesson as floor_heat_loss / roof codes.
(Electric-storage mains were unaffected: they force the secondary already.)

Also adds the Table 11 fraction for main_heating_category=8 (electric underfloor,
"Integrated storage/direct-acting electric systems" = 0.10) — the strict-raise
surfaced this latent gap once cat-8 mains were routed through the lookup.

Eval: 909 computed, 0 raises, 46.9% -> 47.6% within 0.5 (+13 certs: 420 -> 433),
mean|err| 1.633 -> 1.586. 13 improved / 1 regressed (2610, a cat-10 room-heater
cert with an independent over-count). Bucket "Portable electric heaters"
median +2.73 -> ~0 on the gas/cat-2 subset (cat-7 storage was already correct).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 14:28:22 +00:00
Khalim Conn-Kowlessar
0918dd37ec feat(scripts): run_modelling_e2e — inspect recommendations per property_id
Revives the local recommendation-inspection flow for specific Properties.
`scripts/run_modelling_e2e.py` reads each Property's UPRN from the DB
(read-only), fetches the latest EPC live from the gov EPC API by UPRN, runs the
Modelling stage in memory (all Generators → Optimiser → costed, attributed
Plan), and prints a per-Property plan table + writes a Markdown/CSV summary.
Persists nothing — purely for inspection.

The local DB's Properties have no linked ingested EPC (epc_property.property_id
is NULL for all rows; Ingestion's source clients are stubbed, #1136), so the
EPC must be fetched inline rather than read back. Builds the connection from the
`DB_*` env vars in backend/.env and the EPC token from `EPC_AUTH_TOKEN`.

Threads optional solar insights through harness `run_modelling` (so Solar PV
Options can fire once coordinates are wired) and adds the `solar_pv` catalogue
row. Solar + planning restrictions + DB persistence are noted follow-ups.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 14:25:33 +00:00
Daniel Roth
1b673622e2
Merge pull request #1191 from Hestia-Homes/bump-rds-version
Bump RDS to 14.22
2026-06-08 15:19:22 +01:00
Daniel Roth
84d4fa096d remove tests/ from pytest.ini 2026-06-08 14:15:50 +00:00
Daniel Roth
d23946a0a3 bump to 14.22 2026-06-08 14:14:15 +00:00
Daniel Roth
19c0a583cb
Merge pull request #1183 from Hestia-Homes/feature/handle-new-magicplan-response-structure
Get ventilation fields from new magicplan API response structure
2026-06-08 15:07:35 +01:00
Daniel Roth
3136d5a0f8 use correct db session to write to db 2026-06-08 14:07:03 +00:00
Jun-te Kim
18ad879f00 devcontainer: use dynamic host ports so worktrees don't collide
Publish container ports without a fixed host side in the backend and
asset_list compose files, and switch forwardPorts to service-qualified
form. Lets a second worktree's devcontainer spin up without conflicting
on host 8000/5432 (backend) or 8081 (asset_list), matching the pattern
in assessment-model-sales-frontend.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 13:53:22 +00:00
Daniel Roth
aaeb339254 Logging plus invoke locally bypassing subtask handler 2026-06-08 13:32:59 +00:00
Khalim Conn-Kowlessar
5e7ef5c7ff fix(control): no boiler interlock for TRVs+bypass controls 2107/2111 (SAP §9.4.11)
`_BOILER_NO_ROOM_THERMOSTAT_CONTROL_CODES` held only {2101, 2102} — it was
keyed off the Table 4e "+0.6 °C" annotation rather than the actual interlock
criterion. SAP 10.2 §9.4.11 (PDF p.66): "A boiler system with no room
thermostat (or a device equivalent in this context, such as a flow switch or
boiler energy manager) ... must be considered as having no interlock", and
"TRVs alone ... do not perform the boiler interlock function". A fixed bypass
likewise provides no interlock (it keeps water circulating when TRVs close).

So control 2107 ("Programmer, TRVs and bypass") and 2111 ("TRVs and bypass")
lack interlock and must take the Table 4c(2) −5pp Space+DHW seasonal-efficiency
adjustment and the Table 4f footnote a) ×1.3 circulation-pump uplift — both of
which they previously missed. (2108 flow switch / 2109 boiler energy manager
carry interlock-equivalent devices → excluded; 2103-2106/2113 have a room
thermostat.) All affected certs are cat-2 gas boilers, where §9.4.11 applies.

Eval: 909 computed, 45.3% → 46.9% within 0.5 (+14 certs: 412 → 426), mean|err|
1.659 → 1.633. Bucket means corrected: control 2107 +1.50 → +0.32 (n=38),
2111 +1.48 → +0.16 (n=4). 32 improved / 10 regressed (all small; the six that
crossed out of ±0.5 were coincidentally-accurate offsetting-error certs).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 13:17:15 +00:00
Khalim Conn-Kowlessar
24492aa4ba Merge origin/main into feature/bill-derivation (calculator + mapper fixes)
Pulls in 42 commits of calculator/mapper accuracy fixes from the per-cert
mapper-validation and floor/roof/heating fronts.

Conflict resolutions:
- mapper `_is_elmhurst_roof_window`: main dropped the branch's "wall location →
  vertical" guard (it broke cert 000516's rooflight), but that re-broke cert
  001431's two External-wall U>3.0 windows (which must stay vertical). The two
  certs lodge a BYTE-IDENTICAL §11 row, so neither location nor U separates
  them — the real discriminator is the room-in-roof context. Replaced the
  unconditional U>3.0 backstop with one gated on the BP having a room-in-roof
  (`_elmhurst_bp_has_room_in_roof`): 000516's Main BP has a "Room in roof type
  1" (→ rooflight), 001431's does not (→ vertical). Validated against BOTH —
  full Elmhurst worksheet suite 1038 pass + the 001431 window-extraction pin.
- property_postgres_repository: kept main's `ids_by_uprn` method + the branch's
  `_restrictions_of` helper.
- sap_fuel.py: the branch relocated it to domain/billing/ (already carrying
  main's to_table_32_code normalization), so kept the old path deleted.

Fallout from main's fabric fixes (validated by the boiler-3 real-cert pin which
still reproduces at delta 0):
- re-pinned the boiler-1 + boiler-instant-hw ASHP snapshot scores;
- main's §14.2 gas-boiler main-fuel derivation resolved the BGB/102 baseline
  gap, so `test_gas_boiler_instant_hw_before_baselines` is now a passing test
  (was an xfail tripwire).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 13:12:21 +00:00
Khalim Conn-Kowlessar
560c912c0b docs: roof-8 lead closed as data-fidelity (not a bug) + description-vs-code audit
Records the session-4 audit: walls/heating/controls clean; roof_construction=8
("Pitched, insulated", no measured thickness) computing U=2.30 is CORRECT, not a
bug — confirmed by user worksheet sim-case-29 (band C → Elmhurst SAP 55 ≡ our
56.75; lodged 80 is data-fidelity artifact). Lesson: "insulated (assumed)" = the
age-band default insulation level, not "well insulated". DO NOT re-chase roof-8.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 12:43:02 +00:00
Khalim Conn-Kowlessar
2f3b1dbd3f test(modelling): Solar PV Elmhurst cascade pins + battery tripwire
Slice 9 of the Solar PV Recommendation Generator (ADR-0026). Pins the
overlay→calculator PV cascade against Elmhurst's before/after re-lodgements of
cert 001431, across the orientation/pitch/overshading config space (the certs
lodge synthetic 1.00 kWp test vectors):
 - SE/SW, shaded (overshading 3/2), pitch 30°/45°
 - E/W, unshaded, pitch 60°/45°
 - NW/N, unshaded, pitch 60°/45° (the low-yield orientations)
Each hand-built SolarOverlay reproduces the relodged after at abs ≤ 1e-4 on
SAP / CO2 / primary energy.

Battery tripwire (per user): the "with battery" cert lodges a §19 5 kWh battery
the current extractor does NOT parse, so it scores identically to its
no-battery twin — the no-battery overlay reproduces it today, and the pin will
fail (alerting us to switch to the with-battery overlay) once the extractor
parses the battery. A companion test pins that the calculator already models
the 5 kWh battery (it raises SAP), so the fix target is established.

All five certs share an EES 'WGK' / SAP-code-502 main-heating lodgement the
mapper doesn't yet derive a fuel for; the pins patch the shared fuel (mains gas
26) identically on before+after to isolate the PV delta (the solar overlay
never touches heating), and `test_solar_before_baselines` xfails as the
forcing-function tripwire for that separate mapper-front gap.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 12:42:24 +00:00
Daniel Roth
bd4ad9022c Merge branch 'main' into feature/handle-new-magicplan-response-structure 2026-06-08 12:36:27 +00:00
Khalim Conn-Kowlessar
b249f69cb2 feat(modelling): thread SolarPotential into the orchestrator's solar Generator
Slice 8 of the Solar PV Recommendation Generator (ADR-0026). The
ModellingOrchestrator now reads each Property's persisted Google Solar
buildingInsights JSON (uow.solar), projects it once per Property into a typed
SolarPotential via `_solar_potential_for` (None for a missing or error
payload), and threads it into `recommend_solar` alongside planning_restrictions
— mirroring the ASHP wiring. Solar fires only when a feasible potential is
present, so dwellings without fetched solar data are unaffected.

FakeSolarRepo now returns None for an unseeded Property (was raising) and
supports `by_property` seeding, so the orchestrator's new solar read is exercised.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 12:22:56 +00:00
KhalimCK
1b94da16d0
Merge pull request #1189 from Hestia-Homes/feature/per-cert-mapper-validation
Feature/per cert mapper validation
2026-06-08 13:19:58 +01:00
Khalim Conn-Kowlessar
09cb8ceb9d feat(modelling): recommend_solar — eligibility + competing array Options
Slice 6 of the Solar PV Recommendation Generator (ADR-0026). `recommend_solar`
emits one "Solar PV" Recommendation of up to five conservatively-sized configs
× {no battery, battery} = ≤10 competing Options (a free Optimiser candidate).
Each Option folds a SolarOverlay built from the chosen config: one
PhotovoltaicArray per non-north segment (peak_power = panels × panelCapacityW /
1000; orientation/pitch from geometry; generation-calibrated overshading),
is_dwelling_export_capable set True absolutely, a diverter when the dwelling
has a cylinder (None for a combi), a 5 kWh battery for the battery variant, and
the per-config composite cost from Products.solar_bundle_cost.

Eligibility = house/bungalow ∧ not listed/heritage (blocks_internal, the same
gate as ASHP — a conservation area does NOT block PV) ∧ no existing PV ∧ a
feasible SolarPotential. Flats and existing-PV top-up are deferred.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 12:14:24 +00:00