Commit graph

7203 commits

Author SHA1 Message Date
Khalim Conn-Kowlessar
6d9ef1143c S0380.218: golden fixtures for 2 fresh API+Summary+worksheet triples
Two certs fetched fresh from the GOV.UK EPB register, each with an
Elmhurst Summary PDF (input) and a dr87 worksheet PDF (the (1)..(286)
ground truth):

  0340-2467-9260-2006-6521  (Summary_000922 / dr87-0001-000922)
  5500-5070-0822-0201-3663  (Summary_000920 / dr87-0001-000920)

Both run through BOTH front-ends — from_api_response and
from_elmhurst_site_notes — and through the rating + demand cascades.
Cross-mapper parity holds: the two paths agree to <1e-4 on continuous
SAP, fuel cost, CO2 and PE. Both paths reproduce the worksheet exactly:

  0340: (255) cost 776.4295, (272) CO2 2875.0498, (286) PE 16474.5616;
        fabric (33) 171.6188, (37) 205.9358; SAP int 70 = lodged.
  5500: (255) cost 751.8295, (272) CO2 2423.4547, (286) PE 14397.0118;
        fabric (33) 141.1226, (37) 167.3696; SAP int 66 = lodged.

Pinned in two tables of test_golden_fixtures.py:
  - _EXPECTATIONS / test_golden_cert_residual_matches_pin — SAP/PE/CO2
    residual vs the integer-rounded lodged register (SAP resid +0 both).
  - _WORKSHEET_PE_CO2 / test_golden_cert_pe_co2_matches_worksheet —
    PE (286)/(4) and CO2 (272) vs the worksheet at +0.0000 (the
    load-bearing 1e-4 check; lodged register is integer-rounded).

Dropped-field audit (raw JSON keys vs the schema-21.0.1 dataclass
fields consumed by from_dict) re-run on both fresh JSONs: no new
silently-dropped fields — only created_at metadata and the
shower_outlet_type/shower_wwhrs keys already handled by
_normalize_shower_outlets (mapper.py:2047). No calculator or mapper
change required; this is pure validation + regression-pinning.

Full §4 suite: 2392 passed, 1 skipped.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 12:35:50 +00:00
Khalim Conn-Kowlessar
afabfa0147 feat(modelling): sample a year from the EPC bulk export, offline-ready
fetch_epc_bulk_sample streams certificates-<year>.json out of the bulk ZIP via
range requests, keeps the first N SAP-version matches, and writes each cert's
inner document to <out>/<cert>.json for run_property_report. Stops after N, so
only the member prefix transfers, not the 15.7 GB archive (RangeFile.bytes_read
reports the true transfer vs the absolute ZIP offset). Verified on 2026: 100
SAP-10.2 certs -> report ran 81 scorable (MAE 2.03), 46 flagged, 19 raises
(11 full-SAP schema 19.1.0, 7 unmapped floor_construction 0/3, 1 missing
post_town) — real shadow-validation signal vs the curated golden 57.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 12:20:57 +00:00
Khalim Conn-Kowlessar
cf8e5b9ec6 feat(modelling): read the gov EPC bulk export via HTTP range requests
The bulk endpoint 302-redirects to a 15.7 GB S3 ZIP with one NDJSON member per
year; each line wraps the per-cert payload in a stringified 'document' that
parses to the same RdSAP-Schema-21.0.1 shape from_api_response already handles.
parse_bulk_line unwraps a record; is_sap_version filters to SAP 10.2; RangeFile
exposes the S3 object as a seekable file so zipfile streams a single year's
member (and a sampler stops early) without downloading the whole archive.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 12:16:18 +00:00
Khalim Conn-Kowlessar
1085842395 docs: handover — flagged numbers were stale (different branch), Part 1 is the task
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 12:11:12 +00:00
Khalim Conn-Kowlessar
8bd8ff8e5c docs: handover — fresh-API cross-comparison + flagged-cert debugging
Next-agent brief: fetch certs fresh from the EPC API (two new API+Summary+
worksheet triples for cross-mapper parity, plus six dashboard-flagged certs).
Flags the critical reconciliation: the user's flagged numbers don't match the
golden-fixtures cascade (0390-2954-3640 pinned +0 but flagged -6.85; 7536/2130
flags are pre-this-session), so fresh-raw-JSON-vs-curated-fixture or a
different engine must be reconciled before debugging. Documents the EPC API
fetch mechanism, the dropped-field audit method, this session's 4 fixes, and
the conventions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 12:08:28 +00:00
Khalim Conn-Kowlessar
f895dd3ab7 S0380.217: capture wall_insulation_thermal_conductivity (was dropped)
Second silently-dropped field from the 2130 audit: the schema-21
SapBuildingPart never declared `wall_insulation_thermal_conductivity`, so
`from_dict` discarded it. Captured it through schema 21.0.0/21.0.1 → domain
SapBuildingPart → API mapper, and wired it into u_wall's RdSAP 10 §5.8
documentary-evidence R-value calc (both the solid-brick §5.7/§5.8 path and
the cavity-composite path), replacing the bare 0.04 λ constant with a
resolved λ.

Resolver: absent / "Unknown" → the §5.8 default 0.04 W/m·K (mineral wool /
EPS); a mapped code → its λ; an unmapped integer code RAISES so the enum is
confirmed against a worksheet rather than silently mis-factored (same
incremental-coverage discipline as the glazing-type map). Only code 1
(= the default 0.04) is mapped — the sole observed value (cert 2130 Ext1).

Zero cascade effect today: the λ path fires only for solid-brick/cavity
walls with a *measured* wall thickness, and 2130 Ext1 lodges no wall
thickness, so its conductivity is captured-but-unused; all existing §5.8
certs lodge no conductivity → 0.04 default unchanged. The point is to stop
dropping lodged data and make λ correct when a future cert exercises it.

Suite: 2523 passed (1 pre-existing TFA fail); sap10_ml 237 passed (2
pre-existing stone-formula fails). Zero new pyright errors (46=46).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 11:57:00 +00:00
Jun-te Kim
dfd05ba28b tests files 2026-06-04 11:47:42 +00:00
Khalim Conn-Kowlessar
ea3af8d2f4 feat(modelling): CLI to fetch an EPC dump + build the inspection report
run_property_report builds the three-section Markdown+CSV report over a dir of
API-shaped EPC JSON, offline (defaults to the golden 57: 57/57 scorable, MAE
0.54, 6 flagged |Δ|>0.5). fetch_epc_dump pulls raw cert JSON from the live API
by --uprn/--postcode (picking the latest cert per match, skipping existing
files), mirroring fetch_cohort2's proven HTTP shape and reading
OPEN_EPC_API_TOKEN. Report artifacts + epc_dump/ are gitignored.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 11:26:17 +00:00
Khalim Conn-Kowlessar
ae267070b1 feat(modelling): flat per-property CSV for the inspection report
format_report_csv emits one comma-safe row per property: the calculator-error
fields (lodged/calculated/Δ/flag), the Plan headline figures (baseline+post
SAP/band, measures, cost+contingency, bill & CO2 savings, valuation %), the
flattened measure triggers, and any captured error — sortable in a spreadsheet
for a large dump.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 11:16:52 +00:00
Khalim Conn-Kowlessar
1c00708ecd feat(modelling): render the three-section inspection report as Markdown
format_report_markdown emits: (1) cohort parity stats + a per-property
lodged-vs-calculated table flagging |Δ| > 0.5 (errors shown inline),
(2) Plans + costings (SAP/band jump, cost + contingency, bill & CO2 savings,
valuation uplift), (3) each fired measure with the EPC attributes that
triggered it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 11:15:12 +00:00
Khalim Conn-Kowlessar
5e4906dd70 feat(modelling): cohort builder + cohort-level ParityReport
build_property_reports models a dump in order (errors captured per-cert);
parity_report_for aggregates the lodged-vs-calculated SAP across the cohort
into the existing ParityReport (MAE/RMSE/bias/worst-N), excluding certs that
couldn't be mapped or scored. Residual convention is the calculator's own
(predicted - actual), the negative of PropertyReport.sap_error.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 11:13:04 +00:00
Khalim Conn-Kowlessar
2b04dddb06 feat(modelling): surface each fired measure's trigger attributes
Section 3 of the report: build_property_report now runs the Modelling stage
and, for every Plan Measure, records the EPC attribute(s) that caused its
generator to fire (MeasureTrigger) — wall_construction/insulation for cavity
fill, roof thickness for loft, floor thickness/construction for floors, the
absent mechanical kind for ventilation. Modelling raises are captured as
plan_error, independent of the calculator-error capture.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 11:10:43 +00:00
Khalim Conn-Kowlessar
ecebb07c9e feat(modelling): calculator-error per property (lodged vs calculated SAP)
Section 1 of the property inspection report: PropertyReport compares the
cert's lodged energy_rating_current to Sap10Calculator's un-rounded SAP and
flags |Δ| > 0.5 (the ADR-0010/0013 shadow-validation design target). A
mapping/scoring raise is captured per-cert as calculator_error, never
propagated, so one bad cert can't abort the sweep.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 11:06:57 +00:00
Khalim Conn-Kowlessar
6b4f1aec44 docs: finalise 2130 as 0240-like — PV β-split proven exact
Updated 2130's pin notes with the case-18 finding: our cascade reproduces
the worksheet PV split to the decimal (gen 2684.17 / onsite 970.77 / export
1713.40), so the Appendix M1 β-split is exact, not the suspected bug. With
the gas PE factor also exact (1.13) and the wall measurement now wired
(S0380.215), 2130's +2/-11.72 is the irreducible API-only lodged residual
(0240-like), not a closable calculator bug. Notes-only; pin unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 11:04:47 +00:00
Khalim Conn-Kowlessar
712cc6f3f8 S0380.216: extractor — handle wrapped glazing-gap column in §11 labels
pdftotext dumps of hand-entered Elmhurst worksheets wrap the §11 glazing-
GAP column ("16 mm or more") onto the glazing-TYPE token, yielding labels
like "Double between 2002 and 2021 16 mm or [1st]" that
`_elmhurst_glazing_type_code` didn't recognise → UnmappedElmhurstLabel,
blocking the whole Summary from parsing.

Added a fallback: when the lightly-cleaned label isn't a known key, strip a
trailing wrapped gap descriptor (`\s+\d+\s*mm\b.*$`) and retry. Applied
AFTER the direct lookup so explicitly-mapped interleaved variants (e.g.
"Double with unknown 16 mm or install date more", where the gap splits into
the middle) are unaffected. The gap drives the API-path U-value lookup, not
the site-notes glazing-type enum, so dropping it is loss-free for the
cascade.

Unblocks running our cascade on hand-entered worksheet Summaries — used to
validate the PV β-split against simulated case 18 (our split matches the
P960 worksheet exactly: gen 2684.17, onsite 970.77, export 1713.40).

Suite: 2391 passed, 1 skipped. Zero new pyright errors (mapper 32=32).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 11:03:32 +00:00
Daniel Roth
df0c8f69b1 Merge branch 'main' into feature/handle-new-magicplan-response-structure 2026-06-04 10:52:32 +00:00
Khalim Conn-Kowlessar
77e29ac2f8 docs(modelling): handover — EPC API fetch + property inspection report
Next-phase handover: fetch live EPCs via EpcClientService, run the
offline Modelling harness, and save a per-property report covering
(1) lodged-vs-calculated SAP divergence (>0.5), (2) plans + costings,
(3) recommended measures + the EPC attributes that triggered them. Maps
the EPC API client (the user's blocker), the calculator-error ingredients
(parity_report scaffolding), and each generator's exact trigger fields.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 10:46:31 +00:00
Khalim Conn-Kowlessar
2f5ca85854 S0380.215: capture dropped measured wall insulation thickness
The schema-21 SapBuildingPart never declared `wall_insulation_thickness_
measured`, so `from_dict` silently discarded it. When a cert lodges
`wall_insulation_thickness == "measured"` the actual value (mm) lives in
that dropped field, so the cascade fell back to the 50 mm "insulation
present, unknown thickness" default instead of the lodged measurement.

Cert 2130 Ext1 lodges solid brick band B + INTERNAL insulation
"measured"/100 mm. Per RdSAP 10 §5.7 Table 8 (insulated-wall U by age band
+ insulation thickness) the 100 mm row gives U=0.32; the unknown-thickness
fallback gave 0.55. New `_api_resolve_wall_insulation_thickness` substitutes
the measured value for the "measured" sentinel; the existing
`_insulation_bucket`/Table-8 path then computes the correct U. Field added
to schema 21.0.0/21.0.1 SapBuildingPart; domain field widened to
Union[str, int] to match `roof_insulation_thickness`. Isolated: 2130 Ext1
is the only bp lodging "measured" across all 47 fixtures.

This spec-correct fix EXPOSED an offsetting under-count it had been masking
(per the repo's no-special-handling rule — the pre-fix +1 was two bugs
cancelling): 2130 cont SAP 83.35 → 83.78 (resid +1 → +2), PE -7.56 →
-11.72, CO2 -0.045 → -0.095. The exposed -11.72 PE (~-746 kWh/yr) is the
deferred gas-combi-PE + PV-β-credit under-count from S0380.45/.49, now
un-masked — the next slice. Re-pinned 2130 with the cause documented.

Suite: 2391 passed, 1 skipped. Zero new pyright errors (mapper 32=32).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 10:30:49 +00:00
Khalim Conn-Kowlessar
7be4d83ffa chore(billing): refresh off-gas fuel rates onto one consistent basis
Apply the deep-research off-gas figures so oil/smokeless/wood sit on the
same NEP-Apr-2026 retail / DESNZ DUKES gross-CV basis as the new coal
proxy (fuel-input, not useful-heat): OIL 9.16 -> 12.11 (prior value was
materially low vs current kerosene), SMOKELESS 10.0 -> 8.69, WOOD_LOGS
8.83 -> 8.25, WOOD_PELLETS 7.99 -> 7.38. SEG (15.0, Solar Energy UK) and
LPG (17.61, bottled-propane) kept; gas/electricity (Ofgem cap) unchanged.
CV arithmetic recorded in the snapshot _assumptions. OIL pin updated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 10:27:29 +00:00
Khalim Conn-Kowlessar
c6eaa53931 feat(billing): price house coal + heat network as documented proxies
Coal and heat networks have no national retail/cap rate, so the snapshot
left them null and BillDerivation raised UnpricedFuel — dropping those
certs from an offline cohort run. Add researched proxy rates (fuel-input
basis, sources + arithmetic in the JSON _note/_gaps): COAL 7.13 p/kWh
(NEP Nov 2025 coal uprated + DESNZ DUKES house-coal GCV) and HEAT_NETWORK
16.0 p/kWh + 69.4 p/day (Insite Energy operator sample; indicative, schemes
vary ~8-30). Both flagged proxy/indicative — sense-check estimates, not
market rates. Existing curated fuels are unchanged.

Replaces the unpriced-raises pin for these two with a positive rate pin;
off-peak stays unpriced pending the day/night accessor. Golden cohort now
runs 57/57 offline with zero errors.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 10:21:16 +00:00
Daniel Roth
d6030fb369
Merge pull request #1167 from Hestia-Homes/bug/pashub-fetcher-correctly-writes-to-sharepoint
pashub fetcher terraform variable correction
2026-06-04 11:19:31 +01:00
Daniel Roth
ea4c58bef7 terrafform correction 2026-06-04 10:15:58 +00:00
Daniel Roth
e9f84913c7
Merge pull request #1165 from Hestia-Homes/bug/pashub-fetcher-correctly-writes-to-sharepoint
Pashub fetcher dynamically instantiates sharepoint client based on received sharepoint site
2026-06-04 10:44:43 +01:00
Khalim Conn-Kowlessar
e097ce2cef docs: finalise 7536 as 0240-like (resid +1, do not chase)
After three faithful-worksheet iterations (simulated cases 15/16/17), the
7536 +1 SAP residual is confirmed 0240-like — an Elmhurst register-rounding
residual not reproducible from the API-only JSON, not a calculator bug.

Case 17 is faithful on windows (Main 16.98 / Ext1 13.59 / Ext2 1.89) and
ground floors; every per-element value matches our cascade: walls
0.70/0.28/0.40, roofs 0.40/0.18/0.68 (S0380.214), window U-eff
2.4368/1.8519, ground floors 0.97/0.26/1.12. The only worksheet divergences
were manual-entry artifacts: case-16 inverted the floor order (put the
50.98 m² upper floor as ground), and case-17 auto-derived spurious "to
external air" exposed floors from the small-ground/big-upper geometry —
real 7536 lodges floor_heat_loss 2/7/3 (unheated-space / ground), none is
code 1 (exposed). Our spec-correct cont SAP is 68.924; lodged 68 carries
Elmhurst's own residual.

Notes-only; pin unchanged (resid +1, PE -6.1952, CO2 -0.1639). Suite green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 09:41:48 +00:00
Daniel Roth
db2cbc9c57 SharePoint upload is skipped when client is None 🟩 2026-06-04 09:33:07 +00:00
Khalim Conn-Kowlessar
8b5ab1c59e feat(modelling): turnkey offline cohort script (tables + CSV)
CertResult now carries its Plan (with flat baseline/post-SAP/measures
properties), and `format_cohort_csv` renders one browsable row per cert
(SAP transition, band, measures, cost, bill saving, valuation %, error).
`scripts/run_modelling_cohort.py` is turnkey: no args runs the committed
golden cohort, prints a sense-check table for the first measure-bearing
certs (a capped preview so a large dump doesn't flood the terminal), the
summary, and writes modelling_cohort.csv (gitignored). Point it at the
EPC dump when it lands.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 09:30:53 +00:00
Daniel Roth
416dac0512 SharePoint upload is skipped when client is None 🟥 2026-06-04 09:29:24 +00:00
Khalim Conn-Kowlessar
d8ef40c745 feat(modelling): offline cohort runner over an EPC-JSON dump
`harness.cohort.run_cohort(paths)` parses each API-shaped EPC JSON with
from_api_response and models it via run_modelling — no database, no
network — capturing per-cert errors instead of aborting the sweep, plus
`format_cohort_summary`. A thin `scripts/run_modelling_cohort.py` CLI
points it at a directory. Proven over the 57 golden API certs: 56 ran
offline, 15 produced measures, 1 errored (COAL has no Fuel Rates entry —
a BillDerivation coverage gap, not a harness one). Ready for the EPC dump.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 09:23:32 +00:00
Khalim Conn-Kowlessar
98f5ee4fca feat(modelling): robust offline modelling inspection (run_modelling)
Two fixes that unblock offline, no-database inspection over an arbitrary
EPC dump:

- Complete the harness sample catalogue with loft_insulation and
  solid_floor_insulation — the four fabric generators can emit five
  Measure Types, but the catalogue priced only three, so an offline run
  on a property with an uninsulated loft or solid floor raised mid-run.
  A new test pins the catalogue to cover every generator Measure Type.
- Add `run_modelling(epc, ...)` — runs ONLY the Modelling stage (no
  Ingestion / Baseline), so it needs no lodged recorded-performance / RHI
  and inspects recommendations on any calculator-scorable EPC. `run_one`
  (full pipeline) stays for when you want Baseline too.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 09:19:18 +00:00
Daniel Roth
eb72b0223f add ECO_SHAREPOINT_ID env var 2026-06-04 09:11:08 +00:00
Khalim Conn-Kowlessar
b3f4609c2d feat(modelling): wire Valuation Uplift onto the Plan
The Plan derives its Valuation Uplift (ADR-0018) from its baseline -> post
band jump and works+contingency cost, given one external input — the
Property's current market value (a Property Valuation, mostly absent).
`Plan.valuation` / `Plan.baseline_epc_rating` are derived like the other
headline figures; `PlanModel.from_domain` maps the £ forms to the live
plan.valuation_* columns (NULL when no value — the percentage is not
persisted on those columns). `Property.current_market_value` is the new
optional source; the orchestrator threads it onto the Plan. `run_one`
takes a `current_market_value` so the harness can value the uplift, and
the sense-check table shows the average % (always) plus the £ forms when
known.

Sourcing the current market value (upload / default) remains deferred
(ADR-0018); it is None throughout until that lands, so the columns stay
NULL at scale.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 08:59:04 +00:00
Khalim Conn-Kowlessar
ac7f510ccb S0380.214: as-built sloping-ceiling roof → Table 18 col (3)
A "Pitched, sloping ceiling" roof (roof_construction code 8) lodged with
"As Built" insulation (no measured thickness → None) was wrongly routed to
RdSAP 10 Table 18 column (1) "insulation between joists or unknown". A
sloping ceiling has no joist void, so per RdSAP 10 §5.11 roof-input item
5-5 ("Sloping ceiling insulation … unknown / as built → Table 18") and
Table 18 note (b) ("Applies also to roof with sloping ceiling") it takes
column (3) — band F = 0.68, band L = 0.18 (vs col 1 0.40 / 0.16).

Discriminator is the code-8 "sloping ceiling" string only: code-5 vaulted
ceilings stay on column (1) per the 33 cohort-2 "ND" vaulted certs
(S0380.211), and the "NI"/"ND" unknown case is untouched. New
`is_pitched_sloping_ceiling` flag threaded from heat_transmission to
`u_roof`; pre-1950 bands already reach the same col (3) value (2.30) via
the mapper's thickness=0 → Table 16 row-0 override, so the new branch
carries the post-1950 bands where col 1 ≠ col 3.

Worksheet-validated by simulated case 15 (the 7536 replica): our cascade
on its Summary matches the P960 worksheet exactly — roof HLC 29.17 W/K,
cont SAP 65.04 vs 65. Re-pins golden cert 7536: roof 26.77 → 29.17, cont
SAP 69.071 → 68.924, PE -7.0776 → -6.1952, CO2 -0.1875 → -0.1639 (SAP
integer 68, resid +1 unchanged — the remaining +0.92 is a diffuse demand
under-count needing a fully-faithful worksheet). Blast radius: 7536 only.

Suite: 2388 passed, 1 skipped (main); sap10_ml 233 passed + 2 pre-existing
stone-formula failures (out of scope). Zero new pyright errors.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 08:58:37 +00:00
Khalim Conn-Kowlessar
e6f54df92b feat(modelling): ValuationUplift domain class (percentage-primary)
The financial-uplift model per ADR-0018. `estimate_valuation_uplift(
current_band, target_band, current_value=None, total_cost=None)` returns
a `ValuationUplift`: band-transition uplift compounded from four broker
tables (MoneySupermarket / Lloyds per-step, Knight Frank / Rightmove
whole-jump), taking min/max/mean across the covering sources. Always a
percentage; absolute £ forms (increase at each bound + post-retrofit
value) only when a current market value is supplied; the 2x ROI cap
rescales the percentages and can only bite once a value is known. A
non-improving jump is a clean 0% no-op.

Pure function, no external dependency. Persisting it (where the value
lands) and sourcing the current market value stay deferred (ADR-0018).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 08:33:19 +00:00
Daniel Roth
0bec880c6e
Merge pull request #1163 from Hestia-Homes/pashub-fetcher-all-files
Pashub fetcher: Optionally fetch all files. Also make ECS certificate a core file
2026-06-04 09:31:29 +01:00
Khalim Conn-Kowlessar
31da90f5eb feat(modelling): persist recommendation.material_id from the catalogue
Expand half of the recommendation_materials retirement (ADR-0017). A
Plan Measure installs a single Product, so thread its catalogue id end to
end — Product.id -> MeasureOption.material_id -> PlanMeasure.material_id
-> recommendation.material_id — replacing the per-material BOM child
table with one nullable column on the row. ProductPostgresRepository
reads the id from MaterialRow; the four fabric generators set it on their
Option; the orchestrator carries it onto the Plan Measure; the mirror
declares + maps the column. Optional throughout (the JSON stopgap
catalogue carries no ids -> NULL).

The multi-measure integration test now pins each persisted measure's
material_id to its seeded MaterialRow id. Migration spec (live column
must be added before this deploys; contraction is the owner's next step)
in docs/migrations/recommendation-material-id.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 08:26:58 +00:00
Daniel Roth
1c1642c1c5 add TODO comment 2026-06-04 08:19:16 +00:00
Khalim Conn-Kowlessar
c5520b82f9 feat(modelling): run_one console entrypoint for DB-less inspection
Slice 3. `harness.console.run_one(epc, goal_band=...)` wires the full
AraFirstRunPipeline against in-memory fakes — no Postgres, no network —
runs one property, prints the sense-check table, and returns the Plan
for interactive poking from a REPL at the worktree root. Defaults to the
committed harness sample catalogue.

Refactors the slice-1 integration test to delegate to run_one (dropping
~70 lines of duplicated wiring + the now-unused test catalogue fixture),
so it exercises the shipped entrypoint rather than a parallel copy. The
new console test covers run_one's print/return contract.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 08:14:14 +00:00
Daniel Roth
09e31c2a98 correct magicplan orchestration tests 2026-06-04 08:13:20 +00:00
Khalim Conn-Kowlessar
9329978374 feat(modelling): sense-check table for a Plan in the DB-less harness
Slice 2. `harness.plan_table.format_plan_table(plan)` renders a Plan as a
plain-text table — one package summary line (baseline SAP/band -> post
SAP/band, CO2 saved, cost of works + contingency, bill saved) and one
line per Plan Measure (signed SAP points, cost, delivered kWh + £
savings). Pure presentation: reads the Plan, computes nothing. The
DB-less First Run test now prints it (visible under `pytest -s`) so the
modelled package can be eyeballed and debugged by hand.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 08:06:53 +00:00
Khalim Conn-Kowlessar
26d7fc036e docs(modelling): record Valuation Uplift design (ADR-0018 + glossary)
From the grill-with-docs pass on the depth+scale phase. Splits the
overloaded "valuation" into two glossary terms — Property Valuation
(current market value, a Baseline attribute, mostly missing) and
Valuation Uplift (plan-conditional, percentage-primary; absolute £ only
when a Property Valuation exists, 2x ROI cap on the £ form). ADR-0018
records the percentage-primary decision and why (the EPC scale corpus
has no market values, so a value-primary model produces nothing), plus
the deferred sourcing / per-measure / rental-yield items.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 07:52:33 +00:00
Khalim Conn-Kowlessar
d5f1fc335b test(modelling): run First Run with no database via in-memory fakes
Slice 1 of the DB-less inspection harness. Complete the in-memory
FakeUnitOfWork so the ModellingOrchestrator runs with no Postgres:
add FakeScenarioRepository + FakePlanRepository (idempotent, keyed by
(property_id, scenario_id)), expose scenario/product/plan on the fake
unit, and grow FakePropertyRepo to compose the effective EPC from the
EPC repo at read time — mirroring PropertyPostgresRepository, so the
EPC Ingestion persists is visible to Baseline + Modelling (the
through-repos hand-off, in memory).

The new integration test drives the full AraFirstRunPipeline
(Ingestion -> Baseline -> Modelling) against the FakeUnitOfWork — no
Session ever opened — on the uninsulated 000490 fixture with its lodged
recorded-performance filled in (it already carries the RHI block, so
Baseline can run) and asserts a multi-measure Plan is produced. The
committed product catalogue prices the wall/floor/ventilation measures
it fires.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 07:51:44 +00:00
Khalim Conn-Kowlessar
ec64c39d74 docs: correct 2130/7536 drive-to-zero diagnosis (distinct causes, not shared)
Per each cert's notes: 7536 is a glazing-U gap (S0380.97 glazing_type=2
Table 24 default vs the cert's higher lodged U on multi-age bps) — the
tractable target; 2130's SAP +1 is a PV-β cohort cascade interaction, not
a fabric line. The earlier "multi-part wall, shared cause" label was wrong.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 00:14:10 +00:00
Khalim Conn-Kowlessar
f50195ac54 docs: handover — golden SAP drive-to-zero priority (2130/7536); 0240 architectural
Add a CURRENT PRIORITY section: of 53 pinned golden certs, 3 have non-zero
SAP residual. 2130 (+0.85 cont) + 7536 (+0.57 cont) are real multi-part-wall
fabric over-predictions (the drive-to-zero targets, possibly one shared
cause). 0240 (-1) is architectural — the lodged 73 needs an unpreserved
2013+ pump; document the cause, do NOT re-pin (user decision).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 00:00:23 +00:00
Khalim Conn-Kowlessar
b9bbcecb42 docs: Thread 2 cost +4 closed by S0380.213 heat-network standing charge
Record the £120 standing-charge fix (Table 12 note (l) + §C3.2, case 14
(351)), the corrected diagnosis (standing charge, not cost scaling — the
4.24 p/kWh heat price was already right), the double-count avoidance, and
the remaining ~7% demand over-count (SAP -2). Bump HEAD/baseline/next-slice.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 23:46:32 +00:00
Khalim Conn-Kowlessar
ee484d9f4a S0380.213: heat-network standing charge (£120) — fixes 9390 cost under-count
Cert 9390 (community mains-gas boiler, API main_fuel_type=20) drew £0
standing charge → fuel cost under-counted → SAP read +4 high (71 vs 67).

Root cause: the standing-charge logic (`additional_standing_charges_gbp`)
only knows the GAS branch (`_is_gas_code`) and the off-peak-electric branch.
A heat-network community fuel is not a Table-32 gas code — EPC 20 = "mains
gas (community)" normalises to Table-32 code 20 (biomass), so
`_is_gas_code(20)` is False and the standing came out £0. The Summary path
masks this because it lodges community gas as Table-32 code 1 (ordinary
mains gas), which IS gas-recognised and already draws the £120 gas standing
— so the CH1-6 corpus was unaffected while the API path lost the charge.

Spec basis (verified against SAP 10.2 spec PDF):
- Table 12 (p.191) "Heat networks" row standing charge = £120/yr, note (k).
- Note (l): "Include half this value if only DHW is provided by a heat
  network."
- §C3.2 (p.58): the full charge applies when the space heating is also a
  heat network.
Worksheet-validated: simulated case 14 (community boilers + mains gas,
space + water) → worksheet (351) Additional standing charges = £120.

Fix: new `_heat_network_standing_charge_gbp(epc, main)` returns the
heat-network standing (£120 full when the space main is a heat network;
£60 when only DHW is on the network) or None otherwise. Applied at both
fuel-cost call sites, REPLACING the fuel-based `additional_standing_charges
_gbp` for heat-network mains (NOT additive) so a Summary-path community-gas
main — already £120 via the gas branch — is not double-counted to £240. The
CH1-6 community corpus stays exactly £120 (59 corpus tests pass).

9390 SAP +4 → -2 (cont 65.39 vs lodged 67): the spec-correct £120 standing
EXPOSES a separate ~7% demand over-count (also visible as PE 220 vs lodged
205) — a heat-source-efficiency-default / fabric residual, follow-up scope.
9390 is unpinned (retired P2.2 per ADR-0010 §10); helper locked by 2 unit
tests. Full suite 2386 passed, 1 skipped.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 23:45:24 +00:00
Khalim Conn-Kowlessar
f658f7ce71 docs: Thread 2 CO2/PE collision fixed by S0380.212; cost +4 tail open
Record the heat-network fuel-code collision fix (EPC 20 'mains gas
(community)' → Table-12 51), case-14 validation, and the remaining
cost-scaling gap (heat-network cost path missing 1/heat_source_eff).
Bump HEAD/next-slice; update shipped + audit tables.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 23:27:45 +00:00
Khalim Conn-Kowlessar
08dd0b4c73 S0380.212: fix community fuel-code collision in heat-network CO2/PE/cost
Cert 9390-2722-3520 (community mains-gas boiler scheme, sap_main_heating_
code=301, main_fuel_type=20) emitted CO2 0.44 t vs lodged 2.8 t — 6.4x low.

Root cause: the EPC `main_fuel_type` enum and the SAP Table 12 / Table 32
fuel-code numbering COLLIDE in the 18-25 range. Per
`datatypes/epc/domain/epc_codes.csv` (RdSAP-Schema-17.0) EPC fuel
20 = "mains gas (community)", but Table 12/32 code 20 is a solid biomass
fuel (CO2 0.028, PE 1.046, wood-logs price). The factor lookups
(`co2_factor_kg_per_kwh` / `primary_energy_factor` / `unit_price_p_per_kwh`)
check the Table-12/32 dict FIRST, so the EPC community fuel 20 silently
returned the biomass factor instead of translating 20 -> Table 12 code 51
(community mains gas: CO2 0.210, PE 1.130, mains-gas price).

Fix: new `_heat_network_factor_fuel_code(main)` translates the EPC community
fuel to its Table-12 code via `API_FUEL_TO_TABLE_12`, but ONLY for
heat-network mains (`_is_heat_network_main`) — a genuine biomass boiler
(non-community) keeps its raw Table-12 factor. Applied at the five
heat-network factor sites: space-heating CO2 / PE / unit-price and
water-heating (WHC 901) CO2 / PE. The Summary path is unaffected (it maps
"Mains gas - community" to code 1, no collision), so the community-heating
corpus (CH1-6) is untouched.

Worksheet-validated against simulated case 14 (community boilers + mains
gas, SAP code 301): worksheet (367) CO2 factor 0.2100, (467) PE factor
1.1300 — exactly the Table-12 code-51 values the translator now reaches.
9390 CO2 0.44 -> 3.03 t (lodged 2.8; spec-correct factors over the API-only
register residual per [[feedback-worksheet-not-api-reference]]), PE 204 ->
220 (the spec-correct 1.13 factor; the prior 204≈205 match was the
collision coinciding with the register residual). 9390 is unpinned (retired
at P2.2 per ADR-0010 §10); the translator is locked by two unit tests.

REMAINING (separate follow-up): 9390 SAP +4 is a cost-side gap — the
heat-network cost path does not apply the 1/heat_source_eff (1/0.80)
scaling that the CO2/PE paths do, so community fuel cost under-counts.

Suite: 2616 passed, 1 skipped (community corpus green); the 2
test_rdsap_uvalues stone-formula failures are pre-existing (HEAD 58ff7d88).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 23:25:53 +00:00
Khalim Conn-Kowlessar
b8b7e02034 docs(modelling): next-phase handover — depth + scale e2e + grilling prompt
Capture the next phase (close persisted-field gaps + financial uplift, plus a
large-scale e2e run of a SAP 10.2 EPC dump and console manual testing; measure
coverage deferred) and a self-contained handover prompt for a fresh agent to
pick up via a grilling session.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 23:09:08 +00:00
Khalim Conn-Kowlessar
c3c44fa3d0 docs: Thread 2 unblocked — case 14 (code-301 boiler+gas) arbitrates 9390
Record the community mains-gas BOILER worksheet (case 14): target (386)
heat-network CO2 factor 0.2640, distribution loss 1.49, code 301. 9390
decomposition: PE matches (204 vs 205), CO2 6.5x low (collision), SAP +4
separate cost gap.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 23:07:05 +00:00
Khalim Conn-Kowlessar
55af4ee2d7 docs: Thread 1 (roof) CLOSED by S0380.211; Thread 2 needs code-301 gas ws
Record the roof closure (vaulted NI → Table 18 col 1 0.16, cohort-arbitrated
not the guessed 0.25), the AGENT_GUIDE suite-command gap (sap10_ml/tests/ not
run) + pre-existing stone failures, cases 11/12/13 now available, and the
fuel-20 = community-gas (Table 12 code 51) note. Thread 2 still needs a
code-301 community-boiler + mains-gas worksheet (case 13 is code-302 CHP).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 22:59:35 +00:00