Commit graph

6206 commits

Author SHA1 Message Date
Daniel Roth
5178cd02c5 UploadedFile, FileTypeEnum, FileSourceEnum importable from infrastructure.postgres.uploaded_file_table 🟩
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 11:50:51 +00:00
Daniel Roth
41b282042f UploadedFile, FileTypeEnum, FileSourceEnum importable from infrastructure.postgres.uploaded_file_table 🟥
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 11:42:53 +00:00
Daniel Roth
e5e67c203b temp commit to backup prd in case laptop dies 2026-06-08 16:48:37 +00:00
Daniel Roth
209a0c401c audit generator application framework 2026-06-08 16:20:05 +00:00
Daniel Roth
7497865fb2 Retrieve plan by uploaded_file_id 🟪 2026-06-08 15:44:19 +00:00
Daniel Roth
f8fcf38886 get_plan_by_uploaded_file_id 🟩 2026-06-08 15:37:52 +00:00
Jun-te Kim
0ccb0e0bf9
Merge pull request #1196 from Hestia-Homes/feautre/additional_properties_for_tracking
Feautre/additional properties for tracking
2026-06-08 16:19:34 +01:00
Daniel Roth
26ba09a1c7
Merge pull request #1194 from Hestia-Homes/correct-path-to-magicplan-dockerfile
Correct magicplan dockerfile path in terraform
2026-06-08 15:54:47 +01:00
Daniel Roth
e1515b0c00 correct path in terraform 2026-06-08 14:49:41 +00:00
Daniel Roth
c40a7c0af9
Merge pull request #1192 from Hestia-Homes/bug/fix-magicplan-orchestrator-tests
Fix magicplan tests to reflect changes to the orchestrator
2026-06-08 15:44:01 +01:00
Jun-te Kim
a1b4bf4e98 added 4 deal proeprties 2026-06-08 14:42:06 +00:00
Daniel Roth
648e726ca2 fix orchestration tests 2026-06-08 14:40:14 +00:00
Daniel Roth
1b673622e2
Merge pull request #1191 from Hestia-Homes/bump-rds-version
Bump RDS to 14.22
2026-06-08 15:19:22 +01:00
Daniel Roth
84d4fa096d remove tests/ from pytest.ini 2026-06-08 14:15:50 +00:00
Daniel Roth
d23946a0a3 bump to 14.22 2026-06-08 14:14:15 +00:00
Daniel Roth
19c0a583cb
Merge pull request #1183 from Hestia-Homes/feature/handle-new-magicplan-response-structure
Get ventilation fields from new magicplan API response structure
2026-06-08 15:07:35 +01:00
Daniel Roth
3136d5a0f8 use correct db session to write to db 2026-06-08 14:07:03 +00:00
Jun-te Kim
18ad879f00 devcontainer: use dynamic host ports so worktrees don't collide
Publish container ports without a fixed host side in the backend and
asset_list compose files, and switch forwardPorts to service-qualified
form. Lets a second worktree's devcontainer spin up without conflicting
on host 8000/5432 (backend) or 8081 (asset_list), matching the pattern
in assessment-model-sales-frontend.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 13:53:22 +00:00
Daniel Roth
aaeb339254 Logging plus invoke locally bypassing subtask handler 2026-06-08 13:32:59 +00:00
Daniel Roth
bd4ad9022c Merge branch 'main' into feature/handle-new-magicplan-response-structure 2026-06-08 12:36:27 +00:00
KhalimCK
1b94da16d0
Merge pull request #1189 from Hestia-Homes/feature/per-cert-mapper-validation
Feature/per cert mapper validation
2026-06-08 13:19:58 +01:00
Daniel Roth
c22ee3821b Merge branch 'main' into feature/handle-new-magicplan-response-structure 2026-06-08 09:57:26 +00:00
Daniel Roth
41a40c9ba0 Fix Pylance unknowns in SQLModel table tests and correct pytest paths 2026-06-08 09:56:54 +00:00
Daniel Roth
7a1aaf4965 Window carries no opening_type — ventilation table is the sole persistence point 🟩 2026-06-08 09:45:15 +00:00
Daniel Roth
3f5b3cf172 Window carries no opening_type — ventilation table is the sole persistence point 🟥 2026-06-08 09:43:41 +00:00
Khalim Conn-Kowlessar
d0f57a0e94 docs: session-4 handover — floor_heat_loss=3 resolved (U=0.7), 7536 re-pinned
Code 3 = "(other premises below)" = above partially heated space (§3.12 →
U=0.7), confirmed 9/9 on single-BP certs (the diagnostic that dodged the
lossy-floors[] contamination). Records the 7536 re-pin and the lesson that
"irreducible residual" golden notes can mask a real mapper bug.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 22:26:21 +00:00
Khalim Conn-Kowlessar
8741fbdfac fix(floor): floor_heat_loss=3 → above partially heated space, U=0.7 (RdSAP §3.12)
The API `floor_heat_loss` code is authoritative — confirmed by joining each
single-BP cert's code to its independent `floors[].description` (which the
gov register publishes alongside the code):

  code 1 ↔ "To external air"        (exposed,        9/9)
  code 2 ↔ "To unheated space"      (semi-exposed,   6/6)
  code 3 ↔ "(other premises below)" (partially htd,  9/9)
  code 6 ↔ "(another dwelling below)" (party,      176/176)
  code 7 ↔ "Solid"/"Suspended …"    (ground,     all)

Code 3 was mis-mapped to "To unheated space" (semi-exposed) and, on
mid-/top-floor flats, had its floor area zeroed entirely by the
dwelling-level exposure heuristic. RdSAP 10 §3.12 (PDF p.25) classes a
flat's floor over non-domestic "other premises … heated, but at different
times" as "above a partially heated space" → the §5.14 (PDF p.47) constant
U=0.7 W/m²K — distinct from semi-exposed (Table 20) and party (no loss).

Fix: the mapper sets `is_above_partially_heated_space` on the floor=0
dimension for code 3 (string → "(other premises below)" for fidelity), and
the heat-transmission step lets that per-BP lodgement override the flat
suppression upward (mirroring the existing exposed / "another dwelling
below" overrides). The cascade already routes is_above_partial → U=0.7.

Re-pins golden cert 7536-3827: its Ext2 (bp3) lodges code 3, but the cert's
lossy `floors[]` summary dropped that description, so a prior agent guessed
"code 3 = ground" (U=1.12) and concluded the residual was an irreducible
"register-rounding" artifact. It was this bug: Ext2 floor U 1.12 → 0.70,
PE -6.1952 → -5.6414, CO2 -0.1639 → -0.1492 (both toward 0), SAP unchanged.

Eval: 909 computed, 45.1% → 45.3% within 0.5, mean|err| 1.702 → 1.659,
<1.0 59.5% → 60.2%. 13 code-3 certs improve (0380 +3.71 → -0.63, 0350
+7.82 → +0.83, 2610 +7.47 → -1.29); the few that overshoot were already
failing and carry independent fabric bugs (9763's walls = 8 W/K for 60 m²).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 22:25:04 +00:00
Khalim Conn-Kowlessar
75ef250ec8 docs: session-4 handover — exposed-floor fix shipped, floor-3 enum unconfirmed
Record that profiler lead #1 (floor_codes=3) is not a clean single cause
(bimodal + confounded), that the paired worksheet certs confirm only
codes 1/6/7 (code 3 unmapped → needs a worksheet for 0380-2087), and that
immersion_type=2 / main_control=2107 / roof_codes=1 are scatter, not
dispatch bugs. The exposed-floor-on-flats fix (§3.12) shipped at b40e0f67.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 21:53:25 +00:00
Khalim Conn-Kowlessar
b40e0f67b8 fix(floor): exposed floor on a flat carries heat loss (RdSAP §3.12)
A mid-/top-floor flat whose lowest floor is lodged as an exposed floor
(API floor_heat_loss=1) had its floor area zeroed by the dwelling-level
exposure heuristic, which keys only on the flat label and defaults
has_exposed_floor=False (assuming the floor sits over another *heated*
dwelling). RdSAP 10 §3.12 (PDF p.25) is explicit:

  "Otherwise the floor area of the flat ... is:
     - an exposed floor if there is an open space below"

i.e. a flat cantilevered over a passageway IS a heat-loss floor on
Table 20. The per-BP `is_exposed_floor` lodgement is authoritative and
now overrides the dwelling-level suppression upward, mirroring the
existing "another dwelling below" party override (which suppresses
downward). The code-1↔"E To external air" enum is confirmed by the
paired API+Summary worksheet certs (0350, 3800).

Eval: 45.1% → 45.3% within 0.5 (909 computed); cert 3836 +6.79 → +0.77,
5717 +1.31 → -0.07 and 0997 +0.76 → +0.05 cross into <0.5. Two
already-failing under-rated certs (7636, 2241) shift further — both are
dominated by independent cost-side over-counts the exposed floor merely
unmasks (7636 walls = 8.98 W/K for 33.87 m² is the real defect).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 21:47:52 +00:00
Khalim Conn-Kowlessar
ae34ca4d74 docs: session-3 API-profiling handover (raises cleared, profiler-driven leads)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 20:39:52 +00:00
Khalim Conn-Kowlessar
28b1da1e06 feat(diag): profile API SAP error against raw-API characteristics
Joins each computed cert's signed error (eval _results.csv) with a rich
feature set extracted from its RAW API JSON (not the mapped
EpcPropertyData), then ranks (feature, value) buckets by error carried
and by |mean signed| bias. Surfaces systematic API-path handling gaps —
a field the mapper silently drops still shows as an error-correlated
bucket. Companion to eval_api_sap_accuracy.py / decompose_api_cost_error.py.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 20:38:19 +00:00
Khalim Conn-Kowlessar
a8e5563ace fix(warm-air): Table 11 secondary fraction for category 9 → 0.10
main_heating_category=9 (warm-air systems, NOT heat pump) had no entry
in _SECONDARY_HEATING_FRACTION_BY_CATEGORY, so a warm-air main with a
lodged secondary raised UnmappedSapCode in
_secondary_heating_fraction_for_category — the last calc_raise in the
API sample (cert 0380-2197-2590-2996-2715: warm air mains gas code 506 +
electric room-heater secondary).

SAP 10.2 Table 11 (p.188): a gas/oil warm-air unit falls under "All gas,
liquid and solid fuel systems" (0.10), and electric warm air under
"Other electric systems" (also 0.10) — so 0.10 regardless of fuel. The
warm-air efficiency (Table 4a code→eff: 506→0.70) and Table 4f fan
energy were already wired; this was the only missing dispatch entry.

0380 now computes: SAP 78.1 vs lodged 77 (+1.1; the residual is per-cert
fabric/PV, not the warm-air dispatch — a faithful 0380 worksheet isn't
available, sim case 28 diverges at SAP 57 / code 502 / condensing unit).
Eval: zero raises remain, computed 908→909; mean|err| 1.703→1.702.
Regression green (2448 pass incl. golden 6035 + cohort); pyright
net-zero (44=44).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 20:26:32 +00:00
Khalim Conn-Kowlessar
1c5675a063 fix(mapper): floor_heat_loss code 8 → no floor heat loss (extension over heated space)
API floor_heat_loss=8 is observed on EXTENSION building parts whose
floor sits over a heated space within the SAME dwelling (an upper-storey
extension over a heated room). RdSAP 10 §3 gives an internal floor
between heated storeys no floor heat loss — mechanically identical to a
code-6 party floor. `_api_floor_type_str` had no entry for 8, raising
UnmappedApiCode and blocking certs 0370-2254-6520-2426-5971 and
0997-1206-9806-0715-2904.

Map code 8 to the code-6 no-heat-loss string "(another dwelling below)"
(consumed by heat_transmission's party-floor suppression; != "Ground
floor" so the §5 (12) suspended-timber rule stays inert). Empirically
confirmed against both certs: the no-heat-loss treatment lands them
within 0.5 of lodged (0370-2254 68.92 vs 69; 0997-1206 40.68 vs 41),
whereas Ground-floor / unheated / external mappings miss 0997 by ~4 SAP.

Eval computed 906→908. Regression green (only the pre-existing
test_total_floor_area fails); pyright net-zero (38=38).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 18:54:00 +00:00
Khalim Conn-Kowlessar
f40485887d fix(u-value): RdSAP10 ignores gov-API wall insulation conductivity → §5.8 default λ
The gov EPC API field wall_insulation_thermal_conductivity is OUTPUT
metadata in the openly-published EPC, not an input to the RdSAP10 tool
(Elmhurst) that produced it — its wall entry is Type + Insulation +
thickness only, with no conductivity field. So the RdSAP10 reduced-data
method always uses the SAP 10.2 §5.8 (p.41) default λ=0.04 W/m·K,
whatever code the register lodged.

`_resolve_wall_insulation_lambda_w_per_mk` previously mapped only code 1
(→0.04) and RAISED on others, blocking cert 2090-6909-8060-5201-6401
(code 3 on an internally-insulated 360mm solid-brick wall) with
calc_raise:ValueError. Now it returns the §5.8 default for any code.

Validated: 2090 computes to SAP 73.97 vs lodged 74 (err -0.03); λ of
0.04 / 0.03 / 0.025 all round to 74, and Elmhurst exposes no conductivity
input, so 0.04 is the spec-faithful RdSAP10 value. Eval computed
905→906; mean|err| 1.708→1.706. Regression green (only the 2 pre-existing
stone-wall U failures); pyright net-zero (69=69).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 18:39:01 +00:00
Khalim Conn-Kowlessar
449d8c5b95 fix(hw): direct-acting electric boiler (191) → zero primary circuit loss
SAP 10.2 Table 3 (PDF p.160) names "Direct-acting electric boiler"
verbatim in the primary-loss zero list (alongside electric immersion,
combi, CPSU, integral-vessel heat pump). RdSAP 10 §12 (p.62) classifies
SAP code 191 as the direct-acting electric boiler. Its cylinder is
immersion-heated with no primary pipework, so no primary circuit loss
applies — but `_primary_loss_applies` had no 191 branch, so a 191 main
(main_heating_category 2, "Boiler and radiators, electric") fell through
to the cat-{1,2} boiler branch and accrued ~1177 kWh/yr of phantom
primary loss on the electric-flat segment.

Validated against the cert-2474 worksheet: §4 (59) primary loss = 0,
(64) HW output 1760 (cylinder) + (64a) shower 581. Cert 2474 HW kWh
3585 → 2408; SAP 64.66 → 70.35 (the residual to the lodged 78 is an
Unknown-meter data-fidelity artifact — the register recorded meter_type=3
"Unknown" but the lodged rating used an 18-hour off-peak meter, per RdSAP
§12 / the example worksheets).

Eval mean|err| 1.720 → 1.708 (headline 45.0%, flat ±1 cert — the
electric-flat segment is dominated by the meter data-fidelity artifact).
Regression green (2448 pass incl. golden 6035 + ASHP cohort 1e-4);
pyright net-zero.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-06 21:31:27 +00:00
Khalim Conn-Kowlessar
2bc73fb08d fix(cost): HP-DHW from PCDB heat pump bills Table 12a ASHP_APP_N WH split
When DHW is heated by the main heat pump (WHC 901/902/914 = "from main
system") and the main carries a PCDB Table 362 record,
`_hot_water_fuel_cost_gbp_per_kwh` billed the electric HW at 100% off-peak
low rate (its long-standing TODO). SAP 10.2 Table 12a Grid 1 WH column
(PDF p.191) puts HP-DHW on the ASHP/GSHP-from-database row: 0.70
high-rate fraction at 7-hour and 10-hour → 0.70×14.68 + 0.30×7.50 =
12.526 p/kWh (10-hour), not 7.50 p. The low-rate collapse over-credited
the cat-4 HP-DHW cluster.

Fix: pass the cert WHC into the helper and, for HP-DHW (WHC ∈ {901,902,
914} + PCDB-HP main), bill at the ASHP_APP_N WH blended rate. Electric
IMMERSION (WHC 903) is a different Table 12a row (off-peak immersion 0.17
/ Table 13) and stays on the 100%-low-rate fallback until that slice
lands.

cat-4 cluster (20 certs): mean|err| 2.43→2.11, mean signed +0.06→-0.52
(now per-cert scatter, no systematic bias); cert 9472 +6.4→+3.2, 2789
+6.8→+4.0, 4135 +2.7→within 0.5. Headline mean|err| 1.727→1.720.
Regression green (2447 pass incl. golden 6035 + ASHP cohort at 1e-4);
pyright net-zero.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-06 19:54:01 +00:00
Khalim Conn-Kowlessar
e41a0bc0d7 fix(cost): PCDB heat pump without SAP code bills Table 12a ASHP_APP_N split
A heat pump that resolves via its PCDB Table 362 index alone (API path,
data_source=1, no Table-4a SAP code) had sap_main_heating_code=None, so
`_table_12a_system_for_main` fell through the 211-227/521-524 code-range
gate to None → the "100% off-peak low-rate" fallback. On a Dual meter
(RdSAP §12 Rule 3 routes heat pumps to the 10-hour tariff) this billed
space heating at 7.50 p/kWh instead of the SAP 10.2 Table 12a Grid 1
(PDF p.191) ASHP/GSHP-from-database row: 0.80 high-rate fraction →
0.80×14.68 + 0.20×7.50 = 13.244 p/kWh. The collapse over-credited the
whole cat-4 heat-pump cluster.

Fix: route any main with a PCDB heat-pump record to ASHP_APP_N regardless
of SAP code (a Table 362 record IS an Appendix-N heat pump by
definition). ASHP_APP_N and GSHP_APP_N share the 0.80 SH fraction at
7h/10h, so ASHP_APP_N is the canonical Appendix-N row for the SH split.

cat-4 cluster (20 certs): within-0.5 45%→50%, mean signed +1.43→+0.06,
mean|err| 3.81→2.43; cert 9472 +15.0→+6.4, 2789 +13.4→+6.8. Headline
45.0%→45.1%, mean|err| 1.757→1.727. Regression green (only the
pre-existing test_total_floor_area fails); pyright net-zero.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-06 19:48:37 +00:00
Khalim Conn-Kowlessar
fb350036b1 docs: session-2 API-accuracy handover (fabric+tariff fixes, worksheet path)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-06 19:13:21 +00:00
Khalim Conn-Kowlessar
4d1a58b828 fix(tariff): Unknown meter + storage/CPSU main → off-peak (§12)
Electric storage heaters (and CPSU) charge overnight and cannot run
economically on a single rate, so their presence is physical evidence the
dwelling is on an off-peak tariff. RdSAP 10 §12 (PDF p.62) applied Rules
1-4 only for a Dual meter; an "Unknown" (code 3) meter returned STANDARD
without consulting the heating type, so a cat-7 storage main billed its
overnight charge at the standard 13.19 p/kWh instead of the 7-hour low
rate (5.50 p/kWh) — ~2.4x too high → large under-rate.

Two coupled fixes:
- `rdsap_tariff_for_cert`: for an Unknown meter, infer the off-peak tariff
  from a Rule-1 CPSU (→10-hour) or Rule-2 storage (→7-hour) main; keep
  STANDARD otherwise. Direct-acting/room heaters/heat pumps (Rule 3) are
  NOT off-peak evidence (run on demand, exist on single-rate meters) so
  they stay STANDARD — billing them 100% at the low rate over-credits.
- `_fuel_cost` now resolves its tariff via the §12-aware `_rdsap_tariff`
  (not the raw `tariff_from_meter_type`), so the off-peak branch fires for
  these storage certs and the legacy scalar fields bill the low rate.

Mirrors `_is_off_peak_meter`'s existing Unknown+electric heuristic (which
already routes HW/secondary off-peak), closing the main-space-heating gap.
Meter-3 electric cluster: mean |err| 11.18 → 6.52, within-1.0 3 → 5 (cert
7336 -26.1 → -0.16, 0380 -19.9 → +1.0). Eval headline 44.9% → 45.0%, mean
|err| 1.82 → 1.76, mean signed -0.08 → +0.02. A few storage certs overshoot
(other residuals the standard rate was masking).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-06 19:02:34 +00:00
Khalim Conn-Kowlessar
678aa7affd fix(cascade): main-roof U ignores Room-in-Roof "no insulation" leak
The main pitched/flat roof U-value was derived from the JOINED text of
every roofs[] entry. A room-in-roof carries its own §3.9/§3.10 shell
area + U-value cascade (Table 17 / Table 18 col 4), so a multi-roof cert
lodged "Pitched, insulated (assumed) | Roof room(s), no insulation
(assumed)" leaked the RR's "no insulation" marker into the main roof's
u_roof → U=2.30 applied to the WHOLE main roof, ~3x over-stating its heat
loss. This is the 4700-family regular-roof-U leak.

`_joined_main_roof_descriptions` drops "Roof room(s)" entries before the
main-roof u_roof, falling back to the unfiltered join only for pure-RR
dwellings (every entry an RR) to preserve their prior behaviour. The RR
shell U is unaffected (computed separately) — golden 6035 stays green.

RR-leak cluster (18 certs, RR "no insulation" + a non-RR primary roof):
mean |err| 6.14 → 4.85, within-1.0 0 → 8, within-0.5 0 → 3. Eval headline
44.8% → 44.9%, mean |err| 1.851 → 1.824, mean signed -0.152 → -0.081. Two
certs overshoot (other residuals the leak was masking); the spec rule is
applied uniformly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-06 18:27:41 +00:00
Khalim Conn-Kowlessar
a64e857b94 fix(u-value): "Unknown" roof insulation → Table 18 default, not 2.30
A roof lodged "Unknown loft insulation" carries roof_insulation_thickness
"NI" (Not Indicated → parsed to 0) or "ND" (None): the thickness is
UNDETERMINED, not zero. RdSAP 10 §5.11.4 (p.44) is deterministic here —
"U-values in Table 18 are used when thickness of insulation cannot be
determined" — so the roof takes the Table 18 age-band default (column (1)
pitched / column (3) flat), NOT the uninsulated 2.30 the Table 16 row-0
lookup returns for a parsed-0 thickness. The "Unknown" text is RdSAP's
rendering of the undetermined-thickness observation, distinct from a
genuine "no insulation" lodgement (which keeps 2.30).

u_roof gains an "unknown"-description branch ahead of the parsed-0 → 2.30
path, gated on undetermined thickness (None or 0). Top-floor flats with
"Pitched/Flat, Unknown ... insulation" were the worst electric-flat
under-raters: roof U=2.30 gave HLP ~3.7 on dwellings rated SAP 69-70.

Cluster (14 certs, roof desc contains "unknown", no "no insulation"):
mean |err| 7.79 → 1.82, within-0.5 1→4, within-1.0 1→6. Cert 9836
roof_w_per_k 58.2→10.1, SAP -27.8 → -3.5. Eval headline 44.4% → 44.8%,
mean |err| 1.944 → 1.851. Two certs overshoot (other residuals the wrong
roof-U was masking); the spec value is applied uniformly regardless.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-06 18:20:18 +00:00
Khalim Conn-Kowlessar
3aed8f858a fix(cascade): suppress floor heat loss for "another dwelling below" (code 6)
A floor lodged API floor_heat_loss=6 ("another dwelling below") sits over
another heated dwelling, so it is a party floor with no heat loss (RdSAP
10 §3). The mapper mapped code 6 → None and the heat-transmission step
drove floor exposure solely from the dwelling-level `has_exposed_floor`
flag — which is keyed only on the dwelling_type label and defaults a
"Ground-floor flat" to an exposed floor. So a ground-floor flat above a
basement dwelling kept its full ground-floor heat-loss area.

Map code 6 → "(another dwelling below)" (still != "Ground floor", so the
§5 (12) suspended-timber rule stays inert) and have the cascade suppress
that BP's floor when its floor_type carries the signal, mirroring the
roof's existing "another dwelling above" per-BP party override.

Cert 2115-4121-4711-9361-3686 (ground-floor flat, floor_heat_loss=6):
floor_w_per_k 47.85 → 0; SAP -23.44 → -4.41. Cert 0350-…-6435 -12.38 →
-0.55; 0926-…-9024 -2.35 → -0.82. Eval mean |err| 1.982 → 1.944.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-06 18:05:33 +00:00
Khalim Conn-Kowlessar
6b04514645 fix(mapper): resolve gas-boiler main fuel from §14.2 mains-gas meter
A Summary §14.0 Table 4b gas boiler (SAP code 101-119) lodges no §14.0
"Fuel Type" string in the newer Elmhurst export. The carrier was resolved
only from §15.0 "Water Heating Fuel Type" — fine when the same boiler
heats the water, but a gas boiler paired with a SEPARATE electric
immersion lodges §15.0 "Electricity", so `_elmhurst_gas_boiler_main_fuel`
returned None and the cascade strict-raised MissingMainFuelType.

Cert 001431 boiler-1/boiler-2 "before" variants are exactly this config:
§14.0 SAP code 102/104 (mains-gas boiler), §15.0 electric immersion
(code 909), §14.2 Meters "Main gas: Yes". The meter flag is the
authoritative carrier signal — a 101-119 boiler on mains gas burns mains
gas — so adopt it (SAP10 main_fuel 26 per _ELMHURST_MAIN_FUEL_TO_SAP10
"Mains gas") when §15.0 can't disambiguate. §15.0 gas/LPG still wins when
present (keeps LPG-vs-mains-gas precision); no mains-gas meter + non-gas
§15.0 still strict-raises rather than guessing.

Spec: SAP 10.2 Table 4b "Seasonal efficiency for gas and liquid fuel
boilers" (PDF p.168), rows 101-119. Both certs now resolve main_fuel=26
and compute (was: hard raise).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-06 17:48:04 +00:00
Khalim Conn-Kowlessar
bb8307413f fix(mapper): read sloping_ceiling_insulation_thickness for roof code 8
A "Pitched, sloping ceiling" (roof_construction == 8) lodges its
insulation in the dedicated `sloping_ceiling_insulation_thickness` field,
not `roof_insulation_thickness` (which stays None — the loft-joist field
is meaningless for a slope-following ceiling). The schema dataclasses
dropped that field, so `from_dict` discarded it and the cascade treated
the slope as uninsulated; worse, the pre-1950 None-fallback forced 0 mm
(U=2.30), over-stating roof heat loss ~74%.

Surface the field on SapBuildingPart (schemas 21.0.0 / 21.0.1) and prefer
it in `_api_resolve_sloping_ceiling_thickness` when it carries a NUMERIC
thickness: "100mm" now reaches Table 17 column (1a) "Insulated slope –
sloping ceiling, mineral wool/EPS" (RdSAP 10 §5.11.3 p.44 — 100 mm →
U=0.40) instead of 2.30. Categorical lodgements ("AB" As Built / "NI")
are not measured thicknesses, so they fall through to the existing
as-built rule (Table 18 col (3) via is_pitched_sloping_ceiling).

Cert 9884-3059-9202-7506 (code 8, age B, sloping 100 mm): SAP −5.54 → +0.06.
Cert 8036-2925-6600-0202: −4.94 → +1.55. No regressions in the roof-8
cohort (the "AB" certs are unchanged). Eval headline 43.8% → 44.3% within
0.5; golden fixtures incl. 6035 green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-06 17:37:17 +00:00
Khalim Conn-Kowlessar
98f71d2554 feat(diag): per-component cost decomposition for API SAP errors
Mirror of eval_api_sap_accuracy.py that decomposes each cert's SAP error
into per-component energy/cost deltas WITHOUT generating an Elmhurst
worksheet. Calibrates the consumer price from the certs we already get
right (gas £0.0809/kWh n=291, elec £0.2839/kWh n=326 over |SAP err|<0.4),
then for every cert compares our_component_kWh × price to the lodged
heating_cost_current / hot_water_cost_current / lighting_cost_current and
back-calculates a numeric energy target (lodged_cost / price).

Clusters errors by (component × direction). On the 905-cert sample this
reveals heat:high (we over-state heating energy → under-rate SAP) as the
dominant broken cluster: 332 certs, only 36.7% within 0.5. Output CSV at
<cache>/_cost_decomposition.csv.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-06 17:37:05 +00:00
Khalim Conn-Kowlessar
27375d93a4 fix(u-value): solid brick as-built U by thickness — §5.7 Table 13
A 440 mm (>420 mm) solid brick AS-BUILT wall computed U = 1.70 (the
220 mm bucket default) instead of the RdSAP-correct 1.10. The §5.7
Table 13 thickness path only fired for *insulated* brick (external/
internal + thickness > 0); the as-built case fell through to the
Table 6 cavity/solid age-band default.

Spec: RdSAP 10 Specification (9th June 2025), §5.7 "U-values for
uninsulated brick walls, age bands A to E", Table 13 (PDF p.40):
  ≤200 mm → 2.5, 200–280 mm → 1.7, 280–420 mm → 1.4, >420 mm → 1.1.
Table 6 footnote (b) on the "Solid brick as built" row (PDF p.40):
"Or from 5.7 if wall thickness is other than 200mm to 280mm" — the
thickness table supersedes the flat 1.7 default whenever a documentary
wall thickness is lodged (200–280 mm gives 1.7 either way). The §5.8 /
Table 14 dry-lining R is added on top only when the wall is dry-lined,
per the §5.7 closing sentence.

Validated against the user-generated Elmhurst worksheet "simulated
case 21" (replica of API cert 2818-3053-3203-2655-9204: mid-terrace,
age band B, solid brick as-built 440 mm, room-in-roof). New §3 cascade
pin `test_section_3_wall_u_by_thickness_case21_match_pdf` routes the
Summary through the real extractor + mapper and pins:
  (31) 155.1000, (33) 175.6208, (36) 23.2650, (37) 198.8858 — all 1e-4.
External walls Main U → 1.1000; Sheltered RR gable → 1/(1/1.10+0.5) =
0.71 (was 0.92). Pinned on §3 only (case-6 precedent): its code-908
instantaneous multi-point gas water heater has a separate §4 (219) gap.

Cross-check: sim case 20 (220 mm) stays at 1.70 — unchanged.

API SAP accuracy (scripts/eval_api_sap_accuracy.py, 896 computed certs):
% |err| < 0.5 SAP vs lodged: 42.6% → 43.8%; mean |err| 2.045 → 2.010.

Regression: tests/domain/sap10_calculator/ (1861), backend/
documents_parser/tests/ (574), datatypes/epc/ + rdsap golden fixtures
all green (pre-existing test_total_floor_area excepted). pyright strict
net-zero. No solid-brick fixture pin shifted (200–280 mm unchanged).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-06 14:40:06 +00:00
Khalim Conn-Kowlessar
cdf211393c feat(mapper): map API gable_wall_type 2/3 (Sheltered/Connected) — clears 14 raises
The 2026 API sample raised UnmappedApiCode on `gable_wall_type` 2 (10 certs)
and 3 (4 certs) — the two RR gable variants beyond Party(0)/Exposed(1).
Sim case 21 (an Elmhurst replica of API cert 2818-3053-3203-2655-9204:
gable_wall_type_1=2, gable_wall_type_2=3) lodges them as "Sheltered" and
"Connected", confirming **2=Sheltered, 3=Connected**.

- Mapper: `_API_TYPE_1_GABLE_TYPE_TO_KIND` gains 2 → `gable_wall_sheltered`,
  3 → `connected_wall` (U=0, area deducts — already handled).
- Calculator: new `gable_wall_sheltered` branch. The API path lodges no
  per-gable U, so the cascade DERIVES it as RdSAP 10 Table 4 (p.22)
  Sheltered = 1/(1/U_wall + 0.5) — back-solved + validated against case 21
  (U_wall 1.10 → 0.71) and case 20 (1.70 → 0.92). A lodged U (Summary path)
  still rides through as an override.

API sample: 14 raises clear → `computed` 882 → 896, `raise:ValueError` 16 → 2.
Summary path unchanged (Sheltered stays `gable_wall_external` + lodged U, so
cert 000487's hand-built fixture is untouched). 2861 pass (lone
test_total_floor_area pre-existing); pyright strict net-zero (32=32 / 12=12).

NOTE: the derived Sheltered U on cert 2818 lands at 0.92 not 0.71 because the
cascade computes its 440 mm solid-brick wall U as 1.70 (the 220 mm default) —
a SEPARATE wall-U-vs-thickness bug (next slice, validated by case 21's 1.10).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-06 11:59:50 +00:00
Khalim Conn-Kowlessar
7dfe3f2c99 feat(test): case-20 cascade fixture + close its CO2 via E7 per-end-use codes
Locks sim case 20 (storage heaters + Detailed RR + loose-jacket cylinder)
as a golden vector: _elmhurst_worksheet_001431_case20.build_epc() routes the
Summary PDF through extractor → mapper → calculator, registered in
test_e2e_elmhurst_sap_score with all 11 SapResult headline pins at 1e-4.
10 pinned exact off slices 1-2 (window extractor, RR stud walls); this slice
closes the last one, co2_kg_per_yr (was 3797.62 vs (272) 3815.4060).

Root cause: on a dual-rate (E7) meter the CO2 path ignored the tariff's
high/low Table-12 electricity codes that the cost path already uses:
  - Secondary (direct-acting portable heaters, on-peak) keyed the monthly
    Table 12d cascade on standard code 30 (0.15405) instead of the E7 HIGH
    code 32 → (263) 0.1616. SAP 10.2 Table 12a Grid 1 direct-acting electric
    is 100% high-rate; mirrors the cost side billing it at 15.29 p/kWh.
  - Main storage heaters fell through `_table_12a_system_for_main`=None to
    the FLAT annual factor (0.136) rather than the dual-rate LOW code: per
    the Table 12a design intent ("storage … 100% low rate") they charge
    off-peak → E7 LOW code 31 → (261) 0.1357.

case-20 co2 now EXACT. 2433 calculator + 112 golden + documents_parser tests
pass — no dual-meter/storage cohort regression; pyright strict net-zero (32=32).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-06 11:23:10 +00:00
Khalim Conn-Kowlessar
1ed6d06804 fix(mapper): drop only U=0 internal RR stud walls, keep positive-U ones
A Detailed room-in-roof lodges "Stud Wall" surfaces, but the cascade billed
every one through Table 17 from its insulation — over-counting fabric on
internal studs that carry no heat loss. sim case 20's two studs lodge §8.1
Default U-value 0.00 and the P960 worksheet omits them from BOTH fabric heat
loss (§3: (33)=285.9847) and total exposed area (31)=239.68; the cascade
computed ~0.52 each → (33) +4.16 W/K and continuous SAP 43.05 vs 43.6322.

Gate the drop on the lodged Default U-value: 0.00 → internal knee wall,
return None (no heat loss, no area); positive → a real exposed knee wall
(cert 000565 Ext2 Detailed: 0.31 / 0.10) that still falls through to the
Table-17 path. The earlier over-broad "drop all studs" zeroed 000565's
genuine studs — this keeps them.

Pins test_summary_001431_case20_fabric_heat_loss_matches_worksheet_line_33
((33)=285.9847 at 1e-4); case 20 continuous SAP now EXACT (43.6322). 2850
pass (the lone test_total_floor_area failure is pre-existing on base);
pyright strict net-zero (32=32).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-06 10:47:30 +00:00
Khalim Conn-Kowlessar
795d36b732 fix(extractor): re-join §11 windows whose Area cell split onto its own line
Sim case 20's §11 lodges 5 windows but only 1 surfaced. The "W H Area"
cells tokenize inconsistently: a narrow Area column keeps all three on one
line ("1.80 2.10 3.78" — matches _WIDTH_HEIGHT_AREA_RE), but a wider Area
column triggers pdftotext's 2+-space split, dropping the Area onto its own
line ("5.79 2.00" then "11.58"). The 3-decimal data anchor never matched
those four rows, so they were lost — gutting §6 solar gains (5 windows →
1) and dropping continuous SAP 43.05 → 38.32 vs the worksheet's 43.6322.

Pre-merge a "W H" line + a following lone-decimal Area into the canonical
"W H Area" line, gated on Area ≈ W × H (the §11 Area is always the product)
so a frame factor / g-value / U-value below a dimension line is never
absorbed. One-line layouts (3 decimals) are untouched.

Pins via test_summary_001431_case20_extracts_all_five_section11_windows
(Summary_001431_case20.pdf mirrors sap worksheets/golden fixture debugging/
simulated case 20/). 573 documents_parser tests pass; pyright strict net-zero.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-06 10:35:21 +00:00