Amends ADR-0004's both-halves-always-populated invariant: EPC Prediction
borrows a neighbour's recorded figures, so a predicted Property has no Lodged
Performance — its lodged half is absent (NULL), only the Effective half is
persisted. Sharpens the CONTEXT.md glossary for Lodged / Baseline Performance.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replaces sfr/principal_pitch/2_export_data.py, which read the retired
plan_recommendations m2m and recommendation_materials table. The new model
links a recommendation to its plan directly (recommendation.plan_id), keeps
materials inline on the recommendation (material_id), marks the chosen plan
per (scenario, property) with is_default, and stores post-works SAP/EPC and
savings on the plan row (the new SAP calculator's output).
Takes a portfolio id, resolves every modelled scenario (those with plans),
and writes one workbook with a properties sheet per scenario. EPC descriptive
fields are sourced live from the EPC service (property_details_epc is dead);
property_type falls back override -> cert.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The PV-array (ea7f4f43) and electricity-tariff (6ec09892) mapper fixes shifted
the observed output of five frozen gates that weren't updated alongside:
- has_pv component-accuracy floor 0.9798 -> 0.9697: carrying full-SAP lodged PV
now reads the true has_pv=True for full-SAP PV dwellings, so the leave-one-out
scorer's actual changes (ground-truth-method shift, ADR-0037 pattern).
- uprn_10093116528 80->... pin 82 -> 83: tariff=1 (standard) was wrongly read as
dual/Economy 7; translating to "single" re-prices the gas semi's electricity.
- uprn_10096028301 82 -> 84, uprn_10023444324 80 -> 82 (== lodged 82),
uprn_10023444320 81 -> 83: now credit the lodged sap_energy_source.pv_arrays
the schema previously dropped. Comments document the per-cert PV/Elmhurst
relationship (incl. the mid-floor sibling landing +2 over its lodged integer).
Pre-existing, unrelated failures untouched: the missing
sap_16_0_full_no_floor_dims.json fixture and the RdSAP-21 floor-area test (both
reproduce on origin/main).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Confirming regression test for the prime-lead suspicion (SAP-16.x meter
over-rating). A full-SAP-shaped 16.x cert lodges the SAP energy_tariff code
space; the assessment_type=SAP gate routes it to the full-SAP mapper, which
translates the code (1=standard → STANDARD) rather than raw-passing it to the
RdSAP normaliser (where 1="dual" → SEVEN_HOUR, the over-rating bug). No fix
needed — the gate already prevents the drop; this pins that behaviour.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add code 5 → "off-peak 18 hour" to _SAP_TARIFF_TO_RDSAP_METER_TYPE so the
full-SAP meter translation covers the whole energy_tariff code space, not
just 1-4. Table 12a resolves the word alias to Tariff.EIGHTEEN_HOUR.
Real SAP-19.1.0/19.2.0 sample certs lodge tariff 5 and now resolve to the
18-hour off-peak split instead of defaulting to STANDARD (under/over-rated).
The same fix covers 17.0/17.1/18.0.0/19.0.0 — all route through the full-SAP
mapper via the assessment_type=SAP gate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The full-SAP energy_tariff code space includes 5=off-peak-18-hour, but
_sap_17_1_meter_type only maps 1-4, dropping code 5 to "" → STANDARD.
Real SAP-19.1.0/19.2.0 certs lodge tariff 5, so the 18-hour off-peak
split is silently lost.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Crediting the full-SAP lodged PV closes the 2-point under-rating on uprn
10023444324 — the engine now reproduces the accredited lodged value exactly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Full-SAP certs lodge measured PV under sap_energy_source.pv_arrays, but the
SAP-Schema-17.1 SapEnergySource modelled only wind, so from_dict dropped it at
parse and the mapper hardcoded no PV. A new SapPvArray schema field + the
_sap_17_1_pv_arrays mapper helper carry it to the domain photovoltaic_arrays the
Appendix-M generation credit reads — without it an all-electric dwelling the
array lifts to A/B is mis-modelled down a band or more.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A landlord heating override DESCRIBES the dwelling, so its assumed off-peak meter
(Dual/E7) is a coherent default — it must not downgrade a cert that already
lodges a more-off-peak meter (e.g. property 709874's 24-hour all-low tariff). The
overlay opts in via keep_existing_off_peak_meter; _fold_heating then keeps the
cert's meter when both it and the desired meter are off-peak. Heating MEASURES
build HeatingOverlay directly and leave the flag False, so they still re-meter
for real (Elmhurst re-lodges 18-hour -> Dual on a storage install) — the cascade
pins stay green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
System-built is RdSAP code 6, colliding with the gov-EPC code-6 basement
sentinel that main_wall_is_basement falls back to. The wall overlay now pins
wall_is_basement=False for system-built, mirroring the gov-API mapper's
_clear_basement_flag_when_system_built, so the override no longer adds a phantom
basement (which inflated both wall and ground-floor heat loss).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Full-SAP energy_tariff (1=standard/2=7hr/3=10hr/4=24hr) and RdSAP meter_type
(1=dual/2=single/3=unknown/4=24hr) are different code spaces; the mapper passed
the code through untranslated, reading standard-tariff full-SAP certs as Economy 7
(over-rated) and Economy-7 certs as single (under-rated). Map onto the RdSAP word
aliases so the resolved Table 12a tariff is correct; absent/ND -> unknown.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Electric room heaters (691) get a Dual meter from the overlay (not single-rate):
an all-electric room-heater dwelling realistically bills on Economy 7, and the
§12 dispatch then applies a high/low split rather than a single-rate over-penalty.
Overlay owns its assumed-meter policy via _ASSUMED_DUAL_METER_CODES (the §12
off-peak systems + room heaters), keeping OFF_PEAK_IMPLYING_HEATING_CODES pure.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Portfolio-796/scenario-1268 audit found the bulk of MEDIUM/HIGH anomalies
(already-meets-goal-with-works, zero-works-post-differs, plan-score-below-
baseline, and the non-fuel-switch slice of negative-bill-savings) trace to one
root cause: the persisted default plan is stale relative to the live model, so
it resolves on re-model rather than a code change. Confirmed via
run_modelling_e2e on three samples (stored vs live plans differ wholesale).
No new check added — the staleness is already surfaced by the existing checks
and addressed by the override-aware-rebaseline / persistence-fidelity work, so
a new check would only re-flag known divergence. Instead record the expectation
and the "audit the default plan only" rule in the skill Notes so the next
reviewer starts ahead. References kept to durable docs/adr (no PR numbers).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
'Electric room heaters' archetype -> SAP Table 4a code 691 (panel/convector/
radiant), on a single-rate meter (691 is not off-peak-implying, unlike storage).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Appendix N3.7 water-heating 100% floor drops corpus MAE 0.726 -> 0.721
and lifts within-0.5 74.1% -> 74.2% on the 1000-cert RdSAP-21.0.1 sample
(cert 100110101713 moves inside +-0.5). Tighten the MAE ceiling to 0.722 and
the within-0.5 floor to 0.742, and log the slice.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_heat_pump_apm_efficiencies applied the Table N8 in-use factor to the PCDB
water-heating efficiency but omitted the spec's "subject to a minimum
efficiency of 100%" clause (SAP 10.2 Appendix N3.7, PDF p.109). An oversized
heat pump whose PSR-extended water,3 x the 0.60 in-use factor fell below
100% therefore billed water heating at that sub-100% efficiency, over-
counting hot-water fuel.
Apply max(in_use x eta_water, 100%). Accredited Elmhurst worksheet for cert
100110101713 (golden fixture case 56, PCDB 100061): water (216) = 100.0000,
which we now match (was 77.13%). Combined with the space-heating PSR-
extension fix the cert lands 72.51 vs lodged 73 (|err| 18.32 -> 0.49).
Data-driven: only the single oversized-PSR cert moves (in-range heat pumps
keep their > 100% water COP, e.g. case 54 112.5%, case 55 179.6%). Corpus
(RdSAP-21.0.1 n=1000) MAE 0.726 -> 0.721, within-0.5 74.1% -> 74.2%.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SAP 10.2 Appendix N3.7 ("Thermal efficiency for water heating - heat pumps",
PDF p.109): "multiply the thermal efficiency for water heating by the in-use
factor in Table N8; subject to a minimum efficiency of 100%." Our
_heat_pump_apm_efficiencies applies the in-use factor but omits the floor.
Anchored to golden fixture case 56 (PCDB 100061, cert 100110101713): an
oversized HP (PSR 3.107) extends water,3 198.9% -> 128.55%, x 0.60 in-use =
77.13% < 100% -> the accredited Elmhurst worksheet (216) reads 100.0000, we
read 77.13%. In-range PSR keeps 0.60 x 198.9 = 119.34% (above the floor).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The heat-pump PSR-extension fix (SAP 10.2 Appendix N2) drops corpus MAE from
0.740 to 0.726 on the 1000-cert RdSAP-21.0.1 sample; within-0.5 holds at
74.1%. Tighten the ceiling and log the slice.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
interpolate_heat_pump_efficiency_at_psr clamped to the smallest/largest PSR
row when the dwelling's plant size ratio fell outside the record's range.
That is the SAP 10.2 Appendix N rule for *combined heat-pump-and-boiler*
packages, not for a plain air/ground/water source heat pump.
Per Appendix N2 (PDF p.101, footnotes 44/45) a source heat pump whose PSR
exceeds the record's largest value takes a reciprocal-linear interpolation
between the largest-PSR efficiency and 100% at twice that PSR (100% beyond),
and 100% when the PSR is below the record's smallest value. Both the space-
and water-heating PSR-dependent efficiencies extend this way.
Effect: an oversized heat pump in a small dwelling is no longer credited the
full top-of-table COP. Accredited Elmhurst worksheet for cert 100110101713
(golden fixture case 56, PCDB 100061, PSR 3.107 over largest 2.0): (206)
334.4% -> 139.66% = Elmhurst exact. Corpus (RdSAP-21.0.1, n=1000) MAE
0.7397 -> 0.7258, within-0.5 0.7410 held; only two certs move (both
oversized-PSR heat pumps), 100110101713 +18.32 -> -4.97.
Exhaust-air and combined heat-pump-and-boiler packages have different
boundary rules (straight-to-100% / clamp-to-edge) but are not distinguished
by the current PCDB parse; the air/ground/water rule is applied uniformly,
a documented limitation noted in the function docstring.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SAP 10.2 Appendix N2 (PDF p.101, footnote 44/45): for an air/ground/water
source heat pump whose plant size ratio exceeds the record's largest PSR,
the efficiency is reciprocal-interpolated between the largest-PSR value and
100% at twice the largest PSR (100% beyond that); below the smallest PSR the
efficiency is 100%. Our interpolator instead clamps to the top/bottom row.
Anchored to the accredited Elmhurst worksheet for cert 100110101713 (golden
fixture case 56, PCDB 100061): PSR 3.10665 over the record's largest 2.0
gives eta_space,1 = 147.011 -> (206) = 0.95 x 147.011 = 139.660, vs the
clamped 352.0 -> 334.4% that over-rates the dwelling by +18 SAP.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
0 = no PV, 1 = not connected to dwelling meter, 2 = connected. Decoded
empirically from the corpus + accredited Elmhurst (the gov RdSAP schema
types it as a bare int). Points readers to the crediting gate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
within-0.5 floor 0.73->0.74 (now 0.741), MAE ceiling 0.762->0.740 (now
0.7397) on the fixed RdSAP-21.0.1 corpus. Log entry appended.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Gate PV generation/credit in cert_to_inputs on gov-API pv_connection:
credit only when ==2 ('connected'); ==1 ('present but not connected to the
dwelling's meter') contributes zero to the dwelling's cost/CO2/PE per
RdSAP 10 §11.1 / SAP 10.2 Appendix M. Non-int (None / site-notes str) keeps
the credit-if-array behaviour, so the Elmhurst/Summary + synthetic paths are
unchanged (no regression).
Corpus: all 5 pv_connection=1 PV certs move inside ±0.5 (e.g. 100051118081
+6.5→+0.5); MAE 0.760→0.740, within-0.5 73.8→74.3%, no regression
(pv_connection=2 certs keep their credit).
Also corrects a now-load-bearing latent bug: the solar-recommendation
overlay tagged recommended arrays pv_connection=1 ('not connected') — which
the new gate would zero. A new install connects to the dwelling's meter, so
it must be 2; pinned by the overlay test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
