Commit graph

7897 commits

Author SHA1 Message Date
Daniel Roth
6c42065304
Merge pull request #1358 from Hestia-Homes/feature/batch-save-and-delete-epc
Batch saving and deleting of epc property data, bulk saving of plans
2026-06-29 16:46:19 +01:00
Khalim Conn-Kowlessar
8914496456 test(accuracy): ratchet after HP water-heating 100% floor 🟪
The Appendix N3.7 water-heating 100% floor drops corpus MAE 0.726 -> 0.721
and lifts within-0.5 74.1% -> 74.2% on the 1000-cert RdSAP-21.0.1 sample
(cert 100110101713 moves inside +-0.5). Tighten the MAE ceiling to 0.722 and
the within-0.5 floor to 0.742, and log the slice.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 15:40:53 +00:00
Daniel Roth
c202f15d8c fix broken unit tests 2026-06-29 15:40:22 +00:00
Khalim Conn-Kowlessar
dbaf3c49e2 fix(hp): floor heat-pump water-heating efficiency at 100% (App N3.7) 🟩
_heat_pump_apm_efficiencies applied the Table N8 in-use factor to the PCDB
water-heating efficiency but omitted the spec's "subject to a minimum
efficiency of 100%" clause (SAP 10.2 Appendix N3.7, PDF p.109). An oversized
heat pump whose PSR-extended water,3 x the 0.60 in-use factor fell below
100% therefore billed water heating at that sub-100% efficiency, over-
counting hot-water fuel.

Apply max(in_use x eta_water, 100%). Accredited Elmhurst worksheet for cert
100110101713 (golden fixture case 56, PCDB 100061): water (216) = 100.0000,
which we now match (was 77.13%). Combined with the space-heating PSR-
extension fix the cert lands 72.51 vs lodged 73 (|err| 18.32 -> 0.49).

Data-driven: only the single oversized-PSR cert moves (in-range heat pumps
keep their > 100% water COP, e.g. case 54 112.5%, case 55 179.6%). Corpus
(RdSAP-21.0.1 n=1000) MAE 0.726 -> 0.721, within-0.5 74.1% -> 74.2%.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 15:38:31 +00:00
Khalim Conn-Kowlessar
672e6679c8 test(hp): floor heat-pump water-heating efficiency at 100% (App N3.7) 🟥
SAP 10.2 Appendix N3.7 ("Thermal efficiency for water heating - heat pumps",
PDF p.109): "multiply the thermal efficiency for water heating by the in-use
factor in Table N8; subject to a minimum efficiency of 100%." Our
_heat_pump_apm_efficiencies applies the in-use factor but omits the floor.

Anchored to golden fixture case 56 (PCDB 100061, cert 100110101713): an
oversized HP (PSR 3.107) extends water,3 198.9% -> 128.55%, x 0.60 in-use =
77.13% < 100% -> the accredited Elmhurst worksheet (216) reads 100.0000, we
read 77.13%. In-range PSR keeps 0.60 x 198.9 = 119.34% (above the floor).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 15:33:04 +00:00
Jun-te Kim
0536f70162 Take a Postcode value object at the historic EPC repository boundary 🟩
The port accepts a normalised Postcode and rejects malformed/empty ones via
Postcode.is_valid() (PostcodeNotFound) — dropping the per-lookup postcodes.io
HTTP call and the cross-module use of the private _sanitise_postcode. Mapper
helper promoted to public.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 15:24:18 +00:00
Jun-te Kim
23dfb3f899 Add UK-postcode format validity to the Postcode value object 🟩
A pure, offline regex check (no network) so a frozen value object stays pure.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 15:24:18 +00:00
Khalim Conn-Kowlessar
235c2ca466 Merge remote-tracking branch 'origin/main' into fix/heat-pump-flow-temp-efficiency 2026-06-29 15:17:10 +00:00
Khalim Conn-Kowlessar
2d6479536c test(accuracy): ratchet SAP MAE 0.740 -> 0.726 after HP PSR-extension 🟪
The heat-pump PSR-extension fix (SAP 10.2 Appendix N2) drops corpus MAE from
0.740 to 0.726 on the 1000-cert RdSAP-21.0.1 sample; within-0.5 holds at
74.1%. Tighten the ceiling and log the slice.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 15:16:07 +00:00
Jun-te Kim
d732ec3fa3
Merge pull request #1357 from Hestia-Homes/chore/devcontainer-pyright
Add pyright + pandas-stubs to backend devcontainer
2026-06-29 16:14:39 +01:00
Khalim Conn-Kowlessar
51d8f65aac fix(pcdb): extend heat-pump efficiency toward 100% beyond table PSR range 🟩
interpolate_heat_pump_efficiency_at_psr clamped to the smallest/largest PSR
row when the dwelling's plant size ratio fell outside the record's range.
That is the SAP 10.2 Appendix N rule for *combined heat-pump-and-boiler*
packages, not for a plain air/ground/water source heat pump.

Per Appendix N2 (PDF p.101, footnotes 44/45) a source heat pump whose PSR
exceeds the record's largest value takes a reciprocal-linear interpolation
between the largest-PSR efficiency and 100% at twice that PSR (100% beyond),
and 100% when the PSR is below the record's smallest value. Both the space-
and water-heating PSR-dependent efficiencies extend this way.

Effect: an oversized heat pump in a small dwelling is no longer credited the
full top-of-table COP. Accredited Elmhurst worksheet for cert 100110101713
(golden fixture case 56, PCDB 100061, PSR 3.107 over largest 2.0): (206)
334.4% -> 139.66% = Elmhurst exact. Corpus (RdSAP-21.0.1, n=1000) MAE
0.7397 -> 0.7258, within-0.5 0.7410 held; only two certs move (both
oversized-PSR heat pumps), 100110101713 +18.32 -> -4.97.

Exhaust-air and combined heat-pump-and-boiler packages have different
boundary rules (straight-to-100% / clamp-to-edge) but are not distinguished
by the current PCDB parse; the air/ground/water rule is applied uniformly,
a documented limitation noted in the function docstring.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 15:13:16 +00:00
Jun-te Kim
c2e6b88744 Add pyright + pandas-stubs to backend devcontainer for the strict type gate
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 15:10:40 +00:00
Daniel Roth
4764bc7c15 Batch plan saves reduce RDS CPU during bulk modelling runs 🟪
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-29 15:08:47 +00:00
Khalim Conn-Kowlessar
3d93c7b7d5 test(pcdb): PSR-extension toward 100% above largest / below smallest 🟥
SAP 10.2 Appendix N2 (PDF p.101, footnote 44/45): for an air/ground/water
source heat pump whose plant size ratio exceeds the record's largest PSR,
the efficiency is reciprocal-interpolated between the largest-PSR value and
100% at twice the largest PSR (100% beyond that); below the smallest PSR the
efficiency is 100%. Our interpolator instead clamps to the top/bottom row.

Anchored to the accredited Elmhurst worksheet for cert 100110101713 (golden
fixture case 56, PCDB 100061): PSR 3.10665 over the record's largest 2.0
gives eta_space,1 = 147.011 -> (206) = 0.95 x 147.011 = 139.660, vs the
clamped 352.0 -> 334.4% that over-rates the dwelling by +18 SAP.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 15:08:15 +00:00
Daniel Roth
46ca714ef9 Batch plan saves reduce RDS CPU during bulk modelling runs 🟩
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-29 15:06:47 +00:00
Daniel Roth
9c6b477025 Batch plan saves reduce RDS CPU during bulk modelling runs 🟥
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-29 15:04:54 +00:00
Jun-te Kim
fa6357a5cf Re-plug historic EPC lookup onto the repository; a missing postcode now yields empty matches 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 14:58:23 +00:00
Jun-te Kim
03687041c9 Return no historic UPRN for missing, ambiguous, or zero-score matches 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 14:52:42 +00:00
Jun-te Kim
b47c90f9cd Resolve an address to its unambiguous historic EPC UPRN 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 14:52:20 +00:00
Jun-te Kim
b8c376bfd6 Resolve an address to its unambiguous historic EPC UPRN 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 14:52:06 +00:00
Jun-te Kim
a30ab59f3c Compose repository and matcher into scored historic EPC matches 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 14:51:45 +00:00
Jun-te Kim
d7fd093c4f Compose repository and matcher into scored historic EPC matches 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 14:51:33 +00:00
Jun-te Kim
d084a0a785 Rank historic EPC records by address similarity 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 14:50:03 +00:00
Jun-te Kim
d41eb84530 Rank historic EPC records by address similarity 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 14:49:49 +00:00
Jun-te Kim
b059d55dd2 Strip trailing .0 from historic EPC UPRN 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 14:49:08 +00:00
Jun-te Kim
be2b71c235 Strip trailing .0 from historic EPC UPRN 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 14:48:46 +00:00
Jun-te Kim
66b776e0aa Raise PostcodeNotFound for an unusable postcode 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 14:48:06 +00:00
Jun-te Kim
49becf1ada Raise PostcodeNotFound for an unusable postcode 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 14:47:50 +00:00
Jun-te Kim
cd503411d2 Build S3 key from sanitised postcode and propagate non-missing read errors 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 14:47:18 +00:00
Jun-te Kim
88f4739ce8 Return empty list for a postcode with no historic EPC shard 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 14:46:42 +00:00
Jun-te Kim
6a095a2cf2 Return empty list for a postcode with no historic EPC shard 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 14:46:22 +00:00
Jun-te Kim
a372f2995f Map historic EPC S3 shard to domain records 🟩
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 14:45:25 +00:00
Jun-te Kim
74b8ad86b1 Map historic EPC S3 shard to domain records 🟥
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 14:44:56 +00:00
Daniel Roth
9f0dd067f8 Merge branch 'main' into feature/batch-save-and-delete-epc 2026-06-29 14:38:57 +00:00
Daniel Roth
597510b996
Merge pull request #1353 from Hestia-Homes/feature/dont-resave-epc-if-refetch-false
Don't re-save EPC data which has been read from database rather than fetched. Also turn on database insights
2026-06-29 15:14:14 +01:00
Daniel Roth
0718533d3a turn on rds performance insights 2026-06-29 14:09:24 +00:00
KhalimCK
8b0b5c0387
Merge pull request #1352 from Hestia-Homes/fix/ground-floor-flat-default-heating
fix(sap): credit PV only when connected to the dwelling's meter
2026-06-29 15:06:29 +01:00
Daniel Roth
b1406bb482 Skip re-saving EPC data when it was read from DB unchanged
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-29 14:03:23 +00:00
Khalim Conn-Kowlessar
08d9e46481 docs(pv): document gov-API pv_connection enum on the domain field 🟪
0 = no PV, 1 = not connected to dwelling meter, 2 = connected. Decoded
empirically from the corpus + accredited Elmhurst (the gov RdSAP schema
types it as a bare int). Points readers to the crediting gate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 13:50:01 +00:00
Khalim Conn-Kowlessar
9927169e23 test(accuracy): ratchet SAP gauge after PV connection gate 🟪
within-0.5 floor 0.73->0.74 (now 0.741), MAE ceiling 0.762->0.740 (now
0.7397) on the fixed RdSAP-21.0.1 corpus. Log entry appended.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 13:43:30 +00:00
Khalim Conn-Kowlessar
8606cab5f0 fix(pv): credit PV only when connected to the dwelling's meter 🟩
Gate PV generation/credit in cert_to_inputs on gov-API pv_connection:
credit only when ==2 ('connected'); ==1 ('present but not connected to the
dwelling's meter') contributes zero to the dwelling's cost/CO2/PE per
RdSAP 10 §11.1 / SAP 10.2 Appendix M. Non-int (None / site-notes str) keeps
the credit-if-array behaviour, so the Elmhurst/Summary + synthetic paths are
unchanged (no regression).

Corpus: all 5 pv_connection=1 PV certs move inside ±0.5 (e.g. 100051118081
+6.5→+0.5); MAE 0.760→0.740, within-0.5 73.8→74.3%, no regression
(pv_connection=2 certs keep their credit).

Also corrects a now-load-bearing latent bug: the solar-recommendation
overlay tagged recommended arrays pv_connection=1 ('not connected') — which
the new gate would zero. A new install connects to the dwelling's meter, so
it must be 2; pinned by the overlay test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 13:40:51 +00:00
Daniel Roth
b426940136
Merge pull request #1350 from Hestia-Homes/modelling-e2e-12-concurrency
Decrease e2e modelling concurrency from 32 to 12
2026-06-29 14:36:01 +01:00
Daniel Roth
66fa08af19 update comment 2026-06-29 13:33:48 +00:00
Daniel Roth
71b304fa47 decrease lambda concurrency from 32 to 12 2026-06-29 13:32:53 +00:00
Khalim Conn-Kowlessar
775232b4e7 test(pv): credit PV only when connected to dwelling meter (pv_connection) 🟥
RdSAP 10 §11.1 / SAP 10.2 Appendix M: PV is included in a dwelling's
assessment only if connected to the dwelling's own electricity meter. The
gov-API pv_connection enum encodes this — 0=no PV, 1=present-but-not-
connected, 2=connected. Corpus-validated (57 PV certs: pv_connection=1 MAE
4.48->1.22 without credit, 0/5 need it; pv_connection=2 needs it, MAE 0.98
vs 10.29) and Elmhurst-proven (connected SAP 87 vs not-connected 74).

cert_to_inputs currently credits a pv_connection=1 array; the test pins that
it must contribute zero generation. Adds pv_connection to make_minimal_sap10_epc.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 13:32:20 +00:00
Daniel Roth
f27d1e21bb Batch EPC writes in EpcPostgresRepository pass pyright strict 🟪
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-29 13:17:29 +00:00
Daniel Roth
0fa1b9001c Batch EPC writes in _flush_writes: two save_batch() calls instead of N save() calls 🟩
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-29 13:09:56 +00:00
Daniel Roth
587465bff7 Batch EPC writes via save_batch() on EpcPostgresRepository 🟩
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-29 13:04:30 +00:00
Daniel Roth
fe69bccf22 Batch EPC writes via save_batch() on EpcPostgresRepository 🟥
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-29 12:19:49 +00:00
KhalimCK
98ba8d89dc
Merge pull request #1343 from Hestia-Homes/fix/effective-baseline-overrides-and-epc-fidelity
fix(baseline): honour EPC + overrides as the authoritative Effective baseline
2026-06-29 11:19:52 +01:00