§3 close (LINE_31/33/36/37 exact for both non-RR Elmhurst worksheets) is
now landed across slices 344a9c9d..cf244762. HANDOVER_S3_CLOSE.md was
written as a mid-stream working brief; with §3 done it now creates doc
rot, so it's removed in favour of SPEC_COVERAGE.md as the single source
of truth.
SPEC_COVERAGE.md updates:
- §3 marked Full (non-RR); RR sub-area deferral noted
- §4 carries the ordered slice plan for the worksheet-driven rewrite
(xlsx rows 207–304, line refs (42)..(65))
- Hierarchy callout: the canonical SAP10.2 algorithm lives in the
repo-root xlsx, not in any handover doc
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Closes the second non-RR Elmhurst worksheet (mid-terrace, 3 parts).
LINE_33 (209.1084) and LINE_37 (232.1169) reproduce to 0.1 W/K.
Cert inputs lodged on the fixture:
- Ext1 SapFloorDimension(is_exposed_floor=True) — Table 20 route
- Ext2 ground floor (tiny 1.35 m², P=3.30) stays on Table 19 fn 1
suspended-timber default for age B (cascade → U≈1.25, worksheet 1.25)
- door_count=2 → 3.70 m² total door area
- WINDOW_TOTAL_AREA_M2=11.72 split across two glazing types
(Type 1: 6.22 m² post-2002 raw U=2.0, Type 2: 5.50 m² pre-2002 raw
U=2.8). Area-weighted aggregate raw U=2.37 reproduces the worksheet's
25.37 W/K through the curtain-resistance transform.
Non-RR §3 scope closed:
- LINE_31 exact (existing test)
- LINE_33 exact ← this slice + the 000490 slice
- LINE_36 exact (existing test, y × LINE_31)
- LINE_37 exact ← this slice + the 000490 slice
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
End-to-end §3 fabric heat loss now matches the Elmhurst worksheet to
0.1 W/K (the worksheet displays per-element U-values to 2 d.p.; our
cascade keeps full precision so the totals differ at the third decimal).
Cert inputs lodged on the fixture:
- roof_insulation_thickness=300 mm on Main and Ext1 → Table 16 U=0.14
- door_count=2 (cascade default 1.85 m²/door → 3.70 m² worksheet area)
- WINDOW_TOTAL_AREA_M2=9.03 with WINDOW_AVG_RAW_U_VALUE=2.8 (pre-2002
double-glazed PVC, 12mm gap; Table 24 row → U_eff=2.518)
Per-part window/door apportionment cancels in the §3 line totals — net
wall sums to the same value whether openings sit on Main or Ext1 — so a
single aggregate area/U pair reproduces (33) exactly.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per the worksheet docstring on this fixture, Extension 1 hangs off the
main from the first storey upward — its lowest dimension is an exposed
timber floor (over outside air), not a ground floor on soil. Set
is_exposed_floor=True so heat_transmission_from_cert routes Ext1 through
the Table 20 lookup (U=1.20 W/m²K at age B unknown insulation) instead
of BS EN ISO 13370.
Combined with the Table 19 fn 1 default that routes Main to the
suspended-timber branch (U≈0.71), §3 LINE_28A floor sum lands at
≈32.4 W/K — matching the worksheet's 0.71×14.85 + 1.20×18.18.
A new floor-sum regression test pins the combined behaviour; the existing
LINE_31/36 parametrised test still passes (the exposed-floor route
contributes its area to LINE_31 the same way the ground-floor route did).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
SapFloorDimension gains an is_exposed_floor flag (default False) signalling
that the floor sits over outside air or unheated space rather than soil —
typical for an extension that hangs off the main from the first storey
upward (Elmhurst 000490 Extension 1 is exactly this shape).
heat_transmission_from_cert now consults the flag on the part's ground
SapFloorDimension and dispatches to u_exposed_floor (Table 20) instead
of the BS EN ISO 13370 / Table 19 cascade. Basement floor still wins
priority (Table 23 § 5.17 overrides everything else for that part).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
RdSAP10 §5.13 Table 20 (page 47) gives U-values for upper floors that
sit over outside air (exposed) or enclosed unheated space (semi-exposed) —
e.g. an extension hanging off the main from the first storey upward.
The spec collapses both into the same lookup: keyed on age band ×
insulation thickness, no geometry needed.
Elmhurst worksheet U985-0001-000490 Extension 1 records U=1.20 W/m²K
for its exposed timber floor (age B, no insulation). Table 20 row
"A to G, insulation unknown or as built" returns 1.20 exactly.
Caller wiring (heat_transmission_from_cert routing on a floor_position
discriminator) lands in the next slice.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
RdSAP10 §5.12 Table 19 footnote (1): when floor_construction is unknown,
age bands A and B default to suspended timber, not solid. Previously
u_floor always used the BS EN ISO 13370 solid-floor formula, which
under-counted ~14% on pre-1929 dwellings.
Elmhurst worksheet U985-0001-000490 Main Dwelling (A=14.85, P=7.42,
w=0.400, age B) records floor U=0.71 W/m²K — the suspended-floor formula
on §5.12 page 46 reproduces this exactly. The solid branch returned 0.66.
Description prefixes "Solid, ..." / "Suspended, ..." take precedence over
the age-band default since they're explicit assessor observations.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Delete HANDOVER_FRESH_REVIEW (22-slice, MAE-5.34 era) and
HANDOVER_SYSTEMATIC_REVIEW (pre-Elmhurst-conformance). Both described
a state the Elmhurst worksheet work has since superseded.
Add HANDOVER_S3_CLOSE.md with:
- Accurate §3 status: §1/§2 fully done; LINE_31/LINE_36 exact for
non-RR fixtures; LINE_33 gap diagnosed as missing floor_construction
codes (not a window-area problem as previously assumed)
- Concrete investigation steps to close LINE_33 for 000474 + 000490
- Table 11 Secondary Heating framed as next slice after §3
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
LINE_31 (total external element area) = Σ_parts (gross_wall + roof +
floor). Window and door areas cancel in the net-wall expansion, so LINE_31
is independent of the window/door split. This lets us assert the exact
Elmhurst worksheet (31) for the two non-RR fixtures (000474, 000490)
without needing window-area input data.
LINE_36 = y × LINE_31 follows for free. Both 000474 and 000490 use age
band B throughout (y = 0.15), giving:
000474: 0.15 × 153.39 = 23.0085
000490: 0.15 × 164.85 = 24.7275
The per-storey-perimeter fix (e6c768c3) was the prerequisite; without it,
upper storeys with a smaller perimeter than the ground floor were
over-counted (e.g. 000474 Main: 7.07 m ground vs 5.27 m first storey).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace the string literal "Main Dwelling" / "Extension 1" comparisons
in `_building_part_aggregates` and the four affected tests with the
typed `BuildingPartIdentifier.MAIN` / `.EXTENSION_1` enum values, so
the transform is consistent with the typed domain introduced in the P6.1
cert→inputs adapter. Fixes a latent mismatch that would silently return
`main=None` if the string ever drifted.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
SAP §3 wall heat-loss area sums each storey individually:
`Σ (heat_loss_perimeter_i × room_height_i)`. Pre-fix used the short-cut
`ground_perimeter × avg_height × storey_count`, which over-counts upper
storeys whenever they have a smaller perimeter than the ground (set-back
top floors, ground-floor additions, etc.). RdSAP §5.10 party-wall area
follows the same per-storey-sum convention.
Surfaced by Elmhurst 000474 Main (ground perim 7.07, first 5.27): our
gross-wall over-counted by ~10 m², the (29a) W/K downstream by ~15 W/K
on this cert. Documented at the time as follow-up #2; this slice closes
it. The §3 partial-conformance test's gap-#2 entry is removed; gap #1
(RR sub-areas) remains.
Fix lives in two parallel code paths:
- dimensions.py: per-storey accumulation inside the existing fd loop
- heat_transmission.py: _part_geometry now emits gross_wall_area_m2 and
party_wall_area_m2 directly, dropping the avg_height + storey_count
intermediate fields (no other consumer)
Tests:
- New: gross_wall_area_sums_per_storey_perimeter_times_height_…
(2-storey main, ground 10 m / first 6 m, same height — expects
Σ=40 m² not ground×avg×count=50)
- New: party_wall_area_sums_per_storey_party_length_… (same shape,
ground party 5 / first party 3 → Σ=20 not 25)
- New: walls_w_per_k_uses_sum_of_per_storey_perimeter_… (heat-
transmission counterpart: 0.6 × 40 = 24 W/K not 30)
829 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
SAP §2 (9) "ns" is the dwelling height — the tallest part — which drives
the (10) additional-infiltration adjustment. Pre-fix code summed
`len(sap_floor_dimensions)` across parts and incremented for every
sap_room_in_roof block, so a 2-storey main + 1-storey side extension
returned ns=3 instead of 2, and a 2-part RR-bearing cert could return
ns=4 or 5. The (10) ach output overstated by 0.1 per spurious storey.
Fix tracks per-part `(floor_count + 1 if RR else 0)` and emits
`max(per_part)`. TFA and volume sums on §1 are unaffected — those are
genuine Σ per RdSAP §3.9.1.
Surfaced by Elmhurst 000474 (2-storey + 2 side extensions): worksheet
says ns=2; we previously had to pass `storey_count=fixture.LINE_9_STOREYS`
explicitly in the §2 Elmhurst conformance test. With the fix, the test
now derives `storey_count` from `dims.storey_count` and the
`LINE_9_STOREYS` field cross-checks the derivation against (9).
Tests:
- New: dwelling_storey_count_is_max_across_parts_not_sum (2-storey main
+ 1-storey ext expects ns=2)
- New: room_in_roof_on_main_adds_one_to_dwelling_storey_count_only_once
(main with RR + ext without RR expects ns=3, not 5)
- Updated: main_plus_extension_sums_areas_perimeters_and_walls assertion
ns==2 → ns==1 (both parts single-storey)
- Updated: all_rir_shapes_apply_section_1_2_45m_convention_uniformly —
storey_delta is now ≤1 not len(parts_with_rr); TFA/volume deltas
remain Σ per the spec
- Updated: §2 Elmhurst test consumes dims.storey_count + asserts
dims.storey_count == fixture.LINE_9_STOREYS as an Arrange precondition
826 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Threads the strict BuildingPartIdentifier type (introduced in a8b443f6)
through the two remaining backend touchpoints:
- EpcBuildingPartModel.from_*: SQLModel column expects a string, so
unbox the enum with .identifier.value before binding to the DB.
- documents_parser end-to-end tests: swap bare-string equality
("main" / "extension_1") for identity checks against the enum
members (BuildingPartIdentifier.MAIN / EXTENSION_1).
Documents_parser test pack passes (105/105). No dedicated SQLModel test
covers EpcBuildingPartModel.from_*; the .value line is exercised
transitively via db_writer.py / local_runner.py in production.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Lands the production code that the just-committed Elmhurst conformance
fixtures (6455d48b) exercise: the SAP10.3 calculator orchestrator
(domain.sap.calculator.Sap10Calculator), the RdSAP-driven cert→inputs
mapper (domain.sap.rdsap.cert_to_inputs), and the EpcPropertyData
strict-type pass that P6.1 starts.
calculator.py is the entry point. Two surfaces depending on the caller's
shape:
- Sap10Calculator().calculate(epc) — full RdSAP mapper + worksheet loop
- calculate_sap_from_inputs(inputs) — pure physics over typed inputs
P6.1 introduces BuildingPartIdentifier as a strictly-typed replacement
for bare-string matching on SapBuildingPart.identifier (motivated by
the pain point at worksheet/dimensions.py:74-82). Two boundary factories
canonicalise raw inputs: from_api_string for the gov-EPC API, and
extension(n) for site-notes / construction id flows.
Also catches up two transitive deps that 6455d48b implicitly required
but I missed:
- ml/rdsap_uvalues.py — party-wall U-value rows that heat_transmission
resolves; the U=0.0 branch the 000516 fixture exercises lands here.
- ml/tests/_fixtures.py — make_minimal_sap10_epc that every Elmhurst
fixture imports. Without this catch-up, checking out 6455d48b in
isolation would ImportError.
Out of scope (will commit separately): ml/transform.py legacy envelope
drift; backend/ FastAPI + documents_parser layer; etl/ scratch.
824 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Lands real-cert ground-truth conformance tests for the SAP10.2 worksheet,
asserting our §1 dimensions, §2 ventilation, and §3 heat-transmission
output line-by-line against six Elmhurst-lodged worksheets (000474,
000477, 000480, 000487, 000490, 000516). Each fixture covers a distinct
shape: with/without room-in-roof, single-part vs main+extensions, age
A and B, party-wall U=0.0 vs U=0.25, 1/2/3 sheltered sides, varying
draught-proofing %, and the (12) suspended-timber quirk.
§1/§2/§3 module updates back the new line-refs (LINE_31 external-element
area, LINE_33 fabric loss, LINE_37 total fabric loss; per-fixture (12)
floor / (15) window / (21) shelter-adjusted ach; SapRoomInRoof storey
contribution via the 2.45 m §3.9.1 convention).
The §3 test currently asserts invariants only ((33) = Σ per-element,
(37) = (33) + (36)) because SapRoomInRoof only carries floor_area —
gable/slope/stud/flat-ceiling sub-areas the worksheet itemizes are not
yet modelled. LINE_3* constants capture the worksheet ground truth for
when that gap closes.
Adds a SAP-domain README with a step-by-step guide for adding new
Elmhurst fixtures from the assessor's PDF pair (Summary + worksheet),
including the field-by-field cert → EpcPropertyData mapping table and
the gotchas surfaced across the six fixtures (storey-height +0.25
convention, party-wall U code mapping, has_suspended_timber_floor flag
truth table, (25) effective-ach formula, Energy Rating vs EPC Costs
wind-speed trap).
366 tests pass (was 360 pre-pairs 5-6).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Wires slice 1-5 primitives into a deployable splitter:
- orchestration/postcode_splitter_orchestrator.py: PostcodeSplitterOrchestrator
loads addresses via UserAddressRepository, groups by postcode via
iter_postcode_grouped_batches, persists each batch under
ara_postcode_splitter_batches/{task_id}/{subtask_id}/, creates a WAITING
child SubTask, and publishes an address2UPRN SQS message per batch.
- applications/postcode_splitter/: Lambda entrypoint. handler.py is decorated
with @subtask_handler() so the parent SubTask lifecycle is decorator-owned;
PostcodeSplitterTriggerBody validates the body. Dockerfile is the
python:3.11 Lambda base with the DDD-shaped source layers and no pandas.
- tests/orchestration/test_postcode_splitter_orchestrator.py: integration
test using moto S3 + moto SQS + in-memory SQLite that exercises the full
wiring against a fixture CSV spanning three postcode groups (one
oversize) and asserts child count, persisted inputs, queue bodies, and
dispatch order.
backend/postcode_splitter/ and .github/workflows/deploy_terraform.yml are
intentionally unchanged: the dockerfile_path flip is deferred until the
companion backend/address2UPRN/ migration is also ready.
The wrapped function now receives the decorator-owned TaskOrchestrator as
a third positional argument so handlers can compose their own use-case
orchestrator that shares the session, instead of opening a second Postgres
connection per invocation.
Both existing callers (backend/ordnanceSurvey/main.py and
backend/bulk_address2uprn_combiner/main.py) have their signatures extended
to accept the new positional argument (typed Optional[TaskOrchestrator] so
the legacy backend.utils.subtasks.subtask_handler — which only passes two
args — keeps working until the migration to the new decorator lands).
@task_handler is intentionally unchanged in this slice; symmetry is
deferred per issue #1103.
Adds a primitive for creating a new WAITING SubTask under an existing
parent Task, routing all SubTask creation through the orchestrator
(replacing the legacy SubTaskInterface path used by the splitter).
Skips _cascade because a new WAITING child against an IN_PROGRESS
parent is a no-op under Task.recalculate_from_subtasks.
Slice 3/6 of the postcode_splitter refactor (Hestia-Homes/Model#1101).
Introduces a thin typed infrastructure layer wrapping boto3 for the AWS
side of the splitter. S3Client/SqsClient are bucket-/queue-bound byte
adapters; CsvS3Client subclasses S3Client to round-trip CSV row dicts
via the existing parse_s3_uri helper in utils/s3.py; Address2UprnQueueClient
subclasses SqsClient to publish the typed {task_id, sub_task_id, s3_uri}
fan-out body the downstream consumer expects. moto[s3,sqs] is pulled into
test.requirements.txt and the new tests/infrastructure/ suite exercises
each client against the moto backend (S3 round-trip, CSV round-trip,
SQS send + body inspection, typed publish + body inspection). pyright
--strict is clean on the new modules.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Slice 1/6 of the postcode_splitter refactor (Hestia-Homes/Model#1100).
Introduces the pure-domain foundation under domain/, with no AWS, Postgres,
or pandas. UserAddress is a frozen dataclass that sanitises its postcode in
__post_init__ via the canonical sanitise_postcode helper, and
iter_postcode_grouped_batches preserves the legacy splitter's batching
invariants (group-by-postcode in insertion order, never split a group,
oversize single-postcode groups dispatched whole, final flush). Updates
UBIQUITOUS_LANGUAGE.md so the User Address term covers both the dataclass
sense (preferred in domain code) and the raw upstream-string sense.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
100-cert probe, seed=7, sap_score window 5..99. MAE 4.29
(vs 8.41 on 2026-05-18 with the older 20..95 window — the
delta blends calculator improvements with sample-window
change, so this is logged as the post-P5 reference, not as
"P5 reduced MAE".)
P5 itself was pure trace exposure; the calculator's SAP
output should be numerically unchanged. The headline finding
from this run is primary-energy over-prediction: PE MAE
44.40 kWh/m², bias +39.66 — now the dominant signal with
SAP residuals halved. Each end-use PE contribution surfaces
on SapResult.intermediate per P5.12, so the next session
can localise the bias without re-instrumenting.
Closes the second half of P5 (HANDOVER_SYSTEMATIC_REVIEW §2.5):
- Adds test_bre_worked_examples.py — one comprehensive test that
locks every published SapResult.intermediate key against its
SAP 10.2 worksheet item number ((4) TFA, (33) fabric heat loss,
(39) HTC, (40) HLP, (73) gains, (93) mean internal temp, (98c)
space heating, (240e/247/250) costs, (252) PV credit, (256)
deflator, (257) ECF, (261-272) per-end-use CO2, (275-287)
primary energy per m²). All formulas derived independently from
the worksheet pages 131-148; passes against the synthetic
100 m² baseline.
- Explicit caveat in module docstring: BRE-published worked
examples don't exist in any of the three SAP-spec PDFs we have
(rdSAP10, SAP10.2, SAP10.3 — all greppped). The test is
spec-formula-derived, not BRE-validated. Structure stays if
BRE numbers surface later; only expected values change.
Also surfaces and documents an RdSAP10 spec drift in
PARITY_FINDINGS.md: Table 32 (page 95 of rdSAP10) gives
Energy Cost Deflator = 0.42, vs the code's 0.36 (SAP10.2 Table 12,
worksheet item (256)). Not changed in P5 — needs ADR-level
resolution on whether the calculator targets SAP10.2 (0.36) or
RdSAP10 (0.42) ratings.
P5 (SapResult.intermediate population + BRE worked-example
fixtures) is now complete on this branch.
Closes the second §11-sketch gap noted in HANDOVER_SYSTEMATIC_REVIEW
("primary energy AND CO2 per end-use"). Lifts the single co2 = total
× factor expression into five named locals (main_heating, secondary,
hot_water, pumps_fans, lighting) and exposes them on `intermediate`.
The five components sum exactly to the top-level co2_kg_per_yr — no
PV deduction in the current implementation.
P5.9 exposed the four primary-energy components as absolute kWh/yr
keys (space_heating_primary_kwh_per_yr, …). HANDOVER_SYSTEMATIC_REVIEW
§11 specifies these as `_pe_kwh_per_m2` because primary energy enters
the rating equation per floor area. Renamed to match the sketch:
- space_heating_pe_kwh_per_m2
- hot_water_pe_kwh_per_m2
- other_pe_kwh_per_m2
- pv_pe_offset_kwh_per_m2
Chain check now verifies max(0, sum − pv_offset) ≈
result.primary_energy_kwh_per_m2 (the top-level per-m² field).
Absolute kWh/yr values remain recoverable via tfa_m2 on `intermediate`.
Final P5 slice. PV credit was the missing term linking the per-end-use
fuel costs (P5.6) to the top-level total_fuel_cost_gbp: total =
max(0, sum(per-end-use) − pv_credit). With this key, every step of
the §13 cost chain — per-fuel cost → PV credit → total → ECF →
rating — is auditable from `intermediate`. P5 trace exposure is
complete.
Promotes _FLOOR_AREA_OFFSET_M2 → FLOOR_AREA_OFFSET_M2 (§13 ECF
denominator, Table 12) and _ECF_LOG_THRESHOLD → ECF_LOG_THRESHOLD
(SAP rating linear/log regime boundary at ECF = 3.5). Together with
the deflator (P5.7) they fully document the §13 rating curve in
trace mode.
Lifts the inlined primary-energy sum into four named components:
space-heating (main + secondary × space_heating PEF), hot water,
other (pumps_fans + lighting × other PEF), and the PV offset at
other PEF (Appendix M). Together with the top-level
primary_energy_kwh_per_yr they make whether the floor-at-zero
clipped visible.
Adds delivered_fuel_kwh_per_yr (sum of all five end-use kWh) and
co2_factor_kg_per_kwh (mirrors the SAP10 input). Together with the
top-level co2_kg_per_yr they make the §15 equation traceable:
co2 = delivered_fuel × factor.
Promotes `_ENERGY_COST_DEFLATOR` to `ENERGY_COST_DEFLATOR` so the
§13 Table 12 constant can be referenced in trace mode alongside the
ECF it scales. ECF mirrors the top-level field; the deflator is the
only fixed worksheet constant the SAP rating depends on.
Per-end-use £/yr costs (main heating, secondary heating, hot water,
pumps_fans, lighting) lifted from the inlined total_cost sum into named
locals and populated on `intermediate`. §12 sweep slices can now diff
each line against the spec (Table 12 unit prices, future Table 12a
fractional blending, Table 12c heat-network DLF) without re-deriving
the cost decomposition.
Behaviour-preserving — `total_fuel_cost_gbp` reconciles bit-for-bit.
136 SAP tests pass.
