Cert 9501 lodges measured PV: 2.36 kWp South-West, 45° pitch, "None
Or Little" overshading. The worksheet's §10a credit (-250.02 GBP =
PV used in dwelling £-129.49 + PV exported £-120.53) depends on the
Appendix M / Appendix U3.3 cascade reading these from
`SapEnergySource.photovoltaic_arrays`. The prior extractor only
captured the `photovoltaic_panel: "Panel details"` label — the
actual kW / orientation / elevation / overshading were silently
dropped, so the cascade computed total cost ~£250 too high → ECF
2.92 vs worksheet 2.26 → SAP 59.26 vs 68.53 (Δ -9.27).
Changes:
- Extend `surveys.elmhurst_site_notes.Renewables` with 4 new
optional fields: pv_peak_power_kw / pv_orientation /
pv_elevation_deg / pv_overshading.
- Add `ElmhurstSiteNotesExtractor._extract_pv_array_detail` —
anchors on "Photovoltaic panel details" then reads the 4
consecutive value lines (kWp, orientation, elevation, overshading).
- Add `_elmhurst_pv_arrays` mapper helper to build the
`[PhotovoltaicArray(...)]` list when all 4 values are present;
return None for the "PV absent" path the cascade already handles.
- Add `_ELMHURST_PV_OVERSHADING_TO_RDSAP` map: "None Or Little" → 1
(ZPV=1.0 per cert_to_inputs._PV_OVERSHADING_FACTOR), "Modest" →
2, "Significant" → 3, "Heavy" → 4. RdSAP omits SAP10.2 Table M1's
5th "Severe" bucket.
- Wire `photovoltaic_arrays=_elmhurst_pv_arrays(survey.renewables)`
into `from_elmhurst_site_notes`'s `SapEnergySource(...)` call.
Effect on cert 9501 Summary path:
- sap_continuous 59.2585 → 68.7577 (target 68.5252; Δ +0.23)
- total_fuel_cost £1099 → £843 (worksheet £849; -£6 over-credit)
- ECF 2.92 → 2.24 (worksheet 2.26; -0.02 over-credit)
The remaining +0.23 SAP / +£6 cost drift is a precision gap in the
Appendix M cost-offset cascade for measured PV (not a missing-data
gap); next slice closes it to 1e-4.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Cert 9501 (Summary_000784.pdf) is a flat. The Elmhurst Summary's
§1.0 "Property type" section lodges the built-form descriptor
("M Mid-Terrace", "D Detached", ...) only for houses — flats have no
attachment line, and the §2.0 "Number of Storeys" header follows
immediately after the "F Flat" property-type value.
The extractor's prior `_extract_attachment` regex captured the line
right after the property-type value unconditionally, so cert 9501
ended up with `attachment="2.0 Number of Storeys:"` — section-header
noise that the mapper surfaced on `EpcPropertyData.built_form`.
Downstream, this broke the cascade's `_dwelling_exposure` routing
(no prefix match → defaulted to fully-exposed houses) and so the
cert 9501 Summary path was Δ -5.25 SAP vs worksheet 68.5252.
Detect section-header noise via the leading `<digit>.<digit> `
pattern and the "Number of Storeys" substring; return "" in that
case so flats produce empty `built_form`. Houses still pick up their
real attachment (cohort 0330's "M Mid-Terrace" remains correct).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Three extensions closing the last 0.05 SAP residual on 000487 — and
with it, all 6 Elmhurst Summary PDFs match their U985 worksheets to
1e-4 unrounded SAP.
1. Alternative-wall extraction. `WallDetails` gains an
`alternative_walls: List[AlternativeWall]` field; the extractor
parses §7's "Alternative Wall N Area / Type / Insulation /
Thickness / Thickness Unknown / U-value Known" prefixed labels.
Even when an extension lodges "As Main Wall: Yes" we still pull
alt walls from the extension's own subsection (they don't
inherit) — the main wall fields are merged with the extension's
alt-wall list.
2. Alt-wall mapper plumbing. `_map_elmhurst_alternative_wall` builds
a `SapAlternativeWall` per lodged Elmhurst entry; the building-
part mapper attaches up to two via `sap_alternative_wall_1/_2`
per `SapBuildingPart`. When the surveyor flags `Thickness
Unknown: Yes` (cohort's only example — 000487 Ext1's
"TimberWallOneLayer" entry) we route the cascade with
thickness=None so `u_wall` falls through to the age-band-and-
construction default — Timber Frame age B uninsulated → U=1.9,
matching the full-cert-text U=1.90 the handbuilt fixture lodges
for the same 9-mm thin timber wall.
3. "TI" wall-construction code mapping. The §7 "Alternative Wall 1
Type: TI Timber Frame" uses leading code "TI" rather than the
"TF" code seen on the primary wall types — both alias to SAP10
wall_construction=5 (Timber Frame).
Final cohort state — all 6 closed at 1e-4:
000474 0.0000 ✓ Slice 47
000477 0.0000 ✓ Slice 52
000480 0.0000 ✓ Slice 50
000487 0.0000 ✓ THIS SLICE
000490 0.0000 ✓ Slice 49
000516 0.0000 ✓ Slice 51
758 tests pass; pyright net-zero (35 baseline).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Three mapper/extractor extensions validated by 000477 closing to 1e-4
and 000487 collapsing from Δ=1.18 SAP to Δ=0.05 (alt-wall residual).
1. RR detailed-surface area rounded half-up to 2 d.p. via Decimal.
The Elmhurst worksheet rounds 4.39 × 1.50 = 6.585 to 6.59; Python's
builtin `round` (banker's) returns 6.58 and a naïve floor+0.5 trips
on FP precision (the product is 6.5849999… in float64). Compute
the product in `Decimal` first (both operands are exact 2-d.p.
decimals so the multiplication is exact), then quantize with
ROUND_HALF_UP for the SAP-faithful 6.59. Closes the 0.01 m² stud-
wall-area drift that left 000477 at Δ=0.0004 SAP after RR support.
2. Suspended-timber-floor heuristic. The §2(12) wooden-floor ACH (0.2
unsealed / 0.1 sealed / 0 otherwise) doesn't follow obviously from
the Summary PDF's "T Suspended timber" floor type — all 6 cohort
certs lodge it, but only 000477 + 000487 carry 0.2 ACH in their
U985 worksheets. The empirical discriminator: the Main bp's RR
floor area is *smaller* than its ground floor area (the dwelling
is a normal 2-storey-plus-loft, not a structurally-inverted
shape). 000480 trips the inverse (RR 19.83 > ground 15.28 →
False) and 000516 trips on the non-ground floor location.
3. Electric vs mixer shower from outlet_type. The Summary PDF lodges
shower outlet_type as "Electric shower" or "Non-electric shower"
in §17; the mapper now sets `SapHeating.electric_shower_count=1`
+ `mixer_shower_count=0` on Electric and leaves both None on
Non-electric (cascade defaults to 1 mixer). Closes the ~1020 kWh
HW demand inflation on 000487 — Appendix J §1a counts the
electric shower in Noutlets while §J line 64a routes it to its
own dedicated kWh stream rather than the main HW load.
Cohort state after this slice:
000474 0.0000 ✓ Slice 47
000477 0.0000 ✓ THIS SLICE
000480 0.0000 ✓ Slice 50
000487 +0.0519 extension's alternative wall 1 (1.43 m² Timber
Frame, U=1.90 lodged but only via full-cert text
— not exposed in Summary PDF)
000490 0.0000 ✓ Slice 49
000516 0.0000 ✓ Slice 51
5/6 closed at 1e-4. 757 tests pass; pyright net-zero (35 baseline).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Four mapper extensions, validated by 000480 closing to 1e-4 and large
gap reductions across 000477/000487/000516.
1. Room-in-Roof support. `ElmhurstSiteNotes` gains `RoomInRoof` +
`RoomInRoofSurface` dataclasses; extractor parses §8.1 (Flat
Ceiling / Stud Wall / Slope / Gable Wall / Common Wall) with
Length × Height + insulation + gable-type + measured-U cells.
Mapper produces a `SapRoomInRoof` with `detailed_surfaces`
attached to the Main bp: Stud Walls / Slopes / Flat Ceilings
route through Table 17 insulation thickness; Gable Walls split
between `gable_wall` (Party → Table 4 U=0.25) and
`gable_wall_external` (Sheltered → assessor-lodged U-value
override, e.g. 000487 Gable Wall 2 at U=0.86). Empty surfaces
(0×0 — the cohort lodges a full 5-pair table) and Common Walls
(handled by cascade's Simplified Type 2 geometry) are dropped.
`total_floor_area_m2` now includes the RR floor area.
2. Party-wall construction mapping. 000516 lodges "S Solid masonry /
timber / system build" which routes to SAP10 wall_construction=3
(Solid Brick → U=0.0 via Table 4). The previous mapper used the
same wall-type table as `wall_construction`, which lacked the
"S" code and fell through to None (cascade default 0.25). Split
into a dedicated `_elmhurst_party_wall_construction_int` keyed
on the party-wall category codes.
3. Roof "None" insulation. When the §8.0 Roofs subsection lodges
"Insulation N None" without a separate "Insulation Thickness"
line, treat thickness as 0 mm so the cascade picks Table 16
row 0 (U=2.30) rather than the age-band default. Closes the
29 W/K roof-loss gap on 000516.
4. `number_baths` lodgement. `SapHeating.number_baths` now reads
`survey.baths_and_showers.number_of_baths`. The cascade defaults
`None → has-bath` for the modal UK case, but explicit `0` lodged
on 000477/000480 (bathless dwellings, rare) drops the bath HW
demand line per Table 1b. Closes 000480's last ~0.3 SAP gap.
Cohort state after this slice (target 1e-4):
000474 0.0000 ✓ Slice 47
000477 +1.1161 Elmhurst floor_ach quirk (true vs false despite
"T Suspended timber" lodged on all certs)
000480 0.0000 ✓ THIS SLICE
000487 +1.1844 extractor still drops most §11 windows on this
layout variant
000490 0.0000 ✓ Slice 49
000516 +0.1774 roof-window separation by U-value heuristic
3/6 certs now closed at 1e-4. Pyright net-zero (35 baseline). Tests
756 pass (added `test_summary_000480_full_chain_sap_matches_worksheet_
pdf_exactly`).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two mapper extensions, both validated by 000490 closing to 1e-4:
1. Secondary heating extraction. Elmhurst Summary PDFs lodge the
secondary heating SAP code in the §14.1 Main Heating2 sub-section
(between "14.1 Main Heating2" and "14.1 Community Heating") — not
in the §14.0 Main Heating1 block where the main system lives.
`ElmhurstMainHeating` gains a `secondary_heating_sap_code` field;
the extractor reads it from the right section; the mapper threads
it through to `SapHeating.secondary_heating_type`. The cascade
then applies Table 11's 10% secondary fraction.
2. Sheltered-sides derivation per RdSAP §S5. The Summary PDF doesn't
lodge per-dwelling sheltered-sides; the value is derived from
built-form (Detached=0, Semi-Detached=1, End-Terrace=1, Mid-
Terrace=2, Enclosed Mid-Terrace=3, Enclosed End-Terrace=2).
`_map_elmhurst_ventilation` now takes built_form and populates
`SapVentilation.sheltered_sides`. The table is cross-checked
against U985-0001-NNNNNN.pdf line (19) across the 6 worksheet
fixtures.
Cohort SAP deltas after this slice (target 1e-4):
000474 0.0000 ✓ Slice 47
000477 +2.6555 diagnosis pending (lighting bulb count diff)
000480 +4.1955 diagnosis pending
000487 +4.4553 extractor still drops most windows
000490 0.0000 ✓ THIS SLICE
000516 +1.5162 roof-window separation
Pyright net-zero on touched files (35 errors, same baseline). 755
tests pass (up from 754 — new `test_summary_000490_full_chain_sap_
matches_worksheet_pdf_exactly`).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The §11 Windows table in the Summary PDF doesn't lay out identically
across the cohort. Three new quirks added to the layout-style parser
so the remaining 5 certs can be debugged with windows actually
extracted:
1. `Wood 0.70` combined frame_type+frame_factor line — previously the
parser expected them on separate lines (data+1 / data+2) and
rejected the window when the joined form appeared.
2. Trailing glazing-type on the data line — `1.22 1.76 2.15 Double
pre 2002` is the joined-cell variant in 000516; the W/H/Area
anchor now captures the trailing phrase as an optional 4th group
and feeds it through as `inline_glazing_type`, bypassing the
separate-line glazing-prefix scan.
3. Cross-window gap with no glazing marker — `_partition_after_manuf`
now falls back to "second orientation token in gap" when no
glazing-type-prefix word appears. Covers the 000516 layout where
each window has prefix+suffix orient tokens (no inline orient)
and the glazing-type is joined-to-data.
The 5 remaining Summary PDFs are copied into
`backend/documents_parser/tests/fixtures/` ready for per-cert mapper
work. Mirror pin tests deferred — each cert still has its own diff
to close (handover in NEXT_AGENT_PROMPT.md documents the per-cert
state, e.g. 000477 needs secondary-heating extraction, 000516 needs
roof-window separation).
Current cohort SAP deltas vs the U985 worksheet PDFs (target 1e-4):
000474 0.0000 ✓
000477 +6.3655 secondary heating + lighting
000480 +8.2695 diagnosis pending
000487 +8.1433 extractor still drops windows
000490 +5.6551 diagnosis pending
000516 +5.9812 roof-window separation
Wider regression stays green (754 pass). Pyright net-zero on
touched files.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two diffs closed against the hand-built `_elmhurst_worksheet_000474`
target (SAP 62.2584):
1. `pumps_fans_kwh_per_yr` (130 → 160). The cascade keys §4f pumps+fans
electricity on `MainHeatingDetail.main_heating_category` (gas-fired
boilers = cat 2 → 160 kWh/yr). `from_elmhurst_site_notes` wasn't
populating the field, so it fell through to the default 130. Added
`_elmhurst_main_heating_category` deriving cat 2 for the gas/LPG-
PCDB-boiler branch; other categories deferred until a fixture
exercises them (consistent with the cascade lookup).
2. Window [4] orientation `East-South` → `East` and window [5]
orientation `''` → `South-East`. The layout-style parser's
`before_start = prev_manuf + 7` / `after_end = next_data` rule was
over-grabbing prefix tokens of W_{k+1} as suffix tokens of W_k
('South' from W_5's prefix bled into W_4's suffix). Replaced with
a symmetric partition on the first glazing-type-start token
(`Single`/`Double`/`Triple`/`Secondary`) within the cross-window
gap, used as the upper bound of W_k's suffix and the lower bound
of W_{k+1}'s prefix. Same boundary on both sides — prefix tokens
of the next window can no longer be attributed as suffix of the
current one.
After both fixes, Summary_000474 → ElmhurstSiteNotes → EpcPropertyData
→ cascade → SAP matches the worksheet PDF's unrounded line 257 value
to 1e-4 tolerance. All 754 datatypes/epc/ + backend/documents_parser/
tests green; pyright net-zero on touched files.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The legacy `_extract_windows` regex anchors on "Permanent Shutters\n" which is broken across lines by the pdftotext-layout preprocessor. New fallback `_extract_windows_from_layout` anchors on the two stable per-window markers — a "W H Area" data line and the "Manufacturer <U_value>" line a few lines further down — and tolerates the variable-order optional fields (glazing_gap, inline building_part, inline orientation) between them. Prefix/suffix tokens around the data block are re-joined into glazing_type / building_part / orientation strings.
Cert U985-0001-000474's 7 windows across Main + 2 extensions now flow through the mapper to EpcPropertyData.sap_windows (was 0). Textract-style extraction (existing fixture) is unchanged — the legacy path runs first and only falls through when its regex misses.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
ElmhurstSiteNotes had no representation for extensions: singular dimensions / walls / roof / floor fields could only describe the main bp. Summary PDFs lodge "1st Extension" / "2nd Extension" subsections in §4, §7, §8, §9 with optional "As Main: Yes" inheritance. This slice:
- Adds `ExtensionPart` dataclass and `ElmhurstSiteNotes.extensions: List[ExtensionPart]`.
- Adds `_split_section_by_bp` helper + per-bp parsing of dimensions / walls / roof / floor in the extractor; "As Main" inherits from the main bp.
- Refactors `_map_elmhurst_building_part` into a parameterised builder; adds `_map_elmhurst_building_parts` that yields Main + one SapBuildingPart per extension (capped at 4 per RdSAP10 §1.2).
- Scaffold test `test_summary_000474_mapper_produces_three_building_parts` flips from strict-xfail to passing.
Single-bp behaviour is unchanged (empty extensions list defaults). 752 existing tests stay green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
