Model/scripts/hyde/mapping_decisions.md

# Full-SAP 17.1 → EpcPropertyData mapper — decisions log

Running log of load-bearing mapping decisions made during the grilling session.
Each becomes an ADR (`docs/adr/`) when the mapper lands. Corpus evidence:
`backend/epc_api/json_samples/SAP-Schema-17.1/corpus.jsonl` (1,000 real certs).

**Scope (locked):** map full-SAP 17.1 → a valid `EpcPropertyData` so the existing
RdSAP `Sap10Calculator` runs end-to-end. Accurate-enough geometry/inputs is the
bar; **score reconciliation vs lodged is a later, expert-led step** — do NOT tune
the mapping to hit the lodged rating, and do NOT widen tolerances.

---

## D1 — Heat-loss perimeter & party walls (from measured wall areas)
**Decision:** full-SAP lodges no perimeter; derive it from measured per-wall areas
classified by `wall_type`:
- `{1,2,3}` exposed → `heat_loss_perimeter_m` = Σarea ÷ storey_height
- `4` party → `party_wall_length_m` = Σarea ÷ storey_height
- `5` internal partition → discarded
- any other code → `raise UnmappedApiCode("wall_type", code)` (**fail loud**)

`wall_type` is a ~99%-clean classifier (corpus). Accept the ~1% tail where a party
wall is mis-typed as 2 (`Common wall`/`Stair Wall`); revisit with name hints only
if the later score comparison flags it. Geometric `4·√(footprint)` is fallback only
(no usable walls). Detail + worked examples: `perimeter_decision.md`.

## D2 — Window / door / roof opening collapse
**Decision:** join `sap_openings` → `sap_opening_types` by name (100% join in corpus),
then route by opening-type `.type`:
- `4` vertical window → `sap_windows` (`SapWindow`: width, height, `u_value`→
  `window_transmission_details`, orientation, glazing_type, frame_factor)
- `5` roof window / rooflight → `sap_roof_windows` (`SapRoofWindow`: area=w×h,
  u_value, orientation, pitch, frame_factor, g⊥←solar_transmittance)
- `1,2,3` doors → `door_count` = #door-openings; `insulated_door_u_value` =
  area-weighted door U (only 46/1000 certs lodge >1 distinct door U); doors treated
  insulated (new-build)
- any other opening-type code → `raise UnmappedApiCode("sap_opening_type", code)`

Lower-risk than the handoff implied: the engine has first-class fields for all three
opening kinds, and full-SAP supplies richer per-opening geometry than RdSAP.

## D3 — Living-area fraction (back-solve room count)
**Decision:** full-SAP measures `living_area` but the engine only reads it via
`habitable_rooms_count` → Table 27. **Mapper-only:** set `habitable_rooms_count` to
the room count whose Table 27 fraction is closest to the measured `living_area/TFA`,
so the existing engine path reproduces the measured fraction.

**Known caveat (to go back to):** `habitable_rooms_count` is *also* read for the
Table 5 extract-fan minimum (`max(measured_fans, room_count_default)`,
`cert_to_inputs.py:4534`). A back-solved count can raise that floor. Bounded
(fans 1–4) and **moot for the 445/1000 mechanically-ventilated certs**; bites only
in the natural-vent subset where measured fans < default. **v1: accept** (keeps the
decision mapper-only). Principled future fix: suppress the Table-5 "unknown" default
for full-SAP, since its ventilation is measured, not unknown.

---

## D4 — Fabric U-values: carry descriptions through; age_band is fallback only
**Pivotal finding:** the engine's `u_wall`/`u_floor`/`u_roof` first parse the element
`description` for `"Average thermal transmittance X W/m²K"` and return that measured U
directly, bypassing the age-band cascade (`rdsap_uvalues.py:546`; wired at
`heat_transmission.py:589,779`). The parser docstring: *"On full-SAP certs the
assessor enters the BS EN ISO 6946 result directly here in lieu of using the cascade."*

Corpus: walls parse to measured U **984/1000**; floors 692 (rest = "(other premises
below)" internal, no loss); roofs 715 (rest = "(other premises above)" internal).

**Decision:** map full-SAP top-level `walls`/`floors`/`roofs` energy-element lists via
the existing `_map_energy_elements` (preserves `description`). The measured fabric U then
flows into the engine — it is NOT discarded. This reverses the initial "RdSAP engine
re-derives from fabricated band-M and over-rates" concern for ~98% of walls.

`construction_age_band` is therefore a **low-stakes fallback**, not the primary fabric
driver. Fabricate band **M** from `construction_year`/`assessment_date` (new-build era);
used only for the ~2% of walls with no parseable U and for secondary age-band logic
(cylinder-insulation default, suspended-timber-floor sealed/unsealed branch, Table 5 fan
default). **Fail loud** (`UnmappedApiCode`) only if neither a year nor a usable band can
be established. *(Note: this also shrinks the D3 living-area physics-gap framing — the
score is far more faithful than the early no-engine estimate suggested.)*

## Corpus validation (1,000 certs, full chain through Sap10Calculator)

Run after slices 1–7 + D5/D6 landed (`from_api_response` → `Sap10Calculator`):

- **727/1000 (72.7%) run end-to-end; 0 calculator failures** — where a cert maps,
  it calculates cleanly (robust mapping, no engine crashes).
- **Initial 273 map-failures** were all one variant: code-based heating
  (`main_heating_category` 6/10 — community heating, heat pumps) lodging
  `main_heating_code` instead of explicit `heat_emitter_type`/`main_fuel_type`.
  **RESOLVED (slice D6b):** mapped `main_heating_code → sap_main_heating_code`
  (a known variant the calculator already consumes), defaulting the absent
  explicit fields. **Corpus mappability now 1000/1000 (100%); calc 1000/1000.**
- **Score divergence (calc − lodged, full mapping, n=1000):** median **−8**,
  mean **−11.4**; within ±10 = 55%. The engine **systematically under-rates**
  full-SAP certs. This is **deferred to a separate SAP-calc verification task**
  (mapping fidelity is the deliverable here; the score is not). Known drivers,
  in rough order: (a) measured **MEV** ventilation priced as added extract loss
  (−4 isolated on the sample); (b) **code-based heating** (276 certs) currently
  defaults the absent fuel/emitter to 0 — these systems map (no crash) but are
  not yet fully modelled, so their scores are unreliable *by mapping*, not calc;
  (c) fabricated RdSAP proxies (age band, back-solved room count). The mapper is
  **never tuned to the lodged value**; pins are the observed score.

## Open / not yet grilled
- D5 `ventilation_type` (1/5/6/8…) → `MechanicalVentilationKind` (NATURAL/MEV/MVHR/PIV); `air_permeability` → AP4
- D6 heating: full-SAP `sap_heating.main_heating_details` shape vs RdSAP field names
- D7 wall/roof/floor `construction` code mapping (full-SAP `wall_type` → domain `wall_construction` for U-derivation)
- D8 `_clear_basement_flag_when_system_built` applicability
- D9 dwelling_type/built_form/property_type/tenure code mapping; 11/1000 null `built_form`
- D10 schema dataclass optionality (corpus presence: `sap_flat_details` 44%, `design_water_use` 94%, etc.)