fix(mapper): read the dropped rafter_insulation_thickness API field

Roofs lodged insulated at rafters carry their thickness in a DEDICATED
gov-EPC API field, `rafter_insulation_thickness` (e.g. "225mm"), while
`roof_insulation_thickness` stays None (rafters aren't loft joists). That
field was undeclared on the 21.0.x schemas, so `from_dict` silently
dropped it — the rafter certs only *looked* redacted (roof EER 2-4 =
insulated, yet no thickness), and the cascade fell to the Table 18 col (2)
unknown default (2.30), badly under-rating them.

- declare `rafter_insulation_thickness` on RdSapSchema21_0_0/21_0_1 +
  EpcPropertyData.SapBuildingPart (mirrors the existing
  sloping_ceiling_/flat_roof_insulation_thickness dropped-field handling).
- thread it through `from_rdsap_schema_21_0_0/21_0_1` (older schemas get
  None via getattr).
- `heat_transmission` prefers `rafter_insulation_thickness` over
  `roof_insulation_thickness` when the part is at-rafters, so the measured
  RdSAP 10 §5.11.2 Table 16 column (2) row applies (225 mm → 0.25).

Completes the rafters roof fix: with the real thickness read, the rafter
certs are recovered rather than over-stated — cert 3100-8675-0922-8628
(band E, rafters 225mm) +8.93 → +0.43 SAP. Corpus within-0.5 67.0%
(MAE 1.025) and /tmp 71.2% (MAE 0.889) — both NET ABOVE the pre-rafters
baseline (66.9% / 70.6%). Worksheet harness 47/47; regression = only the
3 pre-existing fails; pyright net-zero.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

datatypes/epc/domain/epc_property_data.py

@@ -552,6 +552,13 @@ class SapBuildingPart:
     roof_insulation_thickness: Optional[Union[str, int]] = (
         None  # TODO: make enum/mapping?
     )
+    # Lodged insulation thickness (e.g. "225mm", or "AB" As Built) for a roof
+    # insulated AT RAFTERS (roof_insulation_location == 1). The gov API lodges
+    # rafter insulation in this dedicated field — `roof_insulation_thickness`
+    # stays None for rafter roofs. `heat_transmission` prefers this field over
+    # `roof_insulation_thickness` when the part is at-rafters, so the measured
+    # Table 16 column (2) row applies instead of the unknown-thickness default.
+    rafter_insulation_thickness: Optional[Union[str, int]] = None
     sap_room_in_roof: Optional[SapRoomInRoof] = None
     # Per RdSAP 10 §5.18 (PDF p.48), a curtain wall (wall_construction
     # =WALL_CURTAIN=9) takes its U-value from the per-BP installation

datatypes/epc/domain/mapper.py

@@ -1023,6 +1023,11 @@ class EpcPropertyDataMapper:
                         bp.roof_insulation_thickness,
                         bp.construction_age_band,
                     ),
+                    # Rafter insulation thickness lives in its own gov-API field
+                    # (only on the 21.0.x schemas; getattr is None elsewhere).
+                    rafter_insulation_thickness=getattr(
+                        bp, "rafter_insulation_thickness", None
+                    ),
                     sap_room_in_roof=(
                         SapRoomInRoof(
                             floor_area=_measurement_value(bp.sap_room_in_roof.floor_area),
@@ -1220,6 +1225,11 @@ class EpcPropertyDataMapper:
                         bp.roof_insulation_thickness,
                         bp.construction_age_band,
                     ),
+                    # Rafter insulation thickness lives in its own gov-API field
+                    # (only on the 21.0.x schemas; getattr is None elsewhere).
+                    rafter_insulation_thickness=getattr(
+                        bp, "rafter_insulation_thickness", None
+                    ),
                     sap_room_in_roof=(
                         SapRoomInRoof(
                             # ADR-0028: floor_area is usually a Measurement but
@@ -1470,6 +1480,11 @@ class EpcPropertyDataMapper:
                         bp.roof_insulation_thickness,
                         bp.construction_age_band,
                     ),
+                    # Rafter insulation thickness lives in its own gov-API field
+                    # (only on the 21.0.x schemas; getattr is None elsewhere).
+                    rafter_insulation_thickness=getattr(
+                        bp, "rafter_insulation_thickness", None
+                    ),
                     sap_room_in_roof=(
                         SapRoomInRoof(
                             floor_area=_measurement_value(
@@ -1705,6 +1720,10 @@ class EpcPropertyDataMapper:
                         bp.construction_age_band,
                         bp.sloping_ceiling_insulation_thickness,
                     ),
+                    # RdSAP 10 §5.11.2 — rafter insulation thickness lives in its
+                    # own gov-API field (roof_insulation_thickness stays None for
+                    # rafter roofs); heat_transmission prefers it when at-rafters.
+                    rafter_insulation_thickness=bp.rafter_insulation_thickness,
                     # RdSAP 10 §5.1 — the assessor's lodged roof/wall/floor U
                     # overrides the §5.6/§5.7/§5.11 construction-default cascade
                     # (gov open data can redact the backing insulation).
@@ -2014,6 +2033,10 @@ class EpcPropertyDataMapper:
                         bp.construction_age_band,
                         bp.sloping_ceiling_insulation_thickness,
                     ),
+                    # RdSAP 10 §5.11.2 — rafter insulation thickness lives in its
+                    # own gov-API field (roof_insulation_thickness stays None for
+                    # rafter roofs); heat_transmission prefers it when at-rafters.
+                    rafter_insulation_thickness=bp.rafter_insulation_thickness,
                     # RdSAP 10 §5.1 — the assessor's lodged roof/wall/floor U
                     # overrides the §5.6/§5.7/§5.11 construction-default cascade
                     # (gov open data can redact the backing insulation).

datatypes/epc/domain/tests/test_from_rdsap_schema.py

@@ -406,6 +406,35 @@ class TestFromRdSapSchema21_0_1:
         # worksheet uses per-bp sums and the mapper now mirrors that.
         assert result.total_floor_area_m2 == 45.82
 
+    def test_rafter_insulation_thickness_threaded(
+        self, schema: RdSapSchema21_0_1
+    ) -> None:
+        # Arrange — the gov API lodges rafter insulation in the dedicated
+        # `rafter_insulation_thickness` field (RdSAP 10 §5.11.2); it was
+        # previously undeclared, so `from_dict` dropped it and the cascade
+        # fell to the Table 18 col (2) unknown default. The mapper must
+        # thread it through to the domain SapBuildingPart so
+        # heat_transmission can reach the measured Table 16 col (2) row.
+        import dataclasses
+
+        bps = schema.sap_building_parts
+        patched = dataclasses.replace(
+            schema,
+            sap_building_parts=[
+                dataclasses.replace(
+                    bps[0], roof_insulation_location=1,
+                    rafter_insulation_thickness="225mm",
+                ),
+                *bps[1:],
+            ],
+        )
+
+        # Act
+        result = EpcPropertyDataMapper.from_rdsap_schema_21_0_1(patched)
+
+        # Assert
+        assert result.sap_building_parts[0].rafter_insulation_thickness == "225mm"
+
     # --- property flags ---
 
     def test_solar_water_heating(self, result: EpcPropertyData) -> None:

datatypes/epc/schema/rdsap_schema_21_0_0.py

@@ -265,6 +265,14 @@ class SapBuildingPart:
     # the slope as uninsulated (Table 16 / Table 18 fallback). Consumed by
     # `_api_resolve_sloping_ceiling_thickness` → Table 17 column (1a).
     sloping_ceiling_insulation_thickness: Optional[Union[str, int]] = None
+    # Lodged insulation thickness (e.g. "225mm", or "AB" As Built) for a roof
+    # insulated AT RAFTERS (roof_insulation_location == 1). The gov API lodges
+    # rafter insulation in this dedicated field — NOT `roof_insulation_thickness`
+    # (which stays None for rafter roofs, since rafters aren't loft joists).
+    # Previously undeclared → dropped by `from_dict`, so the cascade fell to the
+    # Table 18 col (2) unknown default (2.30) instead of the measured Table 16
+    # col (2) row. Consumed by `heat_transmission` when at-rafters.
+    rafter_insulation_thickness: Optional[Union[str, int]] = None
     # Lodged roof U-value (W/m²K) — the assessor's RdSAP-assessed roof U,
     # authoritative when the open data redacts the backing insulation
     # thickness. Consumed by `heat_transmission` as a §5.1 documentary-evidence

datatypes/epc/schema/rdsap_schema_21_0_1.py

@@ -303,6 +303,14 @@ class SapBuildingPart:
     # the slope as uninsulated (Table 16 / Table 18 fallback). Consumed by
     # `_api_resolve_sloping_ceiling_thickness` → Table 17 column (1a).
     sloping_ceiling_insulation_thickness: Optional[Union[str, int]] = None
+    # Lodged insulation thickness (e.g. "225mm", or "AB" As Built) for a roof
+    # insulated AT RAFTERS (roof_insulation_location == 1). The gov API lodges
+    # rafter insulation in this dedicated field — NOT `roof_insulation_thickness`
+    # (which stays None for rafter roofs, since rafters aren't loft joists).
+    # Previously undeclared → dropped by `from_dict`, so the cascade fell to the
+    # Table 18 col (2) unknown default (2.30) instead of the measured Table 16
+    # col (2) row. Consumed by `heat_transmission` when at-rafters.
+    rafter_insulation_thickness: Optional[Union[str, int]] = None
     # Lodged roof U-value (W/m²K) — the assessor's RdSAP-assessed roof U. The
     # gov open data can redact the backing insulation thickness, so this is the
     # authoritative per-element value; consumed by `heat_transmission` as a

domain/sap10_calculator/worksheet/heat_transmission.py

@@ -791,7 +791,22 @@ def heat_transmission_from_cert(
             or _described_as_retrofit_insulated(wall_description)
         )
         party_construction = _int_or_none(part.party_wall_construction)
+        # RdSAP 10 §5.11.2 — a roof insulated AT RAFTERS lodges its thickness in
+        # the dedicated gov-API `rafter_insulation_thickness` field, NOT
+        # `roof_insulation_thickness` (which stays None for rafter roofs, since
+        # rafters aren't loft joists). Prefer the rafter field when the part is
+        # at-rafters so the measured Table 16 column (2) row applies instead of
+        # the unknown-thickness default. The Summary path lodges rafter
+        # thickness in `roof_insulation_thickness` (no separate field), so the
+        # fallback covers it.
+        insulation_at_rafters = _roof_insulation_at_rafters(
+            getattr(part, "roof_insulation_location", None)
+        )
         raw_roof_thickness = getattr(part, "roof_insulation_thickness", None)
+        if insulation_at_rafters:
+            raw_rafter_thickness = getattr(part, "rafter_insulation_thickness", None)
+            if raw_rafter_thickness is not None:
+                raw_roof_thickness = raw_rafter_thickness
         roof_thickness = _parse_thickness_mm(raw_roof_thickness)
         floor_ins_thickness = _parse_thickness_mm(getattr(part, "floor_insulation_thickness", None))
 
@@ -895,15 +910,12 @@ def heat_transmission_from_cert(
         # string triggers the col (3) age-band default in `u_roof`.
         is_pitched_sloping_ceiling = "sloping ceiling" in roof_type_lower
         # RdSAP 10 §5.11.2 Table 16 column (2) / Table 18 rafters column —
-        # a roof lodged insulated AT RAFTERS (per-part
-        # `roof_insulation_location` == 1 / "R Rafters") sits on the
-        # shallower sloping side, so the same insulation depth yields a
-        # higher U than the loft-joist column (1). Driven per-part because
-        # the deduplicated `epc.roofs[]` description list cannot attribute
-        # a location to each building part.
-        insulation_at_rafters = _roof_insulation_at_rafters(
-            getattr(part, "roof_insulation_location", None)
-        )
+        # a roof lodged insulated AT RAFTERS sits on the shallower sloping
+        # side, so the same insulation depth yields a higher U than the
+        # loft-joist column (1). `insulation_at_rafters` (computed above) is
+        # driven per-part from `roof_insulation_location` because the
+        # deduplicated `epc.roofs[]` description list cannot attribute a
+        # location to each building part.
         ur = u_roof(country=country, age_band=age_band, insulation_thickness_mm=roof_thickness, description=effective_roof_description, is_flat_roof=is_flat_roof, is_sloping_ceiling=is_sloping_ceiling, is_pitched_sloping_ceiling=is_pitched_sloping_ceiling, insulation_at_rafters=insulation_at_rafters)
         # RdSAP 10 §5.1 — a lodged/known roof U-value (the assessor's RdSAP
         # output, surfaced by the gov-EPC API as `roof_u_value`) is used

tests/domain/sap10_calculator/worksheet/test_heat_transmission.py

@@ -189,6 +189,45 @@ def test_roof_insulation_location_rafters_drives_table16_column_2_api_int_path()
     assert abs(result.roof_w_per_k - 29.0) <= 1e-4
 
 
+def test_rafter_insulation_thickness_field_drives_table16_column_2() -> None:
+    # Arrange — the gov-EPC API lodges rafter insulation in a DEDICATED
+    # `rafter_insulation_thickness` field (e.g. "225mm"), leaving
+    # `roof_insulation_thickness` None for rafter roofs (rafters aren't loft
+    # joists). heat_transmission must prefer the rafter field when the part
+    # is at-rafters (roof_insulation_location == 1) so the measured RdSAP 10
+    # §5.11.2 Table 16 column (2) row applies — 225 mm → U=0.25 — instead of
+    # the Table 18 col (2) unknown default (2.30). Cert 3100-8675-0922-8628
+    # (band E, rafters 225mm) went +8.93 -> +0.43 SAP on this field.
+    # Geometry: 100 m² plan → roof area 100 m². 0.25 × 100 = 25 W/K.
+    main = make_building_part(
+        construction_age_band="E",
+        wall_construction=4,
+        wall_insulation_type=4,
+        party_wall_construction=1,
+        roof_construction=4,
+        floor_dimensions=[
+            make_floor_dimension(
+                total_floor_area_m2=100.0, room_height_m=2.5,
+                party_wall_length_m=0.0, heat_loss_perimeter_m=40.0, floor=0,
+            ),
+        ],
+    )
+    main.roof_insulation_location = 1  # Rafters
+    main.roof_insulation_thickness = None  # gov leaves this None for rafters
+    main.rafter_insulation_thickness = "225mm"  # the thickness lives here
+    epc = make_minimal_sap10_epc(
+        total_floor_area_m2=100.0,
+        country_code="ENG",
+        sap_building_parts=[main],
+    )
+
+    # Act
+    result = heat_transmission_from_cert(epc)
+
+    # Assert
+    assert abs(result.roof_w_per_k - 25.0) <= 1e-4
+
+
 def test_lodged_roof_u_value_overrides_construction_default() -> None:
     # Arrange — RdSAP 10 §5.1: where an element's U-value is known from the
     # assessment (documentary evidence / the lodged RdSAP output) it is used

tests/infrastructure/epc_client/test_sap_accuracy_corpus.py

@@ -68,22 +68,20 @@ _CORPUS = Path(
 # So closing demand over-estimates lifts BOTH the SAP gauge and PE/CO2; there is
 # no one-slice factor fix. RATCHET any ceiling up when a slice tightens it.
 #
-# RAFTERS ROOF U-TABLE (RdSAP 10 §5.11.2 Table 16 col 2 + §5.11 Table 18 col 2):
-# this slice billed roofs lodged insulated AT RAFTERS (roof_insulation_location
-# == 1) on the spec rafters column instead of the joists column. Within-0.5 went
-# 66.9% -> 66.5% (MAE 1.039 -> 1.064) — a SPEC-CORRECT move, NOT a regression to
-# chase. The calculator is worksheet-validated to 1e-4 on simulated case 41
-# (4-bp: measured rafters 200mm -> 0.29; rafters As-Built band F -> 0.68) and
-# case 42 (6 variants: rafters 50mm -> 0.88; rafters unknown band C -> 2.30 per
-# Table 18 footnote 1 "applies for unknown and as built"). The dip is a gov
-# open-data REDACTION artifact: all 15 corpus rafter certs carry NO thickness
-# (blanked to None) yet lodge roof energy_efficiency_rating 2-4 (insulated),
-# proving they had a SPECIFIED thickness the open API redacted. With the
-# thickness gone the spec's unknown-rafter default (2.30) correctly fires but
-# over-states those certs' real (insulated) roof. Recovering them needs a
-# roof-EER -> assumed-thickness inference on the API path (future slice), NOT a
-# change to the spec-correct U-table. Do NOT revert the rafters column to "fix"
-# the gauge.
+# RAFTERS ROOF (RdSAP 10 §5.11.2 Table 16 col 2 + §5.11 Table 18 col 2): roofs
+# insulated AT RAFTERS (roof_insulation_location == 1) are billed on the spec
+# rafters column instead of the joists column, AND their thickness is read from
+# the dedicated gov-API `rafter_insulation_thickness` field. That field was
+# UNDECLARED on the schema, so `from_dict` dropped it — the rafter certs only
+# *looked* redacted (roof EER 2-4 = insulated yet `roof_insulation_thickness`
+# None); the thickness was there all along in `rafter_insulation_thickness`
+# (e.g. "225mm"). Declaring + threading it recovers them: cert 3100-8675-0922
+# (band E, rafters 225mm) +8.93 -> +0.43 SAP. Net of both changes within-0.5
+# went 66.9% -> 67.0% (MAE 1.039 -> 1.025). Worksheet-validated to 1e-4 on
+# simulated case 41 (measured rafters 200mm -> 0.29; rafters As-Built band F
+# -> 0.68) and case 42 (rafters 50mm -> 0.88; rafters genuine-unknown band C
+# -> 2.30 per Table 18 footnote 1 "applies for unknown and as built"). Do NOT
+# revert the rafters column.
 _MIN_WITHIN_HALF_SAP = 0.65
 _MAX_SAP_MAE = 1.08
 _MAX_CO2_MAE_TONNES = 0.35      # t CO2 / yr vs co2_emissions_current