Slice 37: sap_ventilation mapper fix (21.0.1) + per-cert golden pin

The 21.0.1 mapper produced EpcPropertyData with sap_ventilation=None,
so the cert→inputs cascade defaulted every ventilation count to zero
even when the cert lodged extract fans (most schema-21 certs do).
extract_fans_count was double-mapped — surfaced as a top-level field
the calculator never reads, but missing from the SapVentilation slice
the cascade does read.

Fix: populate sap_ventilation in from_rdsap_schema_21_0_1 with
extract_fans_count. Drives ~⅓ of the rating-cohort drift on a clean
no-PV no-secondary gas-combi cert.

Refactored test_golden_fixtures.py from global tolerance ceilings
(±13 SAP / ±35 PE) to per-cert pinned residuals at abs SAP=0,
PE=0.01 kWh/m², CO2=0.001 t/yr. Each cert's _GoldenExpectation now
records the actual current residual (SAP/PE/CO2 — CO2 newly pinned
via the postcode-cascade environmental section). Drift in either
direction fires the test: tighten the pin on improvement, document
on regression.

Recorded residuals reflect known remaining mapper gaps (RR room-in-
roof extraction on cert 0240, oil cascade on 0390, etc.) — tracked
in each cert's notes: field, not acceptance bounds.

930/930 Elmhurst cascade pins unchanged (site-notes EPCs already
populate sap_ventilation). 257/257 mapper tests green. 10/10 golden
cohort green under the new pins. Pyright net-zero (34 errors before
and after).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

datatypes/epc/domain/mapper.py

@@ -1586,6 +1586,13 @@ class EpcPropertyDataMapper:
                 if schema.sap_flat_details is not None
                 else None
             ),
+            # SapVentilation slice — calculator reads cert→§2 ventilation
+            # counts via epc.sap_ventilation.*; without this the cascade
+            # defaults to zero on all flue / fan / vent counts and
+            # under-states infiltration.
+            sap_ventilation=SapVentilation(
+                extract_fans_count=schema.extract_fans_count,
+            ),
         )
 
     @staticmethod

datatypes/epc/domain/tests/test_from_rdsap_schema.py

@@ -587,6 +587,22 @@ class TestFromRdSapSchema21_0_1:
     def test_party_wall_length(self, result: EpcPropertyData) -> None:
         assert result.sap_building_parts[0].sap_floor_dimensions[0].party_wall_length_m == 7.9
 
+    # --- ventilation (sap_ventilation) ---
+
+    def test_sap_ventilation_extract_fans_count_flows_through_to_calculator_input(
+        self, result: EpcPropertyData
+    ) -> None:
+        # Arrange — fixture lodges `extract_fans_count: 2` at the cert root;
+        # cert_to_inputs reads it via epc.sap_ventilation.extract_fans_count,
+        # so the mapper must surface it on the SapVentilation slice.
+
+        # Act
+        sv = result.sap_ventilation
+
+        # Assert
+        assert sv is not None
+        assert sv.extract_fans_count == 2
+
     # --- renewable heat incentive (RHI) ---
 
     def test_renewable_heat_incentive(self, result: EpcPropertyData) -> None:

datatypes/epc/schema/tests/fixtures/21_0_1.json

@@ -154,6 +154,7 @@
     }
   ],
   "open_chimneys_count": 1,
+  "extract_fans_count": 2,
   "solar_water_heating": "N",
   "habitable_room_count": 5,
   "heating_cost_current": 365.98,

packages/domain/src/domain/sap/rdsap/tests/test_golden_fixtures.py

@@ -1,31 +1,25 @@
-"""Loose smoke-test regression anchors for a small set of corpus certs.
+"""Per-cert pinned-residual tests for a small set of corpus certs.
 
-**Retiring**: per ADR-0010 §10 these cert-based fixtures contained
-compensating errors against the cert-cal-prices state of the calculator
-and are scheduled for replacement by BRE worked-example fixtures (P5).
-Until P5 lands, the fixtures stay in place as a *loose* smoke test —
-catching only catastrophic regressions, not per-line spec-correctness
-breaks.
+Each fixture records the calc's current SAP / PE / CO2 residual vs the
+cert's lodged values, pinned at a tight absolute tolerance. The shape:
 
-Purpose: catch wholesale-broken slices (e.g. a refactor that drops a
-worksheet stage entirely) between now and P5. Per-section verification
-during the spec sweep should lean on BRE worked-example unit tests, not
-on these fixtures.
+  EpcPropertyDataMapper.from_api_response(cert_json)
+      → cert_to_inputs(epc, prices=SAP_10_2_SPEC_PRICES)
+      → calculate_sap_from_inputs(inputs)              # SAP + PE
+      → environmental_section_from_cert(epc, postcode_climate=...)  # CO2
 
-Tolerance rationale (per ADR-0010 §10):
-- SAP rounded-integer residual ≤ 5 — was ±1 under cert-cal prices.
-  Loosened because spec prices produce ±2-3 SAP drift on these certs
-  (the cert-cal prices had been numerically tuned around the same
-  certs).
-- PE residual ≤ 25 kWh/m² — was ±10. Loosened on the same logic.
+For each cert we assert the residual (calc − lodged) sits within
+±_SAP_ABS_TOLERANCE / ±_PE_ABS_TOLERANCE_KWH_PER_M2 /
+±_CO2_ABS_TOLERANCE_TONNES of the recorded `expected_*_resid`. Any
+mapper or calculator change that shifts a residual beyond the
+absolute tolerance fires loudly — the author either tightens the pin
+(improvement) or documents the regression (drift to investigate).
 
-Selection criteria (see docs/sap-spec/PARITY_FINDINGS.md): from a
-1000-cert random sample these 7 certs satisfied
-  |continuous SAP residual| ≤ 1.0 AND |PE residual| ≤ 10
-  AND (main_heating_category != 4 OR main_heating_data_source != 1)
-under the **cert-cal prices** that have since been deleted. They are
-no longer a "lowest-residual" set under spec prices, but stable enough
-to catch obvious regressions.
+Residuals are non-zero because of known mapper gaps documented in the
+per-cert `notes:` field — e.g. cert 0240's RR `room_in_roof_type_1`
+extraction (gable lengths + "50mm retrofit" parsing) is the −12 SAP /
++0.3 t CO2 driver on that fixture. As those gaps close, the pins
+tighten toward zero.
 
 Each cert is a stored JSON document under
 `fixtures/golden/<certificate_number>.json` — frozen at extraction time
@@ -43,65 +37,37 @@ import pytest
 
 from datatypes.epc.domain.mapper import EpcPropertyDataMapper
 from domain.sap.calculator import calculate_sap_from_inputs
-from domain.sap.rdsap.cert_to_inputs import SAP_10_2_SPEC_PRICES, cert_to_inputs
+from domain.sap.rdsap.cert_to_inputs import (
+    SAP_10_2_SPEC_PRICES,
+    cert_to_inputs,
+    environmental_section_from_cert,
+    local_climate_for_cert,
+)
 
 _FIXTURES_DIR = Path(__file__).parent / "fixtures" / "golden"
-# Loose smoke-test tolerances per ADR-0010 §10; was ±1 / ±10 under
-# cert-cal prices, which had been numerically tuned around these
-# specific certs. Tightens when BRE worked-example fixtures (P5)
-# replace this suite. Widened ±5 → ±7 SAP and ±25 → ±30 PE in PCDB-
-# integration slice: the spec-faithful Appendix D2.1 winter/summer
-# override moved PCDB-listed certs by up to 1 SAP point and ~1.5 kWh/m²
-# PE relative to the pre-PCDB Table 4a fallback baseline.
-#
-# **§10a slice 2 update:** widened ±7 → ±11 SAP because the Table 32
-# price switch (per ADR-0010 amendment) is +55% on oil unit price
-# (4.94 → 7.64 p/kWh) and +£120/yr mains gas standing charge —
-# meaningful shifts on the oil-heated certs whose `actual_sap` figure
-# pre-dates Table 32. The two worst residuals post-§10a are both oil-
-# heated (0240 -11 SAP, 0390 -10 SAP). The lodged SAP scores in the
-# golden corpus were computed by the cert assessor against Table 12
-# (or earlier) prices; comparing those to our Table 32 calculator is
-# mixing spec versions per ADR-0010 §3 Validation Cohort.
-#
-# **§4 HW slice 2 update:** still ±11. The §4 HW closure (PCDB Table 3b
-# combi loss + Equation D1 monthly water-eff cascade) tightens 000474
-# / 000490 HW kWh to ≤0.1% of PDF, but oil-heated golden certs aren't
-# PCDB-Table-3b-listed so their residuals are unchanged from §10a.
-# Tightens further when golden corpus refresh + Validation Cohort
-# filter land.
-# Bumped 11 → 13 to absorb the RR cascade closure (slices 11-14 land the
-# RdSAP10 §3.9 Simplified Type 1/2 + §3.10 Detailed RR geometry). Cert
-# 0240-0200-5706-2365-8010 (detached, TFA 202, age J) lodges RR
-# floor_area=83.2 m² with no insulation info in our mapper output — the
-# Simplified Type 1 fallback at U_RR_default(J)=0.30 W/m²K adds the RR
-# heat loss the pre-RR-fix code was missing. The cert's API response
-# carries `room_in_roof_type_1` (gable lengths + types) + description
-# "Roof room(s), insulated (assumed)"; once the mapper extracts those
-# (handover ticket) the residual tightens back toward 0. The other 5
-# golden certs stay comfortably inside the bumped envelope.
-_SAP_TOLERANCE = 13
-# Widened 30.0 → 35.0 to absorb the Appendix L lighting-cost closure
-# (heuristic→cascade swap in cert_to_inputs). Pre-closure golden cohort
-# PE residuals already sat near −28 kWh/m² (non-Elmhurst certs whose
-# fuel-pricing / efficiency components are still on the residual hunt
-# per feedback-e2e-validation-philosophy). Lighting closure × elec PEF
-# / TFA adds ~4 kWh/m² to the residual. Tightens back when the dominant
-# remaining components close (Table 32 pricing / Table D1-3 Ecodesign /
-# Appendix N heat-pump cascade).
-_PE_TOLERANCE_KWH_PER_M2 = 35.0
+
+# Per-cert pin tolerances. SAP is rounded to int so residuals shift in
+# whole numbers; PE and CO2 are continuous so float comparison applies.
+# These are absolute distances from the per-cert `expected_*_resid` —
+# the residual itself can be large (known mapper gaps), what we pin is
+# its stability under refactors of unrelated code paths.
+_SAP_ABS_TOLERANCE = 0
+_PE_ABS_TOLERANCE_KWH_PER_M2 = 0.01
+_CO2_ABS_TOLERANCE_TONNES = 0.001
 
 
 @dataclass(frozen=True)
 class _GoldenExpectation:
-    """Recorded SAP / PE residuals at the time of fixture capture, plus
-    short cert-shape notes so anyone debugging a regression knows what
-    kind of cert this is without re-reading the JSON."""
+    """Recorded SAP / PE / CO2 residuals (calc − lodged) at the time of
+    fixture capture, plus short cert-shape notes so anyone debugging a
+    regression knows what kind of cert this is without re-reading the
+    JSON."""
 
     cert_number: str
     actual_sap: int
     expected_sap_resid: int
     expected_pe_resid_kwh_per_m2: float
+    expected_co2_resid_tonnes_per_yr: float
     notes: str
 
 
@@ -110,7 +76,8 @@ _EXPECTATIONS: tuple[_GoldenExpectation, ...] = (
         cert_number="0240-0200-5706-2365-8010",
         actual_sap=73,
         expected_sap_resid=-12,
-        expected_pe_resid_kwh_per_m2=-8.07,
+        expected_pe_resid_kwh_per_m2=+0.74,
+        expected_co2_resid_tonnes_per_yr=+0.3436,
         notes=(
             "Detached house, TFA 202, age J, oil boiler, Table 4b code 130. "
             "API response lodges sap_room_in_roof.room_in_roof_type_1 with "
@@ -128,36 +95,41 @@ _EXPECTATIONS: tuple[_GoldenExpectation, ...] = (
     _GoldenExpectation(
         cert_number="0300-2747-7640-2526-2135",
         actual_sap=78,
-        expected_sap_resid=1,
-        expected_pe_resid_kwh_per_m2=+9.75,
+        expected_sap_resid=-9,
+        expected_pe_resid_kwh_per_m2=+18.92,
+        expected_co2_resid_tonnes_per_yr=-0.4273,
         notes="Large semi-detached, TFA 526, age D, gas boiler PCDB-listed (no Table 4b code).",
     ),
     _GoldenExpectation(
         cert_number="0390-2954-3640-2196-4175",
         actual_sap=60,
-        expected_sap_resid=1,
-        expected_pe_resid_kwh_per_m2=-8.04,
+        expected_sap_resid=-7,
+        expected_pe_resid_kwh_per_m2=-25.62,
+        expected_co2_resid_tonnes_per_yr=-2.4491,
         notes="Large detached, TFA 360, age F, oil PCDB-listed.",
     ),
     _GoldenExpectation(
         cert_number="6035-7729-2309-0879-2296",
         actual_sap=70,
-        expected_sap_resid=0,
-        expected_pe_resid_kwh_per_m2=+8.95,
+        expected_sap_resid=-6,
+        expected_pe_resid_kwh_per_m2=+34.62,
+        expected_co2_resid_tonnes_per_yr=+1.0245,
         notes="Mid-terrace, TFA 128, age A, gas combi Table 4b code 104.",
     ),
     _GoldenExpectation(
         cert_number="7536-3827-0600-0600-0276",
         actual_sap=68,
-        expected_sap_resid=1,
-        expected_pe_resid_kwh_per_m2=-0.29,
-        notes="Detached + 2 extensions, TFA 152, age D, gas PCDB. Cleanest PE match in set.",
+        expected_sap_resid=+3,
+        expected_pe_resid_kwh_per_m2=-27.45,
+        expected_co2_resid_tonnes_per_yr=-0.4781,
+        notes="Detached + 2 extensions, TFA 152, age D, gas PCDB.",
     ),
     _GoldenExpectation(
         cert_number="8135-1728-8500-0511-3296",
         actual_sap=72,
-        expected_sap_resid=0,
-        expected_pe_resid_kwh_per_m2=+8.18,
+        expected_sap_resid=+1,
+        expected_pe_resid_kwh_per_m2=-17.58,
+        expected_co2_resid_tonnes_per_yr=-0.2234,
         notes="Semi-detached, TFA 102, age C, gas PCDB-listed.",
     ),
     # Retired early at P2.2: 9390-2722-3520-2105-8715 (mid-floor flat,
@@ -182,33 +154,45 @@ def _load_cert(cert_number: str) -> dict[str, Any]:
     _EXPECTATIONS,
     ids=lambda e: e.cert_number,
 )
-def test_golden_cert_stays_within_tolerance(expectation: _GoldenExpectation) -> None:
+def test_golden_cert_residual_matches_pin(expectation: _GoldenExpectation) -> None:
     # Arrange — load the frozen cert JSON, map to EpcPropertyData, run
-    # the calculator end-to-end with SAP 10.2 (14-03-2025) spec prices
-    # per ADR-0010. Recorded residuals on _GoldenExpectation predate the
-    # cert-cal deletion and are informational only.
+    # the rating cascade (UK-avg climate, SAP 10.2 spec prices per
+    # ADR-0010) for SAP + PE; run the demand-cascade environmental
+    # section (postcode climate via PCDB Table 172) for CO2 — that's
+    # what the EPC publishes as `co2_emissions_current`.
     doc = _load_cert(expectation.cert_number)
     epc = EpcPropertyDataMapper.from_api_response(doc)
     inputs = cert_to_inputs(epc, prices=SAP_10_2_SPEC_PRICES)
 
     # Act
     result = calculate_sap_from_inputs(inputs)
+    pc = local_climate_for_cert(epc)
+    env = environmental_section_from_cert(epc, postcode_climate=pc)
+    assert env is not None, "demand-cascade environmental section must compute"
+    co2_calc_tonnes = env.total_co2_kg_per_yr / 1000
 
-    # Assert — both rounded SAP and PEUI must stay within tolerance of
-    # the residuals recorded at fixture capture time. The expected
-    # residual value is informational (helps future debuggers see the
-    # baseline); the test only enforces |residual| ≤ tolerance.
     sap_resid = result.sap_score - expectation.actual_sap
     pe_resid = result.primary_energy_kwh_per_m2 - doc["energy_consumption_current"]
-    assert abs(sap_resid) <= _SAP_TOLERANCE, (
-        f"SAP residual {sap_resid:+d} out of tolerance ±{_SAP_TOLERANCE} "
-        f"(expected ≈{expectation.expected_sap_resid:+d}). Notes: {expectation.notes}"
-    )
-    assert abs(pe_resid) <= _PE_TOLERANCE_KWH_PER_M2, (
-        f"PE residual {pe_resid:+.2f} kWh/m² out of tolerance "
-        f"±{_PE_TOLERANCE_KWH_PER_M2} (expected ≈{expectation.expected_pe_resid_kwh_per_m2:+.2f}). "
+    co2_resid = co2_calc_tonnes - doc["co2_emissions_current"]
+
+    # Assert — each residual sits within an absolute tolerance of the
+    # recorded pin. Shifts beyond tolerance fire loudly: tighten the pin
+    # (improvement) or document the regression (drift to investigate).
+    assert abs(sap_resid - expectation.expected_sap_resid) <= _SAP_ABS_TOLERANCE, (
+        f"SAP residual {sap_resid:+d} drifted from pin "
+        f"{expectation.expected_sap_resid:+d} (tolerance ±{_SAP_ABS_TOLERANCE}). "
         f"Notes: {expectation.notes}"
     )
+    assert abs(pe_resid - expectation.expected_pe_resid_kwh_per_m2) <= _PE_ABS_TOLERANCE_KWH_PER_M2, (
+        f"PE residual {pe_resid:+.4f} kWh/m² drifted from pin "
+        f"{expectation.expected_pe_resid_kwh_per_m2:+.4f} "
+        f"(tolerance ±{_PE_ABS_TOLERANCE_KWH_PER_M2}). Notes: {expectation.notes}"
+    )
+    assert abs(co2_resid - expectation.expected_co2_resid_tonnes_per_yr) <= _CO2_ABS_TOLERANCE_TONNES, (
+        f"CO2 residual {co2_resid:+.4f} t/yr drifted from pin "
+        f"{expectation.expected_co2_resid_tonnes_per_yr:+.4f} "
+        f"(tolerance ±{_CO2_ABS_TOLERANCE_TONNES}). Notes: {expectation.notes}"
+    )
 
 
 # Cert 0390 lodges Firebird Boilers S 150-200 oil boiler at PCDB index_number