Model/domain/sap10_calculator/docs/HANDOVER_NEXT.md
Khalim Conn-Kowlessar a7b08a4e8f refactor: move docs/sap-spec/ contents into domain/sap10_calculator/
Locality of reference — SAP-specific docs, specs, and runtime data
now live alongside the calculator that consumes them, mirroring the
prior packages→domain layout moves.

Move targets:

- Narrative MDs → domain/sap10_calculator/docs/
    NEXT_AGENT_PROMPT.md, HANDOVER_NEXT.md, SAP_CALCULATOR.md
- Spec PDFs → domain/sap10_calculator/docs/specs/
    RdSAP 10 Specification 10-06-2025.pdf
    PCDF_Spec_Rev-06b_12_May_2021.pdf
    sap-10-2-full-specification-2025-03-14.pdf
    sap-10-3-full-specification-2026-01-13.pdf
- PCDB runtime data → domain/sap10_calculator/tables/pcdb/data/
    pcdb10.dat (8.3MB) + 7× pcdb_table_*.jsonl (18MB total)

Path code rewrites (load-bearing):

- tables/pcdb/__init__.py: replaced parents[4]/'docs'/'sap-spec' with
  Path(__file__).resolve().parent/'data' for Table 105 JSONL loading.
- tables/pcdb/postcode_weather.py: same rebase for the pcdb10.dat path
  read by _postcode_climate_table().
- tables/pcdb/etl.py __main__: same rebase for the manual ETL invocation
  (source + output_dir both now point inside the package).
- tests/test_pcdb_etl.py: _PCDB_DAT_PATH now derives from
  parents[1]/'tables'/'pcdb'/'data' (was parents[3]/'docs'/'sap-spec').

Citation rewrites:

- 12 .py docstrings and 4 .md docs (ADRs + READMEs + narrative docs)
  had `docs/sap-spec/<file>` strings rewritten to their new locations.
- Two cases where the catch-all sed misfired (an ADR-0009 line about a
  PCDB extract; the pcdb __init__.py docstring about ETL output) were
  hand-corrected to point at tables/pcdb/data/ rather than docs/specs/.

docs/sap-spec/ is now empty (will be removed in a follow-up sweep or
left as a vestigial empty dir for future repurposing). ADRs 0009 and
0010 remain at docs/adr/ — they're part of the chronological
cross-cutting decision log, not calculator-specific narrative.

Verified:

- Calculator's 1e-4 production gate
  (test_api_001479_full_chain_sap_matches_worksheet_pdf_exactly) GREEN.
- Wider sweep (domain/sap10_calculator/ + domain/sap10_ml/): 1654
  passed / 20 failed — exact pre-move baseline. All 20 failures
  pre-existing (10 hand-built skeleton + 4 cohort chain + 6 cohort
  diff).
- Pyright net-zero on the 4 touched runtime/test files (0 errors)
  and unchanged on heat_transmission.py (13) / cert_to_inputs.py (35) /
  mapper.py (33).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-26 13:17:18 +00:00

194 lines
8.4 KiB
Markdown

# Handover — API → SAP integration test
The SAP 10.2 / RdSAP 10 calculator is **closed**: 930/930 pin tests
green against the 6 Elmhurst U985 worksheet PDFs (Rating cascade for
SAP rating + EI rating; Demand cascade for EPC Current Carbon +
Current Primary Energy). Architecture + public API live in
[`SAP_CALCULATOR.md`](./SAP_CALCULATOR.md) — **read that first.**
Your job: build an integration test that runs **API request → cert →
SAP scoring** end-to-end against this calculator, using the 6 Elmhurst
fixtures as the strongest test case in the repo.
---
## What "done" looks like
A test (probably under `backend/` somewhere, exact location TBD by
the codebase shape) that:
1. Spins up the API (FastAPI or whatever the http surface is).
2. Sends a request with a representative `EpcPropertyData` payload
(use one of the 6 Elmhurst fixtures' `build_epc()` outputs as the
reference, or send the upstream JSON shape if that's the boundary).
3. Receives the 4 EPC-facing outputs back through whatever endpoint the
API exposes them on (or invokes the SAP scoring code path the API
would use internally).
4. Asserts the 4 outputs match the fixture's lodged values at the
stated tolerance:
- `sap_score` (integer, exact match)
- `ei_rating` (integer, exact match)
- `current_carbon_kg` (`abs=1e-4` against `DEMAND_LINE_272_TOTAL_CO2`)
- `current_pe_kwh` (`abs=1e-4` against `DEMAND_LINE_286_TOTAL_PE`)
Parametrise the test over all 6 fixtures so any regression in the
plumbing fails loudly.
---
## What's in the box
### Public API (the only thing you need from the SAP module)
```python
from domain.sap10_calculator.rdsap.cert_to_inputs import (
cert_to_inputs, # Rating cascade
cert_to_demand_inputs, # Demand cascade
local_climate_for_cert,
environmental_section_from_cert,
primary_energy_section_from_cert,
)
from domain.sap10_calculator.calculator import calculate_sap_from_inputs, SapResult
```
See `SAP_CALCULATOR.md` §2 for the recommended `dwelling_outputs(epc)`
function shape — copy-paste it as your reference scoring path.
### Fixture cohort (the most comprehensive test case in the repo)
6 real-world certs with full PDF ground-truth:
| Fixture | TFA | Notable cert-shape features |
|---|---|---|
| `_elmhurst_worksheet_000474` | 56.79 | Main + 2 ext, gas combi, no secondary |
| `_elmhurst_worksheet_000477` | 77.58 | RR main-only, electric secondary |
| `_elmhurst_worksheet_000480` | 84.41 | Main + ext + RR, electric secondary |
| `_elmhurst_worksheet_000487` | 81.57 | RR + ext + alt-wall, **electric shower** |
| `_elmhurst_worksheet_000490` | 66.06 | Main + ext |
| `_elmhurst_worksheet_000516` | 90.54 | Main only |
Each fixture exposes:
- `build_epc() -> EpcPropertyData` — encode the cert as our domain type
- `LINE_*` — rating-cascade worksheet expected values (Block 1)
- `DEMAND_LINE_*` — demand-cascade worksheet expected values (Block 2)
- `SAP_VALUE_CONTINUOUS` / `LINE_258_SAP_RATING_INTEGER` — SAP rating
- `LINE_274_EI_RATING_INTEGER` — EI rating
Expected EPC outputs per fixture:
| | sap_score | ei_rating | current_carbon_kg | current_pe_kwh |
|---|---|---|---|---|
| 000474 | 62 | 60 | 3104.1222 | 16931.7227 |
| 000477 | 65 | 69 | 2879.7824 | 16545.4543 |
| 000480 | 61 | 65 | 3479.1552 | 19953.4189 |
| 000487 | 62 | 69 | 3005.2667 | 17755.3174 |
| 000490 | 57 | 61 | 3250.1703 | 18583.7962 |
| 000516 | 63 | 66 | 3501.4376 | 20087.8232 |
---
## What you'll need to investigate
The SAP calculator side is a pure-Python function chain — easy. The API
side is what you need to map out:
1. **Where does cert data enter the system?** Find the FastAPI / Django
/ whatever endpoint that accepts cert input. Look under `backend/`
for routers.
2. **What's the request payload shape?** Is it `EpcPropertyData` JSON
directly, or a different upstream representation that gets mapped?
Check `datatypes/epc/domain/mapper.py` — the mapper from various
schema versions (SAP-Schema-18/19, RdSAP-Schema-18) to
`EpcPropertyData` lives there.
3. **Is SAP scoring already wired to the API?** Search the backend for
imports of `domain.sap10_calculator.rdsap.cert_to_inputs` or
`domain.sap10_calculator.calculator`. If it's not yet wired, the integration test
is a forcing function for wiring it.
4. **What's the response shape?** The 4 outputs above are what the EPC
publishes; the API may already expose them, or may expose a wider
surface (per-section breakdown for retrofit modelling, etc.).
If the API doesn't yet expose SAP scoring, the integration test scope
might include adding the endpoint. Confirm scope with the user before
expanding.
---
## Workflow conventions (from the SAP cleanup work)
- **AAA tests** — `# Arrange / # Act / # Assert` headers on every new
test.
- **One slice = one commit** with Co-Authored-By trailer.
- **`pytest.approx(..., abs=1e-4)` for the EPC outputs** — same bar as
the SAP cascade tests. The 4 expected values above are at 4 d.p. so
abs=1e-4 is the floor.
- **Don't widen tolerances.** If a pin fails, it's a real bug (probably
in the API plumbing, since the calculator is closed).
---
## Files to read on day 1
| File | Why |
|---|---|
| [`domain/sap10_calculator/docs/SAP_CALCULATOR.md`](./SAP_CALCULATOR.md) | Module API + architecture (you're heading there) |
| [`domain/sap10_calculator/calculator.py`](../../domain/sap10_calculator/calculator.py) | `SapResult` fields you'll assert against |
| [`domain/sap10_calculator/rdsap/cert_to_inputs.py`](../../domain/sap10_calculator/rdsap/cert_to_inputs.py) | The 3 public entry points + the section helpers |
| [`domain/sap10_calculator/worksheet/tests/_elmhurst_worksheet_000474.py`](../../domain/sap10_calculator/worksheet/tests/_elmhurst_worksheet_000474.py) | A reference fixture — `build_epc()` shows the EpcPropertyData shape |
| [`domain/sap10_calculator/worksheet/tests/test_e2e_elmhurst_sap_score.py`](../../domain/sap10_calculator/worksheet/tests/test_e2e_elmhurst_sap_score.py) | The current e2e test pattern — model your integration test on this |
| `backend/` (explore) | API entry points |
| [`datatypes/epc/domain/mapper.py`](../../datatypes/epc/domain/mapper.py) | Schema → EpcPropertyData mappers |
---
## Quick orient
```bash
# Confirm SAP calculator is still 930/930 green
python -m pytest \
domain/sap10_calculator/worksheet/tests/test_section_cascade_pins.py \
domain/sap10_calculator/worksheet/tests/test_e2e_elmhurst_sap_score.py \
--no-cov --no-header --tb=no -q
# Show the 4 EPC outputs for fixture 000474
cd packages/domain/src && python -c "
from domain.sap10_calculator.rdsap.cert_to_inputs import (
cert_to_inputs, local_climate_for_cert,
environmental_section_from_cert, primary_energy_section_from_cert,
)
from domain.sap10_calculator.calculator import calculate_sap_from_inputs
from domain.sap10_calculator.worksheet.tests import _elmhurst_worksheet_000474 as w
epc = w.build_epc()
pc = local_climate_for_cert(epc)
rating = calculate_sap_from_inputs(cert_to_inputs(epc))
env_rating = environmental_section_from_cert(epc)
env_demand = environmental_section_from_cert(epc, postcode_climate=pc)
pe_demand = primary_energy_section_from_cert(epc, postcode_climate=pc)
print(f'SAP: {rating.sap_score}') # 62 (UK-avg)
print(f'EI: {env_rating.ei_rating_integer}') # 60 (UK-avg)
print(f'Carbon: {env_demand.total_co2_kg_per_yr:.4f} kg/yr') # 3104.1222 (postcode)
print(f'PE: {pe_demand.total_pe_kwh_per_yr:.4f} kWh/yr') # 16931.7227 (postcode)
"
```
**Important:** SAP rating and EI rating use UK-average climate; Current
Carbon and Current Primary Energy use postcode climate. Don't read EI
from the demand-cascade `environmental_section_from_cert` — that's a
postcode-conditions EI value, not what the EPC publishes.
---
## What's NOT in scope
- **Extending the SAP calculator.** It's closed at the EPC-output layer.
If you find an additional cert-shape variation that breaks the
calculator, capture it as a new conformance fixture (see
`domain/sap10_calculator/README.md`) — don't paper over it in
the integration test.
- **BEDF fuel pricing.** The Fuel Bill on the EPC uses postcode-specific
BEDF prices (PCDB Table 200), which are deferred. The 4 outputs above
cover SAP + EI + Carbon + PE; Fuel Bill is a follow-up.
- **The Demand-SAP "improved dwelling" cascade.** That's Block 3 of the
U985 worksheet (retrofit-applied SAP rating). Out of scope.
Good luck. The SAP side is solid; this is purely a plumbing exercise.