5e0963593a
A full-SAP cert (assessment_type=SAP, the as-designed LIG-* new-builds) lodged
under a SAP-Schema-16.x version failed to map with:
ValueError: RdSapSchema17_1: missing required field 'door_count'
— crashing the property's modelling_e2e subtask. 89 distinct properties in the
2026-06-24 run hit this; ~22 of every 30 sampled are this full-SAP shape.
Root cause: from_api_response dispatches on the schema_type STRING only, and the
SAP-Schema-16.x branch assumed a single shape — "reduced-field (RdSAP-shaped)" —
sending every 16.x cert through _normalize_sap_schema_16_x -> RdSapSchema17_1.
But the SAP-Schema-16.x name covers two structurally different certs:
* reduced RdSAP (assessment_type=RdSAP): top-level door_count / glazed_area
band / construction-code building parts.
* full SAP (assessment_type=SAP): measured shape — sap_opening_types +
structured sap_building_parts carrying measured U-values and door/window
OPENINGS, with NO top-level door_count (the door is an opening). These omit
the reduced-only count fields, so the reduced normaliser failed loud on the
first one it checks (door_count) — and 14 others behind it.
Force-fitting a full-SAP cert to the reduced RdSAP schema was the bug: the data
was never missing, it was being validated against the wrong schema.
Fix: discriminate on shape (_is_full_sap_cert: assessment_type=SAP AND
sap_opening_types present) and route full-SAP 16.x certs to the existing
full-SAP 17.1 mapper, which reads the real measured openings (the lodged
0.9x2.1m door -> door_count 1) — no reduced-field defaulting. The only field the
16.x full-SAP shape omits that SapSchema17_1 needs is `tenure` (register
metadata, no SAP effect), defaulted. Reduced 16.x certs (assessment_type=RdSAP,
no opening types) are untouched — all pinned reduced 16.x fixtures stay on the
RdSAP path.
Regression test pins a real full-SAP-16.0 cert (0293-...-8795, lodged 83) mapping
via the full-SAP path + the reduced 16.2 cert staying on the RdSAP path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|---|---|---|
| .claude/skills | ||
| .devcontainer | ||
| .github/workflows | ||
| .idea | ||
| .vscode | ||
| applications | ||
| asset_list | ||
| backend | ||
| backlog | ||
| datatypes | ||
| deployment/terraform | ||
| docs | ||
| domain | ||
| epr_data_exports | ||
| etl | ||
| harness | ||
| infrastructure | ||
| model_data/requirements | ||
| orchestration | ||
| recommendations | ||
| repositories | ||
| sap worksheets | ||
| scripts | ||
| sfr/principal_pitch | ||
| survey_report | ||
| tests | ||
| utilities | ||
| utils | ||
| .coveragerc | ||
| .dockerignore | ||
| .gitignore | ||
| __init__.py | ||
| ara_backend_design.md | ||
| BaseUtility.py | ||
| CLAUDE.md | ||
| conftest.py | ||
| CONTEXT.md | ||
| devcontainer.sh | ||
| Dockerfile.test | ||
| Dockerfile.test.dockerignore | ||
| Makefile | ||
| MEMORY.md | ||
| next_claude_prompt.txt | ||
| P960-0001-001431-2.pdf | ||
| package-lock.json | ||
| package.json | ||
| playground.py.local-backup | ||
| pyproject.toml | ||
| pyrightconfig.json | ||
| pytest.ini | ||
| README.md | ||
| run_lambda_local.sh | ||
| serverless.yml | ||
| Summary_001431-3.pdf | ||
| test.requirements.txt | ||
| tox.ini | ||
| UBIQUITOUS_LANGUAGE.md | ||
Model Repository
This repository contains the code pertaining to the development of the data science and machine learning products being utilised by Hestia.
The different folders in this repository relate to services that can be used independently, or can be imported and used as part of a larger application
Getting Started
Prerequisites
Dev Container Setup
This repo uses a Docker Compose-based dev container. The model-backend service joins a shared-dev Docker network so it can communicate with other local services (e.g. a frontend container) running on your machine.
VS Code users: The initializeCommand in devcontainer.json creates the shared-dev network automatically before the container starts. No manual step required — just open the repo and select Reopen in Container.
Non-VS Code / CI workflows: Run the following once before starting the container:
make dev-setup
This is idempotent and safe to re-run if the network already exists.
Folders
backend/
This folder contains the code for the fastapi backend service, which provides an interface to much of the functionality in this repository, for the frontend
model_data/
This folder contains related to the reading and preparation of assessment model data, including pulling out epc attributes
Testing
All tests can be run, against the configuration in pytest.ini running
pytest
This will run the complete panel of tests and report on coverage in the locations specified by the pytest.ini file.
To run tests in a specific service, e.g. inside of model_data, simply run
pytest --cov-config=model_data/.coveragerc --cov=model_data
This will produce the test results and coverage reports