fe04cd3a35
Parser/ETL for BRE PCDB pcdb10.dat (April 2026 revision). domain.sap.tables.pcdb.parser exposes parse_table_105 (typed GasOilBoilerRecord with brand/model/winter+summer+comparative-HW efficiency/output kW/final year) plus parse_table_raw for generic positional ingestion (pcdb_id + raw row only). etl.py runs the full ETL: reads pcdb10.dat as latin-1, writes per-table .jsonl files under docs/sap-spec/. Idempotent; runnable via PYTHONPATH=packages/domain/src python -m domain.sap.tables.pcdb.etl. Per Q1=D grilling: all 8 tables of interest ingested — 105 (Gas/Oil Boilers, typed) plus 122/143/313/353/362/391/506 (raw). Per-table typed refinement deferred to the follow-up slices that wire each table's cert-side cascade. Per Q3=B: typed fields decode against ncm-pcdb.org.uk ground-truth records (Baxi 000098 + Potterton 000619 + Saunier Duval 000732 verified by user); full raw row preserved on every record for forensics. Per Q2 user choice: NDJSON .jsonl format chosen over indented JSON to keep diff-friendliness while halving file size (17MB total vs 31MB pretty-printed). Edge cases handled: latin-1 encoding (manufacturer addresses carry the degree sign), `'obsolete'` status string where a year would otherwise live, `'>70kW'` range indicator on output-power fields — non-numeric values fall to None with the raw string preserved on `raw`. Slice 2 lands the domain.sap.tables.pcdb runtime lookup module (per-table by-pcdb-id dicts loaded at import time). Slice 3 wires Table 105 into cert_to_inputs.main_heating_efficiency / water_efficiency precedence cascades per Q5=B (space heating + water heating scalar override; equation D1 monthly + Appendix N HP factor + FGHRS/WWHRS/HIU deferred). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|---|---|---|
| .devcontainer | ||
| .github/workflows | ||
| .idea | ||
| .vscode | ||
| asset_list | ||
| backend | ||
| backlog | ||
| datatypes | ||
| docs | ||
| epr_data_exports | ||
| etl | ||
| infrastructure/terraform | ||
| model_data/requirements | ||
| packages | ||
| recommendations | ||
| scripts | ||
| services | ||
| sfr/principal_pitch | ||
| survey_report | ||
| utils | ||
| .coveragerc | ||
| .dockerignore | ||
| .gitignore | ||
| __init__.py | ||
| AGENTS.md | ||
| ara_backend_design.md | ||
| BaseUtility.py | ||
| CLAUDE.md | ||
| conftest.py | ||
| CONTEXT.md | ||
| devcontainer.sh | ||
| Dockerfile.test | ||
| Dockerfile.test.dockerignore | ||
| Makefile | ||
| MEMORY.md | ||
| package-lock.json | ||
| package.json | ||
| pyproject.toml | ||
| pyrightconfig.json | ||
| pytest.ini | ||
| README.md | ||
| run_backlog.sh | ||
| run_lambda_local.sh | ||
| serverless.yml | ||
| test.requirements.txt | ||
| tox.ini | ||
| UBIQUITOUS_LANGUAGE.md | ||
Model Repository
This repository contains the code pertaining to the development of the data science and machine learning products being utilised by Hestia.
The different folders in this repository relate to services that can be used independently, or can be imported and used as part of a larger application
Getting Started
Prerequisites
Dev Container Setup
This repo uses a Docker Compose-based dev container. The model-backend service joins a shared-dev Docker network so it can communicate with other local services (e.g. a frontend container) running on your machine.
VS Code users: The initializeCommand in devcontainer.json creates the shared-dev network automatically before the container starts. No manual step required — just open the repo and select Reopen in Container.
Non-VS Code / CI workflows: Run the following once before starting the container:
make dev-setup
This is idempotent and safe to re-run if the network already exists.
Folders
backend/
This folder contains the code for the fastapi backend service, which provides an interface to much of the functionality in this repository, for the frontend
model_data/
This folder contains related to the reading and preparation of assessment model data, including pulling out epc attributes
Testing
All tests can be run, against the configuration in pytest.ini running
pytest
This will run the complete panel of tests and report on coverage in the locations specified by the pytest.ini file.
To run tests in a specific service, e.g. inside of model_data, simply run
pytest --cov-config=model_data/.coveragerc --cov=model_data
This will produce the test results and coverage reports