Contains projects associated to the development of modeling products
Find a file
Jun-te Kim 41ded4740e Cascade raw API glazing_type on lodged sap_windows, not pass through raw
datatypes/epc/domain/mapper.py's from_rdsap_schema_{17_0,17_1,18_0,19_0,
20_0_0,21_0_0} all stored a lodged sap_windows[].glazing_type verbatim
instead of routing it through _api_cascade_glazing_type (which
_api_sap_window, used by 21_0_1, already does correctly). The raw API code
is the RdSAP-21 glazing enum (e.g. code 1 = "DG pre-2002"), not the SAP
10.2 Table 6b cascade enum the calculator's U/g-value tables are keyed on
(cascade code 1 = single glazed) -- so any lodged window whose raw code
happened to collide with a different cascade meaning was silently
mis-rated.

Found while validating PR #1503 (property 753950 / uprn 100021969385)
against Elmhurst: our engine's windows_w_per_k was 82.27 vs Elmhurst's
51.44 -- the dwelling's larger window (91% of glazed area) lodges raw
glazing_type=1 and was being modelled as single glazed instead of double,
the dominant contributor to a SAP 42 vs Elmhurst 44 gap. Fixing this
narrows windows_w_per_k to 50.26 (Elmhurst 51.44, ~2% off) and moves the
engine's score to 43.

Extracted the 5 duplicated reduced-schema-window blocks (17.0/17.1/18.0/
19.0/20.0.0, which lodge a minimal window shape with no glazing_gap/
frame_factor/transmission fields) into a shared _reduced_field_api_sap_
window helper that also now populates window_transmission_details/
frame_factor via the SAP10 lookup instead of leaving them None. 21.0.0's
richer inline block gets the same one-line cascade fix.

+6 regression tests (one per affected schema seam + 21.0.0). Full domain/
schema/prediction/comparable-properties/sap10_calculator/modelling_e2e
suites pass (2746 passed, only the 2 pre-existing unrelated failures);
component-accuracy gate 26/26 unaffected.
2026-07-08 13:05:43 +00:00
.claude/skills docs(skill): bank corpus-1000 findings (cost triangulation, software divergences, assumed-roof rule) 2026-07-02 19:12:48 +00:00
.devcontainer Map shard rows to HistoricEpc field-by-field so every column is pyright-checked 🟩 2026-07-04 14:58:09 +00:00
.github/workflows terraform correction 2026-07-07 15:23:44 +00:00
.idea scaffolding for ml pipeline 2026-05-16 14:15:56 +00:00
.vscode added utils to allow easier subtask management 2026-03-02 15:15:39 +00:00
applications abri_api not abri 2026-07-07 14:40:33 +00:00
asset_list redeploy old postcode splitter 2026-05-21 09:46:47 +00:00
backend refactor(address2uprn): name the match/decision return types; rename helper 2026-07-07 16:00:00 +00:00
backlog UploadedFile, FileTypeEnum, FileSourceEnum importable from infrastructure.postgres.uploaded_file_table 🟥 2026-06-09 11:42:53 +00:00
datatypes Cascade raw API glazing_type on lodged sap_windows, not pass through raw 2026-07-08 13:05:43 +00:00
deployment/terraform abri_api not abri 2026-07-07 14:40:33 +00:00
docs test(address2uprn): cover ambiguity withholding + override dedup; add ADR-0057 2026-07-07 15:21:04 +00:00
domain Merge pull request #1481 from Hestia-Homes/feature/modelling-trigger-run 2026-07-07 14:36:45 +01:00
epr_data_exports allowing carbon and energy otimisation by removing slack 2025-07-31 19:13:16 +01:00
etl abri_api not abri 2026-07-07 14:40:33 +00:00
harness Expired-pairs integration gate: frozen single-file corpus + ratcheting floors 🟩 2026-07-06 13:05:18 +00:00
infrastructure Drop property-table wiring for certificate_number 2026-07-07 14:16:55 +00:00
model_data/requirements its working the way khalim wanted wiht postcode and then search that 2026-01-22 15:17:13 +00:00
orchestration Drop property-table wiring for certificate_number 2026-07-07 14:16:55 +00:00
recommendations save 2026-05-07 15:55:44 +00:00
repositories refactor(address2uprn): name the match/decision return types; rename helper 2026-07-07 16:00:00 +00:00
sap worksheets S0380.238: single-point instantaneous water heaters incur no distribution loss 2026-06-05 19:00:02 +00:00
scripts refactor(address2uprn): name the match/decision return types; rename helper 2026-07-07 16:00:00 +00:00
sfr/principal_pitch 17.1 and 18 done by claude 2026-06-12 12:52:36 +00:00
survey_report quidos site notes extraction 2025-02-18 19:49:29 +00:00
tests test(address2uprn): cover ambiguity withholding + override dedup; add ADR-0057 2026-07-07 15:21:04 +00:00
utilities Merge branch 'main' into feature/abri-api-integration 2026-07-07 14:08:52 +00:00
utils SharePoint 404s raise ResourceNotFoundError without error-level logging 🟩 2026-07-03 12:26:33 +00:00
.coveragerc fixed unit tests 2023-10-05 16:04:12 +01:00
.dockerignore deployment from infrastructure 2026-05-19 12:55:30 +00:00
.gitignore Add Hyde property_overrides build script + creds-free smoke test 🟩 2026-06-19 19:02:42 +00:00
__init__.py added checking for directory before creation and made some minor style changes 2023-08-25 15:21:17 +01:00
ara_backend_design.md docs(ara): next-agent handover for Property Baseline (SAP calc) + Modelling 2026-06-01 16:20:06 +00:00
BaseUtility.py fixed missing task and subtask for single remote assessments 2025-11-27 17:50:26 +00:00
CLAUDE.md Annotate locals assigned from cross-module calls in the historic-EPC stack 🟪 2026-07-04 14:59:13 +00:00
conftest.py working on integrating new EPC api into address2UPRN 2026-04-27 11:32:44 +00:00
CONTEXT.md Merge pull request #1481 from Hestia-Homes/feature/modelling-trigger-run 2026-07-07 14:36:45 +01:00
devcontainer.sh add dev container 2026-04-17 14:50:57 +00:00
Dockerfile.test fix: address 22 project-wide test failures from previous sweep 2026-05-26 13:34:51 +00:00
Dockerfile.test.dockerignore deployment from infrastructure 2026-05-19 12:55:30 +00:00
Makefile adding to dev container to create shared network on start up 2026-04-25 15:03:07 +00:00
MEMORY.md memory 2026-04-02 10:24:31 +00:00
modelling_audit.md neighbour_divergence: key cohorts on main-heating fuel 2026-07-01 14:24:31 +00:00
next_claude_prompt.txt scripts? 2026-06-11 07:07:27 +00:00
P960-0001-001431-2.pdf Merge remote-tracking branch 'origin/main' into feature/hyde_make_it_more_accurate_with_tests 2026-06-16 15:23:52 +00:00
package-lock.json restructuring openUrpn code 2023-07-20 11:41:43 +01:00
package.json restructuring openUrpn code 2023-07-20 11:41:43 +01:00
playground.py.local-backup Merge remote-tracking branch 'origin/main' into feature/hyde_make_it_more_accurate_with_tests 2026-06-16 15:23:52 +00:00
pyproject.toml slice 14a: ml_training_data pkg + sample.py (CSV filter + random sample) 2026-05-16 17:39:43 +00:00
pyrightconfig.json slice 14d: build_features wires bulk reader -> mapper -> EpcMlTransform 2026-05-16 18:38:41 +00:00
pytest.ini python upgraded due to enum 2026-06-12 15:47:28 +00:00
README.md adding to dev container to create shared network on start up 2026-04-25 15:03:07 +00:00
run_lambda_local.sh debugging local lambda run and updating the sap point checking condition 2023-09-13 18:47:12 +01:00
serverless.yml added logic to add to serverless 2026-04-22 12:39:44 +00:00
Summary_001431-3.pdf Merge remote-tracking branch 'origin/main' into feature/hyde_make_it_more_accurate_with_tests 2026-06-16 15:23:52 +00:00
test.requirements.txt tests wrong environemnt 2026-05-29 16:17:06 +00:00
tox.ini removing playright install for integration test 2026-04-30 20:08:13 +00:00
UBIQUITOUS_LANGUAGE.md standardist Address 2026-06-01 16:32:48 +00:00

Model Repository

This repository contains the code pertaining to the development of the data science and machine learning products being utilised by Hestia.

The different folders in this repository relate to services that can be used independently, or can be imported and used as part of a larger application

Getting Started

Prerequisites

Dev Container Setup

This repo uses a Docker Compose-based dev container. The model-backend service joins a shared-dev Docker network so it can communicate with other local services (e.g. a frontend container) running on your machine.

VS Code users: The initializeCommand in devcontainer.json creates the shared-dev network automatically before the container starts. No manual step required — just open the repo and select Reopen in Container.

Non-VS Code / CI workflows: Run the following once before starting the container:

make dev-setup

This is idempotent and safe to re-run if the network already exists.

Folders

backend/

This folder contains the code for the fastapi backend service, which provides an interface to much of the functionality in this repository, for the frontend

model_data/

This folder contains related to the reading and preparation of assessment model data, including pulling out epc attributes

Testing

All tests can be run, against the configuration in pytest.ini running

pytest

This will run the complete panel of tests and report on coverage in the locations specified by the pytest.ini file.

To run tests in a specific service, e.g. inside of model_data, simply run

pytest --cov-config=model_data/.coveragerc --cov=model_data

This will produce the test results and coverage reports