mirror of
https://github.com/Hestia-Homes/Model.git
synced 2026-06-30 13:10:47 +00:00
Merge remote-tracking branch 'origin/main' into feature/e2e-runs
This commit is contained in:
Khalim Conn-Kowlessar
commit
01bc93ed33
27 changed files with 1608 additions and 374 deletions
|
|
@ -120,6 +120,53 @@ malformed cert (e.g. PV-as-list AttributeError) is skipped+reported too, not jus
|
|||
missing-field ValueErrors; transient `EpcApiError` (subclasses `Exception`) still
|
||||
propagates. +regression test.
|
||||
|
||||
### 🔧🔍 modelling_e2e failure sweep — later run (2026-06-23 14:48 BST, portfolio 796, scenario 1268)
|
||||
A second `modelling_e2e` run (filter `task_source='modelling_e2e' AND job_started >
|
||||
'2026-06-23 14:48:01.806 +0100'`): **200 tasks, 30 FAILED (15%) → 22 unique props**
|
||||
(8 retried once). The DB only stores bare `property_id` in subtask `outputs`
|
||||
(`cloud_logs_url` empty) — root causes recovered by replaying each through
|
||||
`scripts.run_modelling_e2e --scenario-id 1268`. Four root-cause clusters:
|
||||
|
||||
**① ✅ FIXED — `KeyError: BuildingPartIdentifier.EXTENSION_1` (14 props, the dominant
|
||||
failure).** A **Landlord Override** targeted `extension_1` but the lodged/predicted EPC
|
||||
had no such part; `overlay_applicator.apply_simulations` indexed `parts_by_id[identifier]`
|
||||
unguarded → crashed the whole property. Root cause is a **numbering mismatch**: the
|
||||
override's `building_part` is a positional index (0=main, 1=extension 1…, ADR-0004), but
|
||||
the gov-API EPC can label that slot differently (720142's 2nd part lodged as `other`, no
|
||||
`extension_1`). Fix (per product decision — *honour the override, don't drop it*):
|
||||
`_resolve_part` falls back to the EPC's part **at that position** when the semantic label
|
||||
is absent, so the landlord's correction lands; only a position the EPC models no part at
|
||||
is skipped (no geometry to model a wholly-absent part). +regression tests
|
||||
`test_override_for_an_absent_semantic_part_lands_on_the_part_at_that_position` /
|
||||
`test_override_with_no_part_at_that_position_is_skipped`. Not a mapper gap — a
|
||||
modelling-overlay bug. The 14 (pid / uprn):
|
||||
710295/100020458237, 710482/100020450179, 713040/100020404702, 715575/100020397529,
|
||||
715894/100020604961, 717435/22010468, 720142/100020383544, 720560/100021921443,
|
||||
721241/100020453651, 724945/100021915421, 725415/100020404036, 726517/100020631307,
|
||||
726592/100021918195, 730800/100021920273. (Observed post-fix eng: 720142 69.2,
|
||||
710295 63.2→69.8, 720560 61.8→70.4 predicted.)
|
||||
|
||||
**② 🔍 mapper/cascade gap — `UnmappedSapCode: fuel_code 10`** (1 prop, 730259 / uprn
|
||||
100061905741). EPC-less → prediction synthesises a cohort EPC carrying `fuel_code 10`,
|
||||
absent from the calculator's cascade dispatch dict → FATAL (it's the target prop's own
|
||||
fuel, not a skippable cohort cert). FOR KHALIM: add fuel-code-10 to the cascade.
|
||||
|
||||
**③ ⛔ not-predictable — empty/unresolved postcode cohort** (3 props): 714585/100020612517
|
||||
(`CR0 0DD`), 718580/10013149015 (`BR6 6BS`), 723881/22005280 (`BN41 2TP`). EPC-less AND
|
||||
cohort empty after filtering → Prediction Path 3 can't fire. Coverage gap, not a crash bug.
|
||||
|
||||
**④ ⚠ transient — pass clean on replay** (4 props): 712401/100020394694 (eng 75.3),
|
||||
718138/100020397707 (eng 63.5→69.2), 720844/100020472603 (eng 71.8), 723648/100020480302
|
||||
(eng 70.8). Likely a flaky EPC/Solar API call during the batch — no code defect.
|
||||
|
||||
**E2E candidates to pin (best = the lodged-EPC ones, clean to build in Elmhurst):**
|
||||
`[ ]` 100020383544 (pid 720142, eng 69.2) · `[ ]` 100020458237 (710295) · `[ ]`
|
||||
100020404702 (713040) · `[ ]` 100020450179 (710482) · `[ ]` 100021915421 (724945) ·
|
||||
`[ ]` 100021920273 (730800) · `[ ]` 100020394694 (712401, eng 75.3) · `[ ]`
|
||||
100020397707 (718138) · `[ ]` 100020472603 (720844, eng 71.8) · `[ ]` 100020480302
|
||||
(723648, eng 70.8). (Schema not populated on the gov-API objects — confirm when keying
|
||||
each into Elmhurst.)
|
||||
|
||||
### 📋 PLAN — close the 8 modelling_e2e mapping gaps (2026-06-23 run, portfolio 796)
|
||||
The 8 failed prediction targets reduce to **5 distinct mapper-gap classes** (the fix
|
||||
targets). Per class: fix the mapper GENERICALLY, guard with BOTH the RdSAP-21.0.1
|
||||
|
|
|
|||
|
|
@ -24,6 +24,7 @@ ipykernel>=6.25,<7
|
|||
dotenv
|
||||
psycopg[binary]
|
||||
pytest-postgresql
|
||||
moto[s3,sqs]==5.0.28 # mock_aws (moto 5.x) for S3/SQS in orchestration tests
|
||||
# Formatting
|
||||
black==26.1.0
|
||||
boto3-stubs
|
||||
|
|
|
|||
343
.github/workflows/deploy_fastapi_backend.yml
vendored
343
.github/workflows/deploy_fastapi_backend.yml
vendored
|
|
@ -1,170 +1,173 @@
|
|||
name: Fast Api Backend Deploy
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [ dev, prod ]
|
||||
|
||||
jobs:
|
||||
deploy:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v3
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v2
|
||||
with:
|
||||
python-version: 3.10.12
|
||||
|
||||
- name: Install Serverless and plugins
|
||||
run: |
|
||||
npm install -g serverless@^3.38.0
|
||||
npm install -g serverless-domain-manager@^7.3.8
|
||||
npm install -g serverless-python-requirements
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
|
||||
- name: AWS credentials for dev
|
||||
if: github.ref == 'refs/heads/dev'
|
||||
uses: aws-actions/configure-aws-credentials@v1
|
||||
with:
|
||||
aws-access-key-id: ${{ secrets.DEV_AWS_ACCESS_KEY_ID }}
|
||||
aws-secret-access-key: ${{ secrets.DEV_AWS_SECRET_ACCESS_KEY }}
|
||||
aws-region: eu-west-2
|
||||
|
||||
- name: AWS credentials for prod
|
||||
if: github.ref == 'refs/heads/prod'
|
||||
uses: aws-actions/configure-aws-credentials@v1
|
||||
with:
|
||||
aws-access-key-id: ${{ secrets.PROD_AWS_ACCESS_KEY_ID }}
|
||||
aws-secret-access-key: ${{ secrets.PROD_AWS_SECRET_ACCESS_KEY }}
|
||||
aws-region: eu-west-2
|
||||
|
||||
- name: Set domain name
|
||||
id: set_domain
|
||||
run: echo "::set-output name=domain::${{ secrets[format('{0}_DOMAIN_NAME', github.ref_name)] }}"
|
||||
|
||||
- name: Set EPC auth token
|
||||
id: set_auth_token
|
||||
run: echo "::set-output name=auth_token::${{ secrets[format('{0}_EPC_AUTH_TOKEN', github.ref_name)] }}"
|
||||
|
||||
- name: Set Open EPC API token
|
||||
id: set_open_epc_token
|
||||
run: echo "::set-output name=open_epc_token::${{ secrets[format('{0}_OPEN_EPC_API_TOKEN', github.ref_name)] }}"
|
||||
|
||||
# Store port, name and host in github secrets
|
||||
- name: Set DB credentials
|
||||
id: set_db_credentials
|
||||
run: |
|
||||
echo "::set-output name=db_host::${{ secrets[format('{0}_DB_HOST', github.ref_name)] }}"
|
||||
echo "::set-output name=db_port::${{ secrets[format('{0}_DB_PORT', github.ref_name)] }}"
|
||||
echo "::set-output name=db_name::${{ secrets[format('{0}_DB_NAME', github.ref_name)] }}"
|
||||
|
||||
- name: Set ECR credentials
|
||||
id: set_ecr_credentials
|
||||
run: |
|
||||
echo "::set-output name=ecr_uri::${{ secrets[format('{0}_ECR_URI', github.ref_name)] }}"
|
||||
|
||||
- name: Set Secrets
|
||||
id: set_api_secrets
|
||||
run: |
|
||||
echo "::set-output name=sap_predictions_bucket::${{ secrets[format('{0}_SAP_PREDICTIONS_BUCKET', github.ref_name)] }}"
|
||||
echo "::set-output name=carbon_predictions_bucket::${{ secrets[format('{0}_CARBON_PREDICTIONS_BUCKET', github.ref_name)] }}"
|
||||
echo "::set-output name=heat_predictions_bucket::${{ secrets[format('{0}_HEAT_PREDICTIONS_BUCKET', github.ref_name)] }}"
|
||||
echo "::set-output name=lighting_cost_predictions_bucket::${{ secrets[format('{0}_LIGHTING_COST_PREDICTIONS_BUCKET', github.ref_name)] }}"
|
||||
echo "::set-output name=heating_cost_predictions_bucket::${{ secrets[format('{0}_HEATING_COST_PREDICTIONS_BUCKET', github.ref_name)] }}"
|
||||
echo "::set-output name=hot_water_cost_predictions_bucket::${{ secrets[format('{0}_HOT_WATER_COST_PREDICTIONS_BUCKET', github.ref_name)] }}"
|
||||
echo "::set-output name=heating_kwh_predictions_bucket::${{ secrets[format('{0}_HEATING_KWH_PREDICTIONS_BUCKET', github.ref_name)] }}"
|
||||
echo "::set-output name=hotwater_kwh_predictions_bucket::${{ secrets[format('{0}_HOTWATER_KWH_PREDICTIONS_BUCKET', github.ref_name)] }}"
|
||||
echo "::set-output name=energy_asessments_bucket::${{ secrets[format('{0}_ENERGY_ASSESSMENTS_BUCKET', github.ref_name)] }}"
|
||||
echo "::set-output name=google_solar_api_key::${{ secrets[format('{0}_GOOGLE_SOLAR_API_KEY', github.ref_name)] }}"
|
||||
echo "::set-output name=sap_baseline_predictions_bucket::${{ secrets[format('{0}_SAP_BASELINE_PREDICTIONS_BUCKET', github.ref_name)] }}"
|
||||
echo "::set-output name=carbon_baseline_predictions_bucket::${{ secrets[format('{0}_CARBON_BASELINE_PREDICTIONS_BUCKET', github.ref_name)] }}"
|
||||
echo "::set-output name=heat_baseline_predictions_bucket::${{ secrets[format('{0}_HEAT_BASELINE_PREDICTIONS_BUCKET', github.ref_name)] }}"
|
||||
|
||||
- name: Setup Docker
|
||||
uses: docker/setup-buildx-action@v1
|
||||
|
||||
# - name: Setup Docker Buildx
|
||||
# run: |
|
||||
# docker buildx create --use
|
||||
|
||||
- name: Build Docker Image For Engine
|
||||
run: |
|
||||
docker buildx build \
|
||||
--platform linux/amd64 \
|
||||
--provenance=false \
|
||||
--output=type=docker \
|
||||
-t fastapi-lambda-image:${{ github.sha }} \
|
||||
-f backend/docker/engine.Dockerfile \
|
||||
.
|
||||
|
||||
- name: Login to ECR
|
||||
run: |
|
||||
aws ecr get-login-password --region eu-west-2 | docker login --username AWS --password-stdin ${{ steps.set_ecr_credentials.outputs.ecr_uri }}
|
||||
|
||||
- name: Tag and Push Docker Image to ECR
|
||||
run: |
|
||||
docker tag fastapi-lambda-image:${{ github.sha }} ${{ steps.set_ecr_credentials.outputs.ecr_uri }}:${{ github.sha }}
|
||||
docker push ${{ steps.set_ecr_credentials.outputs.ecr_uri }}:${{ github.sha }}
|
||||
|
||||
- name: Deploy to AWS Lambda via Serverless
|
||||
env:
|
||||
API_KEY: ${{ secrets.FASTAPI_API_KEY }}
|
||||
ENVIRONMENT: ${{ github.ref_name }}
|
||||
SECRET_KEY: ${{ secrets.NEXTAUTH_SECRET }}
|
||||
PLAN_TRIGGER_BUCKET: 'retrofit-plan-inputs-${{ github.ref_name }}'
|
||||
DATA_BUCKET: 'retrofit-data-${{ github.ref_name }}'
|
||||
PREDICTIONS_BUCKET: 'retrofit-sap-predictions-${{ github.ref_name }}'
|
||||
SAP_PREDICTIONS_BUCKET: ${{ steps.set_api_secrets.outputs.sap_predictions_bucket }}
|
||||
CARBON_PREDICTIONS_BUCKET: ${{ steps.set_api_secrets.outputs.carbon_predictions_bucket }}
|
||||
HEAT_PREDICTIONS_BUCKET: ${{ steps.set_api_secrets.outputs.heat_predictions_bucket }}
|
||||
LIGHTING_COST_PREDICTIONS_BUCKET: ${{ steps.set_api_secrets.outputs.lighting_cost_predictions_bucket }}
|
||||
HEATING_COST_PREDICTIONS_BUCKET: ${{ steps.set_api_secrets.outputs.heating_cost_predictions_bucket }}
|
||||
HOT_WATER_COST_PREDICTIONS_BUCKET: ${{ steps.set_api_secrets.outputs.hot_water_cost_predictions_bucket }}
|
||||
HEATING_KWH_PREDICTIONS_BUCKET: ${{ steps.set_api_secrets.outputs.heating_kwh_predictions_bucket }}
|
||||
HOTWATER_KWH_PREDICTIONS_BUCKET: ${{ steps.set_api_secrets.outputs.hotwater_kwh_predictions_bucket }}
|
||||
ENERGY_ASSESSMENTS_BUCKET: ${{ steps.set_api_secrets.outputs.energy_asessments_bucket }}
|
||||
GOOGLE_SOLAR_API_KEY: ${{ steps.set_api_secrets.outputs.google_solar_api_key }}
|
||||
DOMAIN_NAME: ${{ steps.set_domain.outputs.domain }}
|
||||
EPC_AUTH_TOKEN: ${{ steps.set_auth_token.outputs.auth_token }}
|
||||
OPEN_EPC_API_TOKEN: ${{ steps.set_open_epc_token.outputs.open_epc_token }}
|
||||
DB_HOST: ${{ steps.set_db_credentials.outputs.db_host }}
|
||||
DB_PORT: ${{ steps.set_db_credentials.outputs.db_port }}
|
||||
DB_NAME: ${{ steps.set_db_credentials.outputs.db_name }}
|
||||
ECR_URI: ${{ steps.set_ecr_credentials.outputs.ecr_uri }}
|
||||
GITHUB_SHA: ${{ github.sha }}
|
||||
SAP_BASELINE_PREDICTIONS_BUCKET: ${{ steps.set_api_secrets.outputs.sap_baseline_predictions_bucket }}
|
||||
CARBON_BASELINE_PREDICTIONS_BUCKET: ${{ steps.set_api_secrets.outputs.carbon_baseline_predictions_bucket }}
|
||||
HEAT_BASELINE_PREDICTIONS_BUCKET: ${{ steps.set_api_secrets.outputs.heat_baseline_predictions_bucket }}
|
||||
run: |
|
||||
# Fetch database credentials from AWS Secrets Manager
|
||||
SECRET_VALUE=$(aws secretsmanager get-secret-value --secret-id ${{ github.ref_name }}/assessment_model/db_credentials --query SecretString)
|
||||
DB_USERNAME=$(echo "$SECRET_VALUE" | jq -r '. | fromjson | .db_assessment_model_username')
|
||||
DB_PASSWORD=$(echo "$SECRET_VALUE" | jq -r '. | fromjson | .db_assessment_model_password')
|
||||
|
||||
# Set the database credentials as environment variables
|
||||
export DB_USERNAME
|
||||
export DB_PASSWORD
|
||||
|
||||
# Deploy to AWS Lambda via Serverless
|
||||
sls deploy --stage ${{ github.ref_name }} --verbose
|
||||
|
||||
- name: Smoke test deployed /health
|
||||
env:
|
||||
EXPECTED_SHA: ${{ github.sha }}
|
||||
HEALTH_URL: https://api.${{ steps.set_domain.outputs.domain }}/health
|
||||
run: |
|
||||
set -euo pipefail
|
||||
echo "Probing $HEALTH_URL"
|
||||
RESPONSE=$(curl -fsSL --max-time 30 --retry 3 --retry-delay 5 --retry-connrefused "$HEALTH_URL")
|
||||
echo "Response: $RESPONSE"
|
||||
ACTUAL_SHA=$(echo "$RESPONSE" | jq -r '.sha')
|
||||
if [[ "$ACTUAL_SHA" != "$EXPECTED_SHA" ]]; then
|
||||
echo "::error::SHA mismatch. expected=$EXPECTED_SHA actual=$ACTUAL_SHA"
|
||||
exit 1
|
||||
fi
|
||||
echo "Health check passed. sha=$ACTUAL_SHA"
|
||||
# Temporarily disabled — the FastAPI backend deploy pipeline (deploys must be run manually via `sls deploy` while disabled).
|
||||
# Commented out to cut GitHub Actions minutes; uncomment to re-enable.
|
||||
#
|
||||
# name: Fast Api Backend Deploy
|
||||
#
|
||||
# on:
|
||||
# push:
|
||||
# branches: [ dev, prod ]
|
||||
#
|
||||
# jobs:
|
||||
# deploy:
|
||||
# runs-on: ubuntu-latest
|
||||
#
|
||||
# steps:
|
||||
# - name: Checkout code
|
||||
# uses: actions/checkout@v3
|
||||
#
|
||||
# - name: Set up Python
|
||||
# uses: actions/setup-python@v2
|
||||
# with:
|
||||
# python-version: 3.10.12
|
||||
#
|
||||
# - name: Install Serverless and plugins
|
||||
# run: |
|
||||
# npm install -g serverless@^3.38.0
|
||||
# npm install -g serverless-domain-manager@^7.3.8
|
||||
# npm install -g serverless-python-requirements
|
||||
#
|
||||
# - name: Install dependencies
|
||||
# run: |
|
||||
# python -m pip install --upgrade pip
|
||||
#
|
||||
# - name: AWS credentials for dev
|
||||
# if: github.ref == 'refs/heads/dev'
|
||||
# uses: aws-actions/configure-aws-credentials@v1
|
||||
# with:
|
||||
# aws-access-key-id: ${{ secrets.DEV_AWS_ACCESS_KEY_ID }}
|
||||
# aws-secret-access-key: ${{ secrets.DEV_AWS_SECRET_ACCESS_KEY }}
|
||||
# aws-region: eu-west-2
|
||||
#
|
||||
# - name: AWS credentials for prod
|
||||
# if: github.ref == 'refs/heads/prod'
|
||||
# uses: aws-actions/configure-aws-credentials@v1
|
||||
# with:
|
||||
# aws-access-key-id: ${{ secrets.PROD_AWS_ACCESS_KEY_ID }}
|
||||
# aws-secret-access-key: ${{ secrets.PROD_AWS_SECRET_ACCESS_KEY }}
|
||||
# aws-region: eu-west-2
|
||||
#
|
||||
# - name: Set domain name
|
||||
# id: set_domain
|
||||
# run: echo "::set-output name=domain::${{ secrets[format('{0}_DOMAIN_NAME', github.ref_name)] }}"
|
||||
#
|
||||
# - name: Set EPC auth token
|
||||
# id: set_auth_token
|
||||
# run: echo "::set-output name=auth_token::${{ secrets[format('{0}_EPC_AUTH_TOKEN', github.ref_name)] }}"
|
||||
#
|
||||
# - name: Set Open EPC API token
|
||||
# id: set_open_epc_token
|
||||
# run: echo "::set-output name=open_epc_token::${{ secrets[format('{0}_OPEN_EPC_API_TOKEN', github.ref_name)] }}"
|
||||
#
|
||||
# # Store port, name and host in github secrets
|
||||
# - name: Set DB credentials
|
||||
# id: set_db_credentials
|
||||
# run: |
|
||||
# echo "::set-output name=db_host::${{ secrets[format('{0}_DB_HOST', github.ref_name)] }}"
|
||||
# echo "::set-output name=db_port::${{ secrets[format('{0}_DB_PORT', github.ref_name)] }}"
|
||||
# echo "::set-output name=db_name::${{ secrets[format('{0}_DB_NAME', github.ref_name)] }}"
|
||||
#
|
||||
# - name: Set ECR credentials
|
||||
# id: set_ecr_credentials
|
||||
# run: |
|
||||
# echo "::set-output name=ecr_uri::${{ secrets[format('{0}_ECR_URI', github.ref_name)] }}"
|
||||
#
|
||||
# - name: Set Secrets
|
||||
# id: set_api_secrets
|
||||
# run: |
|
||||
# echo "::set-output name=sap_predictions_bucket::${{ secrets[format('{0}_SAP_PREDICTIONS_BUCKET', github.ref_name)] }}"
|
||||
# echo "::set-output name=carbon_predictions_bucket::${{ secrets[format('{0}_CARBON_PREDICTIONS_BUCKET', github.ref_name)] }}"
|
||||
# echo "::set-output name=heat_predictions_bucket::${{ secrets[format('{0}_HEAT_PREDICTIONS_BUCKET', github.ref_name)] }}"
|
||||
# echo "::set-output name=lighting_cost_predictions_bucket::${{ secrets[format('{0}_LIGHTING_COST_PREDICTIONS_BUCKET', github.ref_name)] }}"
|
||||
# echo "::set-output name=heating_cost_predictions_bucket::${{ secrets[format('{0}_HEATING_COST_PREDICTIONS_BUCKET', github.ref_name)] }}"
|
||||
# echo "::set-output name=hot_water_cost_predictions_bucket::${{ secrets[format('{0}_HOT_WATER_COST_PREDICTIONS_BUCKET', github.ref_name)] }}"
|
||||
# echo "::set-output name=heating_kwh_predictions_bucket::${{ secrets[format('{0}_HEATING_KWH_PREDICTIONS_BUCKET', github.ref_name)] }}"
|
||||
# echo "::set-output name=hotwater_kwh_predictions_bucket::${{ secrets[format('{0}_HOTWATER_KWH_PREDICTIONS_BUCKET', github.ref_name)] }}"
|
||||
# echo "::set-output name=energy_asessments_bucket::${{ secrets[format('{0}_ENERGY_ASSESSMENTS_BUCKET', github.ref_name)] }}"
|
||||
# echo "::set-output name=google_solar_api_key::${{ secrets[format('{0}_GOOGLE_SOLAR_API_KEY', github.ref_name)] }}"
|
||||
# echo "::set-output name=sap_baseline_predictions_bucket::${{ secrets[format('{0}_SAP_BASELINE_PREDICTIONS_BUCKET', github.ref_name)] }}"
|
||||
# echo "::set-output name=carbon_baseline_predictions_bucket::${{ secrets[format('{0}_CARBON_BASELINE_PREDICTIONS_BUCKET', github.ref_name)] }}"
|
||||
# echo "::set-output name=heat_baseline_predictions_bucket::${{ secrets[format('{0}_HEAT_BASELINE_PREDICTIONS_BUCKET', github.ref_name)] }}"
|
||||
#
|
||||
# - name: Setup Docker
|
||||
# uses: docker/setup-buildx-action@v1
|
||||
#
|
||||
# # - name: Setup Docker Buildx
|
||||
# # run: |
|
||||
# # docker buildx create --use
|
||||
#
|
||||
# - name: Build Docker Image For Engine
|
||||
# run: |
|
||||
# docker buildx build \
|
||||
# --platform linux/amd64 \
|
||||
# --provenance=false \
|
||||
# --output=type=docker \
|
||||
# -t fastapi-lambda-image:${{ github.sha }} \
|
||||
# -f backend/docker/engine.Dockerfile \
|
||||
# .
|
||||
#
|
||||
# - name: Login to ECR
|
||||
# run: |
|
||||
# aws ecr get-login-password --region eu-west-2 | docker login --username AWS --password-stdin ${{ steps.set_ecr_credentials.outputs.ecr_uri }}
|
||||
#
|
||||
# - name: Tag and Push Docker Image to ECR
|
||||
# run: |
|
||||
# docker tag fastapi-lambda-image:${{ github.sha }} ${{ steps.set_ecr_credentials.outputs.ecr_uri }}:${{ github.sha }}
|
||||
# docker push ${{ steps.set_ecr_credentials.outputs.ecr_uri }}:${{ github.sha }}
|
||||
#
|
||||
# - name: Deploy to AWS Lambda via Serverless
|
||||
# env:
|
||||
# API_KEY: ${{ secrets.FASTAPI_API_KEY }}
|
||||
# ENVIRONMENT: ${{ github.ref_name }}
|
||||
# SECRET_KEY: ${{ secrets.NEXTAUTH_SECRET }}
|
||||
# PLAN_TRIGGER_BUCKET: 'retrofit-plan-inputs-${{ github.ref_name }}'
|
||||
# DATA_BUCKET: 'retrofit-data-${{ github.ref_name }}'
|
||||
# PREDICTIONS_BUCKET: 'retrofit-sap-predictions-${{ github.ref_name }}'
|
||||
# SAP_PREDICTIONS_BUCKET: ${{ steps.set_api_secrets.outputs.sap_predictions_bucket }}
|
||||
# CARBON_PREDICTIONS_BUCKET: ${{ steps.set_api_secrets.outputs.carbon_predictions_bucket }}
|
||||
# HEAT_PREDICTIONS_BUCKET: ${{ steps.set_api_secrets.outputs.heat_predictions_bucket }}
|
||||
# LIGHTING_COST_PREDICTIONS_BUCKET: ${{ steps.set_api_secrets.outputs.lighting_cost_predictions_bucket }}
|
||||
# HEATING_COST_PREDICTIONS_BUCKET: ${{ steps.set_api_secrets.outputs.heating_cost_predictions_bucket }}
|
||||
# HOT_WATER_COST_PREDICTIONS_BUCKET: ${{ steps.set_api_secrets.outputs.hot_water_cost_predictions_bucket }}
|
||||
# HEATING_KWH_PREDICTIONS_BUCKET: ${{ steps.set_api_secrets.outputs.heating_kwh_predictions_bucket }}
|
||||
# HOTWATER_KWH_PREDICTIONS_BUCKET: ${{ steps.set_api_secrets.outputs.hotwater_kwh_predictions_bucket }}
|
||||
# ENERGY_ASSESSMENTS_BUCKET: ${{ steps.set_api_secrets.outputs.energy_asessments_bucket }}
|
||||
# GOOGLE_SOLAR_API_KEY: ${{ steps.set_api_secrets.outputs.google_solar_api_key }}
|
||||
# DOMAIN_NAME: ${{ steps.set_domain.outputs.domain }}
|
||||
# EPC_AUTH_TOKEN: ${{ steps.set_auth_token.outputs.auth_token }}
|
||||
# OPEN_EPC_API_TOKEN: ${{ steps.set_open_epc_token.outputs.open_epc_token }}
|
||||
# DB_HOST: ${{ steps.set_db_credentials.outputs.db_host }}
|
||||
# DB_PORT: ${{ steps.set_db_credentials.outputs.db_port }}
|
||||
# DB_NAME: ${{ steps.set_db_credentials.outputs.db_name }}
|
||||
# ECR_URI: ${{ steps.set_ecr_credentials.outputs.ecr_uri }}
|
||||
# GITHUB_SHA: ${{ github.sha }}
|
||||
# SAP_BASELINE_PREDICTIONS_BUCKET: ${{ steps.set_api_secrets.outputs.sap_baseline_predictions_bucket }}
|
||||
# CARBON_BASELINE_PREDICTIONS_BUCKET: ${{ steps.set_api_secrets.outputs.carbon_baseline_predictions_bucket }}
|
||||
# HEAT_BASELINE_PREDICTIONS_BUCKET: ${{ steps.set_api_secrets.outputs.heat_baseline_predictions_bucket }}
|
||||
# run: |
|
||||
# # Fetch database credentials from AWS Secrets Manager
|
||||
# SECRET_VALUE=$(aws secretsmanager get-secret-value --secret-id ${{ github.ref_name }}/assessment_model/db_credentials --query SecretString)
|
||||
# DB_USERNAME=$(echo "$SECRET_VALUE" | jq -r '. | fromjson | .db_assessment_model_username')
|
||||
# DB_PASSWORD=$(echo "$SECRET_VALUE" | jq -r '. | fromjson | .db_assessment_model_password')
|
||||
#
|
||||
# # Set the database credentials as environment variables
|
||||
# export DB_USERNAME
|
||||
# export DB_PASSWORD
|
||||
#
|
||||
# # Deploy to AWS Lambda via Serverless
|
||||
# sls deploy --stage ${{ github.ref_name }} --verbose
|
||||
#
|
||||
# - name: Smoke test deployed /health
|
||||
# env:
|
||||
# EXPECTED_SHA: ${{ github.sha }}
|
||||
# HEALTH_URL: https://api.${{ steps.set_domain.outputs.domain }}/health
|
||||
# run: |
|
||||
# set -euo pipefail
|
||||
# echo "Probing $HEALTH_URL"
|
||||
# RESPONSE=$(curl -fsSL --max-time 30 --retry 3 --retry-delay 5 --retry-connrefused "$HEALTH_URL")
|
||||
# echo "Response: $RESPONSE"
|
||||
# ACTUAL_SHA=$(echo "$RESPONSE" | jq -r '.sha')
|
||||
# if [[ "$ACTUAL_SHA" != "$EXPECTED_SHA" ]]; then
|
||||
# echo "::error::SHA mismatch. expected=$EXPECTED_SHA actual=$ACTUAL_SHA"
|
||||
# exit 1
|
||||
# fi
|
||||
# echo "Health check passed. sha=$ACTUAL_SHA"
|
||||
|
|
|
|||
73
.github/workflows/integration_tests.yml
vendored
73
.github/workflows/integration_tests.yml
vendored
|
|
@ -1,35 +1,38 @@
|
|||
name: Rebaselining Integration Test
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches:
|
||||
- main
|
||||
|
||||
jobs:
|
||||
rebaselining-integration-test:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python 3.11
|
||||
uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: '3.11'
|
||||
|
||||
- name: Install tox via Makefile
|
||||
run: |
|
||||
make setup
|
||||
|
||||
- name: Configure AWS credentials for dev
|
||||
uses: aws-actions/configure-aws-credentials@v1
|
||||
with:
|
||||
aws-access-key-id: ${{ secrets.DEV_AWS_ACCESS_KEY_ID }}
|
||||
aws-secret-access-key: ${{ secrets.DEV_AWS_SECRET_ACCESS_KEY }}
|
||||
aws-region: eu-west-2
|
||||
|
||||
- name: Run only rebaselining integration test
|
||||
env:
|
||||
EPC_AUTH_TOKEN: ${{ secrets.DEV_EPC_AUTH_TOKEN }}
|
||||
run: |
|
||||
make test ARGS="-m integration"
|
||||
# Temporarily disabled — the rebaselining integration suite.
|
||||
# Commented out to cut GitHub Actions minutes; uncomment to re-enable.
|
||||
#
|
||||
# name: Rebaselining Integration Test
|
||||
#
|
||||
# on:
|
||||
# pull_request:
|
||||
# branches:
|
||||
# - main
|
||||
#
|
||||
# jobs:
|
||||
# rebaselining-integration-test:
|
||||
# runs-on: ubuntu-latest
|
||||
# steps:
|
||||
# - name: Checkout code
|
||||
# uses: actions/checkout@v4
|
||||
#
|
||||
# - name: Set up Python 3.11
|
||||
# uses: actions/setup-python@v4
|
||||
# with:
|
||||
# python-version: '3.11'
|
||||
#
|
||||
# - name: Install tox via Makefile
|
||||
# run: |
|
||||
# make setup
|
||||
#
|
||||
# - name: Configure AWS credentials for dev
|
||||
# uses: aws-actions/configure-aws-credentials@v1
|
||||
# with:
|
||||
# aws-access-key-id: ${{ secrets.DEV_AWS_ACCESS_KEY_ID }}
|
||||
# aws-secret-access-key: ${{ secrets.DEV_AWS_SECRET_ACCESS_KEY }}
|
||||
# aws-region: eu-west-2
|
||||
#
|
||||
# - name: Run only rebaselining integration test
|
||||
# env:
|
||||
# EPC_AUTH_TOKEN: ${{ secrets.DEV_EPC_AUTH_TOKEN }}
|
||||
# run: |
|
||||
# make test ARGS="-m integration"
|
||||
|
|
|
|||
37
.github/workflows/protect_releases.yml
vendored
37
.github/workflows/protect_releases.yml
vendored
|
|
@ -1,17 +1,20 @@
|
|||
name: Restrict PR source
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches:
|
||||
- dev
|
||||
|
||||
jobs:
|
||||
check-source-branch:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Fail if PR is not from main
|
||||
run: |
|
||||
if [[ "${{ github.head_ref }}" != "main" ]]; then
|
||||
echo "Only PRs from main are allowed into dev"
|
||||
exit 1
|
||||
fi
|
||||
# Temporarily disabled — the main→dev PR-source guardrail.
|
||||
# Commented out to cut GitHub Actions minutes; uncomment to re-enable.
|
||||
#
|
||||
# name: Restrict PR source
|
||||
#
|
||||
# on:
|
||||
# pull_request:
|
||||
# branches:
|
||||
# - dev
|
||||
#
|
||||
# jobs:
|
||||
# check-source-branch:
|
||||
# runs-on: ubuntu-latest
|
||||
# steps:
|
||||
# - name: Fail if PR is not from main
|
||||
# run: |
|
||||
# if [[ "${{ github.head_ref }}" != "main" ]]; then
|
||||
# echo "Only PRs from main are allowed into dev"
|
||||
# exit 1
|
||||
# fi
|
||||
|
|
|
|||
137
.github/workflows/unit_tests.yml
vendored
137
.github/workflows/unit_tests.yml
vendored
|
|
@ -1,67 +1,70 @@
|
|||
name: Run unit tests
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches:
|
||||
- "**"
|
||||
|
||||
|
||||
jobs:
|
||||
test-docker:
|
||||
name: Tests (Docker)
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:15
|
||||
env:
|
||||
POSTGRES_USER: test
|
||||
POSTGRES_PASSWORD: test
|
||||
POSTGRES_DB: test
|
||||
ports:
|
||||
- 5432:5432
|
||||
options: >-
|
||||
--health-cmd pg_isready
|
||||
--health-interval 10s
|
||||
--health-timeout 5s
|
||||
--health-retries 5
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Build test image
|
||||
run: docker build -f Dockerfile.test -t model-test .
|
||||
|
||||
- name: Initialise database schema
|
||||
run: |
|
||||
docker run --rm \
|
||||
--network host \
|
||||
-e DB_HOST=localhost \
|
||||
-e DB_NAME=test \
|
||||
-e DB_USERNAME=test \
|
||||
-e DB_PASSWORD=test \
|
||||
-e DB_PORT=5432 \
|
||||
model-test python scripts/init_db.py
|
||||
|
||||
- name: Run tests
|
||||
run: |
|
||||
docker run --rm \
|
||||
--network host \
|
||||
-e EPC_AUTH_TOKEN=${{ secrets.DEV_EPC_AUTH_TOKEN }} \
|
||||
-e OPEN_EPC_API_TOKEN=${{ secrets.DEV_OPEN_EPC_API_TOKEN }} \
|
||||
-e HUBSPOT_API_KEY=${{ secrets.HUBSPOT_API_KEY }} \
|
||||
-e AWS_ACCESS_KEY_ID=${{ secrets.DEV_AWS_ACCESS_KEY_ID }} \
|
||||
-e AWS_SECRET_ACCESS_KEY=${{ secrets.DEV_AWS_SECRET_ACCESS_KEY }} \
|
||||
-e AWS_DEFAULT_REGION=${{ secrets.DEV_AWS_REGION }} \
|
||||
-e DB_HOST=localhost \
|
||||
-e DB_NAME=test \
|
||||
-e DB_USERNAME=test \
|
||||
-e DB_PASSWORD=test \
|
||||
-e DB_PORT=5432 \
|
||||
model-test pytest -vv -m 'not integration'
|
||||
|
||||
# The DDD rewrite (tests/) runs in its own workflow (ddd_tests.yml): its
|
||||
# SQLModel table classes map to the same physical tables as the legacy
|
||||
# backend models and share the one global SQLModel.metadata, so the two
|
||||
# suites cannot be imported into the same pytest process.
|
||||
# Temporarily disabled — this Docker-based suite was consuming too many GitHub
|
||||
# Actions minutes. Uncomment to re-enable.
|
||||
#
|
||||
# name: Run unit tests
|
||||
#
|
||||
# on:
|
||||
# pull_request:
|
||||
# branches:
|
||||
# - "**"
|
||||
#
|
||||
#
|
||||
# jobs:
|
||||
# test-docker:
|
||||
# name: Tests (Docker)
|
||||
# runs-on: ubuntu-latest
|
||||
#
|
||||
# services:
|
||||
# postgres:
|
||||
# image: postgres:15
|
||||
# env:
|
||||
# POSTGRES_USER: test
|
||||
# POSTGRES_PASSWORD: test
|
||||
# POSTGRES_DB: test
|
||||
# ports:
|
||||
# - 5432:5432
|
||||
# options: >-
|
||||
# --health-cmd pg_isready
|
||||
# --health-interval 10s
|
||||
# --health-timeout 5s
|
||||
# --health-retries 5
|
||||
#
|
||||
# steps:
|
||||
# - name: Checkout code
|
||||
# uses: actions/checkout@v4
|
||||
#
|
||||
# - name: Build test image
|
||||
# run: docker build -f Dockerfile.test -t model-test .
|
||||
#
|
||||
# - name: Initialise database schema
|
||||
# run: |
|
||||
# docker run --rm \
|
||||
# --network host \
|
||||
# -e DB_HOST=localhost \
|
||||
# -e DB_NAME=test \
|
||||
# -e DB_USERNAME=test \
|
||||
# -e DB_PASSWORD=test \
|
||||
# -e DB_PORT=5432 \
|
||||
# model-test python scripts/init_db.py
|
||||
#
|
||||
# - name: Run tests
|
||||
# run: |
|
||||
# docker run --rm \
|
||||
# --network host \
|
||||
# -e EPC_AUTH_TOKEN=${{ secrets.DEV_EPC_AUTH_TOKEN }} \
|
||||
# -e OPEN_EPC_API_TOKEN=${{ secrets.DEV_OPEN_EPC_API_TOKEN }} \
|
||||
# -e HUBSPOT_API_KEY=${{ secrets.HUBSPOT_API_KEY }} \
|
||||
# -e AWS_ACCESS_KEY_ID=${{ secrets.DEV_AWS_ACCESS_KEY_ID }} \
|
||||
# -e AWS_SECRET_ACCESS_KEY=${{ secrets.DEV_AWS_SECRET_ACCESS_KEY }} \
|
||||
# -e AWS_DEFAULT_REGION=${{ secrets.DEV_AWS_REGION }} \
|
||||
# -e DB_HOST=localhost \
|
||||
# -e DB_NAME=test \
|
||||
# -e DB_USERNAME=test \
|
||||
# -e DB_PASSWORD=test \
|
||||
# -e DB_PORT=5432 \
|
||||
# model-test pytest -vv -m 'not integration'
|
||||
#
|
||||
# # The DDD rewrite (tests/) runs in its own workflow (ddd_tests.yml): its
|
||||
# # SQLModel table classes map to the same physical tables as the legacy
|
||||
# # backend models and share the one global SQLModel.metadata, so the two
|
||||
# # suites cannot be imported into the same pytest process.
|
||||
|
|
|
|||
|
|
@ -25,6 +25,7 @@ from __future__ import annotations
|
|||
|
||||
import dataclasses
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
from collections.abc import Callable
|
||||
from typing import Any, Optional, cast
|
||||
|
|
@ -94,6 +95,7 @@ from repositories.property.property_overrides_postgres_reader import (
|
|||
from repositories.scenario.scenario_postgres_repository import (
|
||||
ScenarioPostgresRepository,
|
||||
)
|
||||
from repositories.solar.solar_postgres_repository import SolarPostgresRepository
|
||||
from utilities.aws_lambda.task_handler import task_handler
|
||||
from utilities.logger import setup_logger
|
||||
|
||||
|
|
@ -288,8 +290,9 @@ def handler(body: dict[str, Any], context: Any) -> Optional[dict[str, Any]]:
|
|||
try:
|
||||
scenario = ScenarioPostgresRepository(read_session).get_many([scenario_id])[0]
|
||||
products = catalogue_with_off_catalogue_overrides(read_session)
|
||||
solar_reader = SolarPostgresRepository(read_session)
|
||||
|
||||
errors: list[int] = []
|
||||
failures: list[dict[str, Any]] = []
|
||||
|
||||
for property_id in property_ids:
|
||||
try:
|
||||
|
|
@ -356,9 +359,18 @@ def handler(body: dict[str, Any], context: Any) -> Optional[dict[str, Any]]:
|
|||
landlord_overrides=overrides,
|
||||
).effective_epc
|
||||
|
||||
solar_insights: Optional[dict[str, Any]] = (
|
||||
None if no_solar else _solar_insights_for(solar_client, spatial)
|
||||
)
|
||||
# Read-before-fetch: the Google Solar call is paid, so skip it
|
||||
# when this UPRN's insights are already persisted. Only a cache
|
||||
# miss hits Google — re-runs cost nothing for solar.
|
||||
solar_insights: Optional[dict[str, Any]]
|
||||
solar_was_fetched = False
|
||||
if no_solar:
|
||||
solar_insights = None
|
||||
else:
|
||||
solar_insights = solar_reader.get(uprn)
|
||||
if solar_insights is None:
|
||||
solar_insights = _solar_insights_for(solar_client, spatial)
|
||||
solar_was_fetched = solar_insights is not None
|
||||
|
||||
# All Measure Types are considered: the off-catalogue overlay
|
||||
# (catalogue_with_off_catalogue_overrides) prices the measures the
|
||||
|
|
@ -405,7 +417,8 @@ def handler(body: dict[str, Any], context: Any) -> Optional[dict[str, Any]]:
|
|||
if spatial is not None:
|
||||
uow.spatial.save(uprn, spatial)
|
||||
if (
|
||||
solar_insights is not None
|
||||
solar_was_fetched
|
||||
and solar_insights is not None
|
||||
and spatial is not None
|
||||
and spatial.coordinates is not None
|
||||
):
|
||||
|
|
@ -437,10 +450,18 @@ def handler(body: dict[str, Any], context: Any) -> Optional[dict[str, Any]]:
|
|||
|
||||
except Exception as error: # noqa: BLE001
|
||||
logger.error(
|
||||
f"property={property_id}: {type(error).__name__}: {error}",
|
||||
f"property={property_id} uprn={uprns.get(property_id)}: "
|
||||
f"{type(error).__name__}: {error}",
|
||||
exc_info=True,
|
||||
)
|
||||
errors.append(property_id)
|
||||
failures.append(
|
||||
{
|
||||
"property_id": property_id,
|
||||
"uprn": uprns.get(property_id),
|
||||
"error_type": type(error).__name__,
|
||||
"error": str(error),
|
||||
}
|
||||
)
|
||||
|
||||
# Cohort certs the mapper could not consume were skipped (not aborted on)
|
||||
# so prediction could proceed; surface them — with cert numbers — in the
|
||||
|
|
@ -455,12 +476,29 @@ def handler(body: dict[str, Any], context: Any) -> Optional[dict[str, Any]]:
|
|||
f"{[s['certificate_number'] for s in skipped_certs]}"
|
||||
)
|
||||
|
||||
if errors:
|
||||
message = f"failed property_ids: {errors}"
|
||||
# A property that errored AND a cohort cert the mapper could not consume
|
||||
# are both surfaced as failures, so the subtask is marked failed and
|
||||
# shows up for debugging. The whole batch has already run by this point —
|
||||
# every property that could be modelled was written to DB above — so
|
||||
# failing here flags the run without discarding the work done so far.
|
||||
if failures or skipped_certs:
|
||||
parts: list[str] = []
|
||||
if failures:
|
||||
failed_ids = [f["property_id"] for f in failures]
|
||||
# Persisted verbatim into the subtask's outputs.error (via
|
||||
# SubTask.fail): include each property's error type + message,
|
||||
# not just the IDs, so failed runs are diagnosable without
|
||||
# cross-referencing CloudWatch.
|
||||
parts.append(
|
||||
f"failed property_ids: {failed_ids}; "
|
||||
f"details: {json.dumps(failures)}"
|
||||
)
|
||||
if skipped_certs:
|
||||
message += f"; skipped_unmappable_cohort_certs: {skipped_certs}"
|
||||
raise RuntimeError(message)
|
||||
parts.append(
|
||||
f"skipped_unmappable_cohort_certs: {json.dumps(skipped_certs)}"
|
||||
)
|
||||
raise RuntimeError("; ".join(parts))
|
||||
|
||||
return {"skipped_unmappable_cohort_certs": skipped_certs} if skipped_certs else None
|
||||
return None
|
||||
finally:
|
||||
read_session.close()
|
||||
|
|
|
|||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
|
|
@ -0,0 +1,281 @@
|
|||
{
|
||||
"uprn": 10070004512,
|
||||
"roofs": [
|
||||
{
|
||||
"description": "(another dwelling above)",
|
||||
"energy_efficiency_rating": 0,
|
||||
"environmental_efficiency_rating": 0
|
||||
}
|
||||
],
|
||||
"walls": [
|
||||
{
|
||||
"description": "Cavity wall, filled cavity",
|
||||
"energy_efficiency_rating": 4,
|
||||
"environmental_efficiency_rating": 4
|
||||
}
|
||||
],
|
||||
"floors": [
|
||||
{
|
||||
"description": "To external air, no insulation (assumed)",
|
||||
"energy_efficiency_rating": 0,
|
||||
"environmental_efficiency_rating": 0
|
||||
}
|
||||
],
|
||||
"status": "entered",
|
||||
"windows": [
|
||||
{
|
||||
"description": "Fully double glazed",
|
||||
"energy_efficiency_rating": 3,
|
||||
"environmental_efficiency_rating": 3
|
||||
}
|
||||
],
|
||||
"lighting": {
|
||||
"description": "Low energy lighting in all fixed outlets",
|
||||
"energy_efficiency_rating": 5,
|
||||
"environmental_efficiency_rating": 5
|
||||
},
|
||||
"postcode": "BR1 4QF",
|
||||
"hot_water": {
|
||||
"description": "Electric immersion, off-peak",
|
||||
"energy_efficiency_rating": 3,
|
||||
"environmental_efficiency_rating": 1
|
||||
},
|
||||
"post_town": "BROMLEY",
|
||||
"created_at": "2012-04-20 10:59:22.000000",
|
||||
"door_count": 1,
|
||||
"glazed_area": 1,
|
||||
"region_code": 14,
|
||||
"report_type": 2,
|
||||
"sap_heating": {
|
||||
"wwhrs": {
|
||||
"rooms_with_bath_and_or_shower": 1,
|
||||
"rooms_with_mixer_shower_no_bath": 0,
|
||||
"rooms_with_bath_and_mixer_shower": 0
|
||||
},
|
||||
"cylinder_size": 1,
|
||||
"water_heating_code": 903,
|
||||
"water_heating_fuel": 29,
|
||||
"main_heating_details": [
|
||||
{
|
||||
"has_fghrs": "N",
|
||||
"main_fuel_type": 29,
|
||||
"heat_emitter_type": 0,
|
||||
"main_heating_number": 1,
|
||||
"main_heating_control": 2401,
|
||||
"main_heating_category": 7,
|
||||
"main_heating_fraction": 1,
|
||||
"sap_main_heating_code": 402,
|
||||
"main_heating_data_source": 2
|
||||
}
|
||||
],
|
||||
"immersion_heating_type": 1,
|
||||
"has_fixed_air_conditioning": "false"
|
||||
},
|
||||
"sap_version": 9.91,
|
||||
"schema_type": "SAP-Schema-16.0",
|
||||
"uprn_source": "Energy Assessor",
|
||||
"country_code": "EAW",
|
||||
"main_heating": [
|
||||
{
|
||||
"description": "Electric storage heaters",
|
||||
"energy_efficiency_rating": 3,
|
||||
"environmental_efficiency_rating": 1
|
||||
}
|
||||
],
|
||||
"dwelling_type": "Ground-floor flat",
|
||||
"language_code": 1,
|
||||
"property_type": 2,
|
||||
"address_line_1": "54a, Boyland Road",
|
||||
"schema_version": "LIG-16.0",
|
||||
"assessment_type": "RdSAP",
|
||||
"completion_date": "2012-04-20",
|
||||
"inspection_date": "2012-04-20",
|
||||
"extensions_count": 0,
|
||||
"measurement_type": 1,
|
||||
"sap_flat_details": {
|
||||
"level": 1,
|
||||
"top_storey": "N",
|
||||
"flat_location": 0,
|
||||
"heat_loss_corridor": 0
|
||||
},
|
||||
"total_floor_area": 33,
|
||||
"transaction_type": 3,
|
||||
"conservatory_type": 1,
|
||||
"heated_room_count": 2,
|
||||
"registration_date": "2012-04-20",
|
||||
"restricted_access": 1,
|
||||
"sap_energy_source": {
|
||||
"main_gas": "N",
|
||||
"meter_type": 1,
|
||||
"photovoltaic_supply": {
|
||||
"percent_roof_area": 0
|
||||
},
|
||||
"wind_turbines_count": 0,
|
||||
"wind_turbines_terrain_type": 1
|
||||
},
|
||||
"secondary_heating": {
|
||||
"description": "Portable electric heaters (assumed)",
|
||||
"energy_efficiency_rating": 0,
|
||||
"environmental_efficiency_rating": 0
|
||||
},
|
||||
"sap_building_parts": [
|
||||
{
|
||||
"identifier": "Main Dwelling",
|
||||
"wall_dry_lined": "N",
|
||||
"floor_heat_loss": 1,
|
||||
"roof_construction": 3,
|
||||
"wall_construction": 4,
|
||||
"building_part_number": 1,
|
||||
"sap_floor_dimensions": [
|
||||
{
|
||||
"floor": 0,
|
||||
"room_height": 2.42,
|
||||
"floor_insulation": 1,
|
||||
"total_floor_area": 33.24,
|
||||
"floor_construction": 0,
|
||||
"heat_loss_perimeter": 10.39
|
||||
}
|
||||
],
|
||||
"wall_insulation_type": 2,
|
||||
"construction_age_band": "B",
|
||||
"wall_thickness_measured": "N",
|
||||
"roof_insulation_location": "ND",
|
||||
"roof_insulation_thickness": "ND"
|
||||
}
|
||||
],
|
||||
"low_energy_lighting": 100,
|
||||
"solar_water_heating": "N",
|
||||
"bedf_revision_number": 321,
|
||||
"habitable_room_count": 2,
|
||||
"heating_cost_current": {
|
||||
"value": 303,
|
||||
"currency": "GBP"
|
||||
},
|
||||
"insulated_door_count": 0,
|
||||
"co2_emissions_current": {
|
||||
"value": 2.9,
|
||||
"quantity": "tonnes per year"
|
||||
},
|
||||
"energy_rating_average": 60,
|
||||
"energy_rating_current": 66,
|
||||
"lighting_cost_current": {
|
||||
"value": 23,
|
||||
"currency": "GBP"
|
||||
},
|
||||
"main_heating_controls": [
|
||||
{
|
||||
"description": "Manual charge control",
|
||||
"energy_efficiency_rating": 2,
|
||||
"environmental_efficiency_rating": 2
|
||||
}
|
||||
],
|
||||
"multiple_glazing_type": 3,
|
||||
"open_fireplaces_count": 0,
|
||||
"has_hot_water_cylinder": "false",
|
||||
"heating_cost_potential": {
|
||||
"value": 190,
|
||||
"currency": "GBP"
|
||||
},
|
||||
"hot_water_cost_current": {
|
||||
"value": 106,
|
||||
"currency": "GBP"
|
||||
},
|
||||
"mechanical_ventilation": 0,
|
||||
"percent_draughtproofed": 100,
|
||||
"suggested_improvements": [
|
||||
{
|
||||
"sequence": 1,
|
||||
"typical_saving": {
|
||||
"value": 86,
|
||||
"currency": "GBP"
|
||||
},
|
||||
"indicative_cost": "\u00a3800 - \u00a31,200",
|
||||
"improvement_type": "W",
|
||||
"improvement_details": {
|
||||
"improvement_number": 47
|
||||
},
|
||||
"improvement_category": 5,
|
||||
"energy_performance_rating": 73,
|
||||
"environmental_impact_rating": 58
|
||||
},
|
||||
{
|
||||
"sequence": 2,
|
||||
"typical_saving": {
|
||||
"value": 27,
|
||||
"currency": "GBP"
|
||||
},
|
||||
"indicative_cost": "\u00a3600 - \u00a3800",
|
||||
"improvement_type": "L",
|
||||
"improvement_details": {
|
||||
"improvement_number": 25
|
||||
},
|
||||
"improvement_category": 5,
|
||||
"energy_performance_rating": 75,
|
||||
"environmental_impact_rating": 60
|
||||
}
|
||||
],
|
||||
"co2_emissions_potential": {
|
||||
"value": 2.1,
|
||||
"quantity": "tonnes per year"
|
||||
},
|
||||
"energy_rating_potential": 75,
|
||||
"lighting_cost_potential": {
|
||||
"value": 23,
|
||||
"currency": "GBP"
|
||||
},
|
||||
"alternative_improvements": [
|
||||
{
|
||||
"sequence": 1,
|
||||
"typical_saving": {
|
||||
"value": 24,
|
||||
"currency": "GBP"
|
||||
},
|
||||
"improvement_type": "J2",
|
||||
"improvement_details": {
|
||||
"improvement_number": 54
|
||||
},
|
||||
"improvement_category": 6,
|
||||
"energy_performance_rating": 76,
|
||||
"environmental_impact_rating": 93
|
||||
},
|
||||
{
|
||||
"sequence": 2,
|
||||
"typical_saving": {
|
||||
"value": 84,
|
||||
"currency": "GBP"
|
||||
},
|
||||
"improvement_type": "Z1",
|
||||
"improvement_details": {
|
||||
"improvement_number": 51
|
||||
},
|
||||
"improvement_category": 6,
|
||||
"energy_performance_rating": 80,
|
||||
"environmental_impact_rating": 81
|
||||
}
|
||||
],
|
||||
"hot_water_cost_potential": {
|
||||
"value": 106,
|
||||
"currency": "GBP"
|
||||
},
|
||||
"renewable_heat_incentive": {
|
||||
"water_heating": 1434,
|
||||
"space_heating_existing_dwelling": 4064
|
||||
},
|
||||
"seller_commission_report": "Y",
|
||||
"energy_consumption_current": 497,
|
||||
"has_fixed_air_conditioning": "false",
|
||||
"multiple_glazed_proportion": 100,
|
||||
"calculation_software_version": 4.1,
|
||||
"energy_consumption_potential": 365,
|
||||
"environmental_impact_current": 47,
|
||||
"fixed_lighting_outlets_count": 5,
|
||||
"current_energy_efficiency_band": "D",
|
||||
"environmental_impact_potential": 60,
|
||||
"has_heated_separate_conservatory": "false",
|
||||
"potential_energy_efficiency_band": "C",
|
||||
"co2_emissions_current_per_floor_area": {
|
||||
"value": 88,
|
||||
"quantity": "kg/m2 per year"
|
||||
},
|
||||
"low_energy_fixed_lighting_outlets_count": 5
|
||||
}
|
||||
|
|
@ -2150,7 +2150,10 @@ class EpcPropertyDataMapper:
|
|||
door_count=schema.door_count,
|
||||
habitable_rooms_count=schema.habitable_room_count,
|
||||
heated_rooms_count=schema.heated_room_count,
|
||||
wet_rooms_count=schema.wet_rooms_count,
|
||||
# 21.0.1 lodges wet_rooms_count as Optional; None violates the
|
||||
# epc_property NOT-NULL column (and the calc's `> 0` check). Coalesce
|
||||
# to 0 like every other mapper (RdSAP "not lodged" → calc minimum 1).
|
||||
wet_rooms_count=schema.wet_rooms_count or 0,
|
||||
extensions_count=schema.extensions_count,
|
||||
open_chimneys_count=schema.open_chimneys_count,
|
||||
insulated_door_count=schema.insulated_door_count,
|
||||
|
|
|
|||
|
|
@ -379,6 +379,22 @@ class TestFromRdSapSchema21_0_1:
|
|||
|
||||
# --- general ---
|
||||
|
||||
def test_omitted_wet_rooms_count_defaults_to_zero_not_none(self) -> None:
|
||||
# 21.0.1 lodges wet_rooms_count as Optional, so a cert that omits it
|
||||
# mapped to EpcPropertyData.wet_rooms_count=None. That None then
|
||||
# violated the epc_property.wet_rooms_count NOT-NULL constraint when a
|
||||
# predicted EPC (which deep-copies a comparable template) was persisted,
|
||||
# and would crash the calc's `wet_rooms_count > 0` check. 37 modelling_e2e
|
||||
# properties failed on the 2026-06-23 run for this reason. Default to 0,
|
||||
# matching every other mapper (RdSAP 0 → calc's Table-?? minimum 1 fan).
|
||||
data = load("21_0_1.json")
|
||||
data.pop("wet_rooms_count", None)
|
||||
schema = from_dict(RdSapSchema21_0_1, data)
|
||||
|
||||
result = EpcPropertyDataMapper.from_rdsap_schema_21_0_1(schema)
|
||||
|
||||
assert result.wet_rooms_count == 0
|
||||
|
||||
def test_uprn(self, result: EpcPropertyData) -> None:
|
||||
assert result.uprn == 12457
|
||||
|
||||
|
|
|
|||
|
|
@ -21,7 +21,7 @@ scoring (ADR-0016).
|
|||
from dataclasses import dataclass
|
||||
from typing import Final, Optional
|
||||
|
||||
from datatypes.epc.domain.epc_property_data import EpcPropertyData
|
||||
from datatypes.epc.domain.epc_property_data import EpcPropertyData, SapWindow
|
||||
from domain.geospatial.planning_restrictions import PlanningRestrictions
|
||||
from domain.modelling.measure_type import MeasureType
|
||||
from domain.modelling.recommendation import Cost, MeasureOption, Recommendation
|
||||
|
|
@ -48,6 +48,7 @@ class _GlazingTarget:
|
|||
glazing_type: int
|
||||
u_value: float
|
||||
solar_transmittance: float
|
||||
frame_factor: float
|
||||
|
||||
|
||||
# Unrestricted: replace the units with double glazing (gt=5 "Double post 2022";
|
||||
|
|
@ -58,6 +59,10 @@ _DOUBLE: Final[_GlazingTarget] = _GlazingTarget(
|
|||
glazing_type=5,
|
||||
u_value=1.40,
|
||||
solar_transmittance=0.72,
|
||||
# Replacement double units re-lodge a standard FF=0.70 (cert 001431's
|
||||
# after), overriding the panes they replace (e.g. FF 1.00 / 0.50 on the
|
||||
# "single glazing, known data" windows) — feeds §6 solar gains.
|
||||
frame_factor=0.70,
|
||||
)
|
||||
# Protected (conservation/listed/heritage): fit an internal secondary pane
|
||||
# (gt=11 "Secondary glazing - Normal emissivity", what cert 001431 re-lodges;
|
||||
|
|
@ -70,9 +75,61 @@ _SECONDARY: Final[_GlazingTarget] = _GlazingTarget(
|
|||
glazing_type=11,
|
||||
u_value=2.90,
|
||||
solar_transmittance=0.85,
|
||||
frame_factor=0.70, # cert 001431's after re-lodges FF=0.70.
|
||||
)
|
||||
|
||||
|
||||
def _is_draught_proofed(window: SapWindow) -> bool:
|
||||
"""`SapWindow.draught_proofed` is `Union[bool, str]` (bool from the
|
||||
site-notes mapper, the string "true"/"false" from the API). Normalise
|
||||
to a bool."""
|
||||
flag = window.draught_proofed
|
||||
if isinstance(flag, bool):
|
||||
return flag
|
||||
return flag.strip().lower() in {"true", "yes"}
|
||||
|
||||
|
||||
def _recompute_percent_draughtproofed(
|
||||
epc: EpcPropertyData, upgraded_indices: tuple[int, ...]
|
||||
) -> Optional[int]:
|
||||
"""RdSAP 10 §8.1 draught-proofing percentage after a glazing upgrade.
|
||||
|
||||
§8.1: "[(number of draughtproofed openable windows & doors) / (total
|
||||
number of openable windows & doors)] × 100", as an integer. Sealed
|
||||
double/secondary units are draught-proofed, so every upgraded window
|
||||
that was NOT draught-proofed flips into the numerator.
|
||||
|
||||
Anchored on the lodged dwelling-level `percent_draughtproofed` (the
|
||||
value the §2 cascade reads) rather than re-deriving the before-count
|
||||
from per-window flags: the unchanged openings are already folded into
|
||||
that aggregate, so this is robust to incomplete window extraction.
|
||||
|
||||
d0 = round(before% / 100 × N) # draught-proofed before
|
||||
after = round((d0 + flips) / N × 100) # RdSAP 10 §8.1, integer
|
||||
|
||||
N counts every openable window (vertical + roof) plus external doors.
|
||||
Returns None when no before% is lodged or there are no openings.
|
||||
"""
|
||||
before = epc.percent_draughtproofed
|
||||
if before is None:
|
||||
return None
|
||||
n_openings = (
|
||||
len(epc.sap_windows)
|
||||
+ len(epc.sap_roof_windows or [])
|
||||
+ (epc.door_count or 0)
|
||||
)
|
||||
if n_openings <= 0:
|
||||
return None
|
||||
flips = sum(
|
||||
1
|
||||
for index in upgraded_indices
|
||||
if not _is_draught_proofed(epc.sap_windows[index])
|
||||
)
|
||||
draught_proofed_before = round(before / 100 * n_openings)
|
||||
after = round((draught_proofed_before + flips) / n_openings * 100)
|
||||
return max(0, min(100, after))
|
||||
|
||||
|
||||
def recommend_glazing(
|
||||
epc: EpcPropertyData,
|
||||
products: ProductRepository,
|
||||
|
|
@ -99,9 +156,15 @@ def recommend_glazing(
|
|||
glazing_type=target.glazing_type,
|
||||
u_value=target.u_value,
|
||||
solar_transmittance=target.solar_transmittance,
|
||||
frame_factor=target.frame_factor,
|
||||
)
|
||||
for index in single_indices
|
||||
}
|
||||
},
|
||||
# Sealed units draught-proof the panes they replace — RdSAP 10
|
||||
# §8.1 re-lodges the dwelling's percentage (cert 001431: 84 → 100).
|
||||
percent_draughtproofed=_recompute_percent_draughtproofed(
|
||||
epc, single_indices
|
||||
),
|
||||
)
|
||||
cost = Cost(
|
||||
total=len(single_indices) * product.unit_cost_per_m2,
|
||||
|
|
|
|||
|
|
@ -9,10 +9,12 @@ then discarded). See ADR-0016.
|
|||
|
||||
import copy
|
||||
from dataclasses import fields
|
||||
from typing import Optional, Sequence
|
||||
from typing import Final, Optional, Sequence
|
||||
|
||||
from datatypes.epc.domain.epc_property_data import (
|
||||
BuildingPartIdentifier,
|
||||
EpcPropertyData,
|
||||
SapBuildingPart,
|
||||
SapVentilation,
|
||||
SapWindow,
|
||||
WindowTransmissionDetails,
|
||||
|
|
@ -29,6 +31,39 @@ from domain.modelling.simulation import (
|
|||
)
|
||||
|
||||
|
||||
# A Landlord Override's building part is a POSITIONAL index (0=main, 1=extension
|
||||
# 1…, ADR-0004), translated to a `BuildingPartIdentifier` upstream. This recovers
|
||||
# that position so an override can fall back onto the part the EPC actually models
|
||||
# at that slot when the gov-API labelled it differently (e.g. lodged a 2nd part as
|
||||
# `other` rather than `extension_1`).
|
||||
_POSITION_BY_IDENTIFIER: Final[dict[BuildingPartIdentifier, int]] = {
|
||||
BuildingPartIdentifier.MAIN: 0,
|
||||
BuildingPartIdentifier.EXTENSION_1: 1,
|
||||
BuildingPartIdentifier.EXTENSION_2: 2,
|
||||
BuildingPartIdentifier.EXTENSION_3: 3,
|
||||
BuildingPartIdentifier.EXTENSION_4: 4,
|
||||
}
|
||||
|
||||
|
||||
def _resolve_part(
|
||||
epc: EpcPropertyData,
|
||||
parts_by_id: dict[BuildingPartIdentifier, SapBuildingPart],
|
||||
identifier: BuildingPartIdentifier,
|
||||
) -> Optional[SapBuildingPart]:
|
||||
"""The building part an overlay targets: the EPC's part with that identifier
|
||||
when present, else the part at the override's positional index (so a
|
||||
correction for `extension_1` still lands on the EPC's 2nd part even when the
|
||||
gov-API lodged it under a different label). ``None`` when the EPC models no
|
||||
part at that position."""
|
||||
part = parts_by_id.get(identifier)
|
||||
if part is not None:
|
||||
return part
|
||||
position = _POSITION_BY_IDENTIFIER.get(identifier)
|
||||
if position is None or position >= len(epc.sap_building_parts):
|
||||
return None
|
||||
return epc.sap_building_parts[position]
|
||||
|
||||
|
||||
def apply_simulations(
|
||||
baseline: EpcPropertyData, simulations: Sequence[EpcSimulation]
|
||||
) -> EpcPropertyData:
|
||||
|
|
@ -41,7 +76,12 @@ def apply_simulations(
|
|||
|
||||
for simulation in simulations:
|
||||
for identifier, overlay in simulation.building_parts.items():
|
||||
part = parts_by_id[identifier]
|
||||
part = _resolve_part(result, parts_by_id, identifier)
|
||||
# No part at this position — the EPC models fewer parts than the
|
||||
# override's index. We have no geometry to model the missing part, so
|
||||
# skip it rather than crash the whole property's modelling.
|
||||
if part is None:
|
||||
continue
|
||||
for overlay_field in fields(overlay):
|
||||
value = getattr(overlay, overlay_field.name)
|
||||
if value is not None:
|
||||
|
|
@ -66,6 +106,8 @@ def apply_simulations(
|
|||
result.property_type = simulation.property_type
|
||||
if simulation.built_form is not None:
|
||||
result.built_form = simulation.built_form
|
||||
if simulation.percent_draughtproofed is not None:
|
||||
result.percent_draughtproofed = simulation.percent_draughtproofed
|
||||
|
||||
return result
|
||||
|
||||
|
|
@ -178,11 +220,13 @@ def _fold_lighting(epc: EpcPropertyData, overlay: LightingOverlay) -> None:
|
|||
|
||||
def _fold_window(window: SapWindow, overlay: WindowOverlay) -> None:
|
||||
"""Write a `WindowOverlay`'s non-``None`` fields onto a (copied) window:
|
||||
``glazing_type`` flat on the window, ``u_value`` / ``solar_transmittance``
|
||||
into its `WindowTransmissionDetails` (where the cascade reads them), starting
|
||||
a fresh one when the window lodged none."""
|
||||
``glazing_type`` and ``frame_factor`` flat on the window, ``u_value`` /
|
||||
``solar_transmittance`` into its `WindowTransmissionDetails` (where the
|
||||
cascade reads them), starting a fresh one when the window lodged none."""
|
||||
if overlay.glazing_type is not None:
|
||||
window.glazing_type = overlay.glazing_type
|
||||
if overlay.frame_factor is not None:
|
||||
window.frame_factor = overlay.frame_factor
|
||||
if overlay.u_value is None and overlay.solar_transmittance is None:
|
||||
return
|
||||
details: Optional[WindowTransmissionDetails] = window.window_transmission_details
|
||||
|
|
|
|||
|
|
@ -77,12 +77,16 @@ class WindowOverlay:
|
|||
are written into the window's `WindowTransmissionDetails` — where the
|
||||
calculator reads heat loss and solar gain from — because our calculator
|
||||
consumes the lodged values directly rather than deriving them from
|
||||
`glazing_type`. A `None` field means "leave the baseline value unchanged".
|
||||
`glazing_type`. `frame_factor` is written flat on the window (the §6
|
||||
solar-gain area factor); a replacement unit re-lodges its own FF, which
|
||||
can differ from the pane it replaced. A `None` field means "leave the
|
||||
baseline value unchanged".
|
||||
"""
|
||||
|
||||
glazing_type: Optional[int] = None
|
||||
u_value: Optional[float] = None
|
||||
solar_transmittance: Optional[float] = None
|
||||
frame_factor: Optional[float] = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
|
|
@ -263,3 +267,8 @@ class EpcSimulation:
|
|||
# built form (property_type drives party-wall heat loss + measure eligibility).
|
||||
property_type: Optional[str] = None
|
||||
built_form: Optional[str] = None
|
||||
# Whole-dwelling RdSAP 10 §8.1 draught-proofing percentage, when a
|
||||
# Measure changes it (e.g. glazing: sealed units draught-proof the
|
||||
# panes they replace). The §2 cascade reads this dwelling-level value,
|
||||
# so the overlay sets it directly. `None` leaves the baseline's value.
|
||||
percent_draughtproofed: Optional[int] = None
|
||||
|
|
|
|||
288
scripts/hyde/build_10070004512.py
Normal file
288
scripts/hyde/build_10070004512.py
Normal file
|
|
@ -0,0 +1,288 @@
|
|||
"""Elmhurst build for UPRN 10070004512 (SAP-Schema-16.0, GROUND-FLOOR FLAT,
|
||||
band B, cavity FILLED, ELECTRIC STORAGE HEATERS (SAP 402 SEB, manual charge
|
||||
control CSA/2401) + electric immersion off-peak (Economy-7 Dual meter) with a
|
||||
cylinder (size 1), roof = another dwelling above, floor to EXTERNAL AIR, double
|
||||
glazed, TFA 33.24, window 4.88 m². Engine 66 = lodged 66.
|
||||
|
||||
P1 of the modelling_e2e corpus validation — the built_form fix cert (16.0 omitted
|
||||
built_form; mapper derives it from dwelling_type → flat→modal 4). built_form is
|
||||
ML-only so SAP-neutral; engine reproduces lodged exactly. Storage-heater build
|
||||
(see build_10022893721.py). Engine models NO secondary (sap_heating.
|
||||
secondary_heating_type is None) → secondary present=No to match. Run:
|
||||
DISPLAY=:99 python scripts/hyde/build_10070004512.py <page>
|
||||
"""
|
||||
from __future__ import annotations
|
||||
import sys
|
||||
import elmhurst_lib as E
|
||||
|
||||
DIM = "TabContainer_TabPanelMain_WebUserControlDimensionsMain_"
|
||||
WALL = ("TabContainer_TabPanelMain_InnerTabContainerMain_"
|
||||
"TabPanelExternalWallMain_WebUserControlWallMain_")
|
||||
ROOF = "TabContainer_TabPanelMain_WebUserControlRoofMain_"
|
||||
FLOOR = "TabContainer_TabPanelMain_WebUserControlFloorsMain_"
|
||||
WP = "TabContainer_TabPanelWindowsPanel_"
|
||||
DP = "TabContainer_TabPanelDoorsPanel_"
|
||||
VP = "TabContainer_TabPanelVentilationPanel_"
|
||||
APT = "TabContainer_TabPanelAirPressureTest_"
|
||||
LP = "TabContainer_TabPanelLighting_"
|
||||
MV = "TabContainer_TabPanelMechVent_"
|
||||
WH = "TabContainer_TabPanelWaterHeating_"
|
||||
|
||||
|
||||
def _pick(page, suffix, contains):
|
||||
val = page.evaluate(
|
||||
"""(a)=>{const s=document.getElementById(a[0]);if(!s)return null;
|
||||
for(const o of s.options){if(o.text.toLowerCase().includes(a[1].toLowerCase()))return o.value;}return null;}""",
|
||||
[f"{E.FP}{suffix}", contains])
|
||||
if val is not None:
|
||||
E.set_select(page, suffix, val)
|
||||
return val
|
||||
|
||||
|
||||
def _options(page, suffix):
|
||||
return page.evaluate(
|
||||
"""(id)=>{const s=document.getElementById(id);if(!s)return [];
|
||||
return Array.from(s.options).map(o=>o.text);}""", f"{E.FP}{suffix}")
|
||||
|
||||
|
||||
def property_description(page):
|
||||
E.goto(page, "PropertyDescription", "WebFormPropertyDescription.aspx")
|
||||
E.set_select(page, "DropDownListPropertyType1", "F Flat")
|
||||
_pick(page, "DropDownListPropertyType2", "mid-terrace") # built_form 4
|
||||
E.set_text(page, "TextBoxStoreys", "1")
|
||||
E.set_text(page, "TextBoxHabitableRooms", "2")
|
||||
E.set_text(page, "TextBoxHeatedHabitableRooms", "2")
|
||||
print("date ->", _pick(page, "DropDownListDateBuiltMain", "1900-1929")) # band B
|
||||
E.set_select(page, "DropDownListDateBuiltFirst", "")
|
||||
E.set_select(page, "DropDownListRoomInRoofMain", "")
|
||||
E.save_close(page)
|
||||
|
||||
|
||||
def flats(page):
|
||||
E.goto(page, "Flats", "WebFormFlats.aspx")
|
||||
E.set_select(page, "DropDownListPositionOfFlat", "Ground Floor")
|
||||
E.set_text(page, "TextBoxFloor", "0")
|
||||
E.set_select(page, "RadioButtonListFlatCoridor", "None")
|
||||
E.save_close(page)
|
||||
|
||||
|
||||
def dimensions(page):
|
||||
E.goto(page, "Dimensions", "WebFormDimensions.aspx")
|
||||
E.set_text(page, f"{DIM}TextBoxFloorAreaLowestFloor", "33.24")
|
||||
E.set_text(page, f"{DIM}TextBoxRoomHeightLowestFloor", "2.42")
|
||||
E.set_text(page, f"{DIM}TextBoxWallPerimeterLowestFloor", "10.39")
|
||||
# 16.0 lodges no party_wall_length; a flat's side party walls are unmodelled by
|
||||
# the engine. Try 0 (Elmhurst may require non-zero — adjust if Recommendations
|
||||
# complains).
|
||||
E.set_text(page, f"{DIM}TextBoxPartyWallLengthLowestFloor", "0")
|
||||
E.save_close(page)
|
||||
|
||||
|
||||
def walls(page):
|
||||
E.goto(page, "Walls", "WebFormWalls.aspx")
|
||||
E.set_select(page, f"{WALL}DropDownListType", "CA Cavity")
|
||||
page.wait_for_timeout(400)
|
||||
print("insulation ->", _pick(page, f"{WALL}DropDownListInsulation", "filled"))
|
||||
E.save_close(page)
|
||||
|
||||
|
||||
def roofs(page):
|
||||
E.goto(page, "Roofs", "WebFormRoofs.aspx")
|
||||
_pick(page, f"{ROOF}DropDownListType", "another dwelling above")
|
||||
E.save_close(page)
|
||||
|
||||
|
||||
def floors(page):
|
||||
E.goto(page, "Floors", "WebFormFloors.aspx")
|
||||
# Floor is "to external air" — that is the LOCATION/exposure, not a TYPE.
|
||||
_pick(page, f"{FLOOR}DropDownListLocation", "external air") # E To external air
|
||||
page.wait_for_timeout(400)
|
||||
_pick(page, f"{FLOOR}DropDownListType", "solid") # construction; U from exposure
|
||||
ins = page.locator(f"#{E.FP}{FLOOR}DropDownListInsulation")
|
||||
if ins.count():
|
||||
E.set_select(page, f"{FLOOR}DropDownListInsulation", "A As built")
|
||||
E.save_close(page)
|
||||
|
||||
|
||||
def openings(page):
|
||||
E.goto(page, "Openings", "WebFormOpenings.aspx")
|
||||
E.click_tab(page, "TabContainer_TabPanelWindowsPanel")
|
||||
_add_window(page, 4.88, "North", _glazing(page))
|
||||
_delete_zero_rows(page)
|
||||
E.click_tab(page, "TabContainer_TabPanelDoorsPanel")
|
||||
E.set_text(page, f"{DP}TextBoxDoors", "1")
|
||||
E.set_text(page, f"{DP}TextBoxDoorsInsulated", "0")
|
||||
E.set_text(page, f"{DP}TextBoxDraughtProofedDoors", "0")
|
||||
E.save_close(page)
|
||||
|
||||
|
||||
def _glazing(page):
|
||||
for needle in ("unknown install date", "before 2002", "pre 2002"):
|
||||
for opt in _options(page, f"{WP}DropDownListExtGlazing"):
|
||||
low = opt.lower()
|
||||
if needle in low and "triple" not in low and "single" not in low and "known data" not in low:
|
||||
return opt
|
||||
return "Double post or during 2022"
|
||||
|
||||
|
||||
def _add_window(page, area, orientation, glazing):
|
||||
print("glazing ->", glazing)
|
||||
E.set_select(page, f"{WP}DropDownListExtGlazing", glazing)
|
||||
page.wait_for_timeout(400)
|
||||
ft = page.locator(f"#{E.FP}{WP}DropDownListExtFrameType")
|
||||
if ft.count():
|
||||
ft.select_option("PVC")
|
||||
gg = page.locator(f"#{E.FP}{WP}DropDownListExtGlazingGap")
|
||||
if gg.count():
|
||||
gg.select_option("12 mm")
|
||||
wid = f"{E.FP}{WP}TextBoxExtWidth"
|
||||
page.evaluate(
|
||||
"""(a)=>{const e=document.getElementById(a[0]);if(e){e.value=a[1];
|
||||
e.dispatchEvent(new Event('input',{bubbles:true}));
|
||||
e.dispatchEvent(new Event('change',{bubbles:true}));
|
||||
e.dispatchEvent(new Event('blur',{bubbles:true}));}}""", [wid, str(area)])
|
||||
page.locator(f"#{E.FP}{WP}TextBoxExtHeight").fill("1.00")
|
||||
page.locator(f"#{E.FP}{WP}DropDownListExtOrientation").select_option(orientation)
|
||||
page.locator(f"#{E.FP}{WP}DropDownListExtBuildingPartId").select_option("Main")
|
||||
page.locator(f"#{E.FP}{WP}DropDownListExtLocation").select_option("External wall")
|
||||
page.wait_for_timeout(300)
|
||||
before = E.window_row_count(page)
|
||||
page.evaluate("(id)=>{const e=document.getElementById(id); if(e)e.click();}", f"{E.FP}{WP}ButtonAddWindow")
|
||||
for _ in range(25):
|
||||
page.wait_for_timeout(200)
|
||||
if E.window_row_count(page) > before:
|
||||
break
|
||||
|
||||
|
||||
def _grid_rows(page):
|
||||
return page.evaluate(
|
||||
"""()=>{const t=document.querySelector("[id*=GridViewExtendedWidows]");
|
||||
if(!t)return[];return Array.from(t.querySelectorAll('tr')).slice(1)
|
||||
.map(r=>Array.from(r.querySelectorAll('td')).map(c=>c.innerText.trim()));}""")
|
||||
|
||||
|
||||
def _delete_zero_rows(page):
|
||||
g = 0
|
||||
while g < 6 and E.window_row_count(page) > 1:
|
||||
g += 1
|
||||
rows = _grid_rows(page)
|
||||
bad = next((i for i, c in enumerate(rows) if len(c) > 1 and c[1] in ("0.00", "0", "0.0")), None)
|
||||
if bad is None:
|
||||
break
|
||||
_delete_row(page, bad)
|
||||
page.wait_for_timeout(400)
|
||||
|
||||
|
||||
def _delete_row(page, idx):
|
||||
before = E.window_row_count(page)
|
||||
btn = page.evaluate(
|
||||
"""(i)=>{const b=document.querySelectorAll("[id*='GridViewExtendedWidows_DeleteButton_']");return b[i]?b[i].id:null;}""", idx)
|
||||
if not btn:
|
||||
return
|
||||
page.evaluate("(id)=>{const e=document.getElementById(id); if(e)e.click();}", btn)
|
||||
page.wait_for_selector(f"#{E.FP}DeleteWindowDialog_LinkButtonYes", state="visible", timeout=5000)
|
||||
page.evaluate("(id)=>{const e=document.getElementById(id); if(e)e.click();}", f"{E.FP}DeleteWindowDialog_LinkButtonYes")
|
||||
for _ in range(20):
|
||||
page.wait_for_timeout(200)
|
||||
if E.window_row_count(page) < before:
|
||||
break
|
||||
|
||||
|
||||
def ventilation(page):
|
||||
E.goto(page, "VentilationAndCooling", "WebFormVentilationAndCooling.aspx")
|
||||
E.click_tab(page, "TabContainer_TabPanelVentilationPanel")
|
||||
E.set_text(page, f"{VP}TextBoxIntermittentFans", "0")
|
||||
cool = page.locator(f"#{E.FP}{VP}CheckBoxFixedSpaceCooling")
|
||||
if cool.count() and cool.is_checked():
|
||||
E.commit(page, cool.uncheck)
|
||||
E.click_tab(page, "TabContainer_TabPanelMechVent")
|
||||
mv = page.locator(f"#{E.FP}{MV}CheckBoxMechanicalVentilation")
|
||||
if mv.count() and mv.is_checked():
|
||||
E.commit(page, mv.uncheck)
|
||||
E.click_tab(page, "TabContainer_TabPanelAirPressureTest")
|
||||
E.set_select(page, f"{APT}DropDownListTestMethod", "Not available")
|
||||
E.click_tab(page, "TabContainer_TabPanelLighting")
|
||||
E.set_text(page, f"{LP}TextBoxLightsTotal", "5")
|
||||
E.set_text(page, f"{LP}TextBoxLedLightsTotal", "5") # 100% low energy
|
||||
E.set_text(page, f"{LP}TextBoxCflLightsTotal", "0")
|
||||
E.save_close(page)
|
||||
|
||||
|
||||
def space_heating(page):
|
||||
# Electric storage heaters (SAP 402 = SEB), manual charge control (SAP 2401 =
|
||||
# CSA). Two passes: clear bound PCDB boiler first, then set the SAP-table code.
|
||||
E.goto(page, "SpaceHeating", "WebFormSpaceHeating.aspx")
|
||||
page.wait_for_timeout(1000)
|
||||
rid = f"{E.MH1}TextBoxPCDFBoilerReference"
|
||||
ref = page.locator(f"#{rid}").input_value()
|
||||
if ref not in ("0", ""):
|
||||
print(f"clearing bound PCDB boiler {ref} -> 0 (rerun space_heating)")
|
||||
page.evaluate("""(rid)=>{const r=document.getElementById(rid);r.value='0';
|
||||
r.dispatchEvent(new Event('change',{bubbles:true}));}""", rid)
|
||||
page.wait_for_timeout(500)
|
||||
E.save_close(page)
|
||||
return
|
||||
E.set_heating_dialog(page, "TabContainer_TabPanelMainHeating1_WebUserControlMainHeating1_ButtonMainHeatingCode",
|
||||
"^Electric", "^Electric", "Storage", "SEB Modern slimline")
|
||||
print("code:", page.locator(f"#{E.MH1}TextBoxMainHeatingCode").input_value())
|
||||
E.set_heating_dialog(page, "TabContainer_TabPanelMainHeating1_WebUserControlMainHeating1_ButtonMainHeatingControls",
|
||||
"Storage Radiator", "CSA Manual charge control")
|
||||
print("control:", page.locator(f"#{E.MH1}TextBoxMainHeatingControls").input_value())
|
||||
E.set_select(page, "DropDownListSecondaryHeatingPresent", "No")
|
||||
# Economy-7 Dual meter (cert meter_type 1) — hidden Meters sub-tab.
|
||||
E.click_tab(page, "TabContainer_TabPanelMeters")
|
||||
E.set_select(page, "TabContainer_TabPanelMeters_RadioButtonListElectricityType", "Dual")
|
||||
print("meter:", page.locator("#ContentBody_ContentPlaceHolder1_TabContainer_TabPanelMeters_RadioButtonListElectricityType").input_value())
|
||||
E.save_close(page)
|
||||
|
||||
|
||||
def water_heating(page):
|
||||
# Electric immersion off-peak (Dual) WITH cylinder (size 1 = small). The
|
||||
# immersion code REQUIRES a cylinder.
|
||||
E.goto(page, "WaterHeating", "WebFormWaterHeating.aspx")
|
||||
E.click_tab(page, "TabContainer_TabPanelWaterHeating")
|
||||
page.wait_for_timeout(600)
|
||||
E.set_heating_dialog(page, f"{WH}ButtonWaterHeatingCode",
|
||||
"Water Heater", "^Electric", "Immersion")
|
||||
print("water code:", page.locator(f"#{E.FP}{WH}TextBoxWaterHeatingCode").input_value())
|
||||
cid = f"{E.FP}{WH}CheckBoxHotWaterCylinder"
|
||||
cyl = page.locator(f"#{cid}")
|
||||
if cyl.count() and not cyl.is_checked():
|
||||
try:
|
||||
with page.expect_navigation(wait_until="load", timeout=8000):
|
||||
page.evaluate(
|
||||
"""(id)=>{const c=document.getElementById(id);if(c){c.checked=true;
|
||||
c.dispatchEvent(new Event('click',{bubbles:true}));
|
||||
c.dispatchEvent(new Event('change',{bubbles:true}));}}""", cid)
|
||||
except Exception:
|
||||
page.wait_for_timeout(2000)
|
||||
print("cylinder present:", cyl.is_checked() if cyl.count() else "n/a")
|
||||
print("cyl sizes:", _options(page, f"{WH}DropDownListCylinderSize"))
|
||||
_pick(page, f"{WH}DropDownListCylinderSize", "small") or \
|
||||
E.set_select(page, f"{WH}DropDownListCylinderSize", "Normal")
|
||||
E.set_select(page, f"{WH}DropDownListInsulated", "Foam")
|
||||
isuf = f"{WH}DropDownListInsulationThickness"
|
||||
if page.locator(f"#{E.FP}{isuf}").count():
|
||||
_pick(page, isuf, "Unknown") or E.set_select(page, isuf, "25 mm")
|
||||
imm = page.locator(f"#{E.FP}{WH}RadioButtonListImmersionHeater")
|
||||
if imm.count():
|
||||
E.set_select(page, f"{WH}RadioButtonListImmersionHeater", "Dual")
|
||||
E.save_close(page)
|
||||
|
||||
|
||||
_ORDER = ["property_description", "flats", "dimensions", "walls", "roofs",
|
||||
"floors", "openings", "ventilation", "space_heating", "water_heating"]
|
||||
|
||||
|
||||
def main():
|
||||
if len(sys.argv) < 2 or sys.argv[1] not in _ORDER:
|
||||
print("usage: build_10070004512.py <" + "|".join(_ORDER) + ">")
|
||||
return 2
|
||||
with E.session() as (ctx, page):
|
||||
globals()[sys.argv[1]](page)
|
||||
print("done:", sys.argv[1], "->", page.url)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
307
scripts/hyde/build_22086693.py
Normal file
307
scripts/hyde/build_22086693.py
Normal file
|
|
@ -0,0 +1,307 @@
|
|||
"""Elmhurst build for UPRN 22086693 (RdSAP-Schema-20.0.0, SEMI-DETACHED HOUSE,
|
||||
2-storey, band C 1930-1949, cavity UNINSULATED, mains-gas COMBI (no PCDB index →
|
||||
generic SAP Table 4b combi), control 2106 (CBE), pitched 200 mm loft, suspended
|
||||
uninsulated floor, party wall 6.8 m, double glazed, 2× PV ARRAYS @1.14 kW
|
||||
(orientations SE/SW, pitch 30°, overshading modest), electric SECONDARY room
|
||||
heater (SAP 691 REA), TFA ~74, window 11.84 m². Engine 66 / lodged 72.
|
||||
|
||||
P2 of the modelling_e2e corpus validation — the photovoltaic_supply-as-list fix
|
||||
cert. The PV adds +5 (engine 61→66) so it IS credited; the −6 vs lodged is a
|
||||
fabric/heating gap to localise. NEW build elements: generic (no-PCDB) combi +
|
||||
the Renewables/PV page. Run:
|
||||
DISPLAY=:99 python scripts/hyde/build_22086693.py <page>
|
||||
"""
|
||||
from __future__ import annotations
|
||||
import sys
|
||||
import elmhurst_lib as E
|
||||
|
||||
DIM = "TabContainer_TabPanelMain_WebUserControlDimensionsMain_"
|
||||
WALL = ("TabContainer_TabPanelMain_InnerTabContainerMain_"
|
||||
"TabPanelExternalWallMain_WebUserControlWallMain_")
|
||||
PWALL = "TabContainer_TabPanelMain_InnerTabContainerMain_TabPanelPartyWallMain_WebUserControlPartyWallMain_"
|
||||
ROOF = "TabContainer_TabPanelMain_WebUserControlRoofMain_"
|
||||
FLOOR = "TabContainer_TabPanelMain_WebUserControlFloorsMain_"
|
||||
WP = "TabContainer_TabPanelWindowsPanel_"
|
||||
DP = "TabContainer_TabPanelDoorsPanel_"
|
||||
VP = "TabContainer_TabPanelVentilationPanel_"
|
||||
APT = "TabContainer_TabPanelAirPressureTest_"
|
||||
LP = "TabContainer_TabPanelLighting_"
|
||||
MV = "TabContainer_TabPanelMechVent_"
|
||||
WH = "TabContainer_TabPanelWaterHeating_"
|
||||
MH1B = "TabContainer_TabPanelMainHeating1_WebUserControlMainHeating1_"
|
||||
|
||||
|
||||
def _pick(page, suffix, contains):
|
||||
val = page.evaluate(
|
||||
"""(a)=>{const s=document.getElementById(a[0]);if(!s)return null;
|
||||
for(const o of s.options){if(o.text.toLowerCase().includes(a[1].toLowerCase()))return o.value;}return null;}""",
|
||||
[f"{E.FP}{suffix}", contains])
|
||||
if val is not None:
|
||||
E.set_select(page, suffix, val)
|
||||
return val
|
||||
|
||||
|
||||
def _options(page, suffix):
|
||||
return page.evaluate(
|
||||
"""(id)=>{const s=document.getElementById(id);if(!s)return [];
|
||||
return Array.from(s.options).map(o=>o.text);}""", f"{E.FP}{suffix}")
|
||||
|
||||
|
||||
def property_description(page):
|
||||
E.goto(page, "PropertyDescription", "WebFormPropertyDescription.aspx")
|
||||
E.set_select(page, "DropDownListPropertyType1", "H House")
|
||||
_pick(page, "DropDownListPropertyType2", "semi") # built_form 2
|
||||
E.set_text(page, "TextBoxStoreys", "2")
|
||||
E.set_text(page, "TextBoxHabitableRooms", "4")
|
||||
E.set_text(page, "TextBoxHeatedHabitableRooms", "4")
|
||||
print("date ->", _pick(page, "DropDownListDateBuiltMain", "1930-1949")) # band C
|
||||
E.set_select(page, "DropDownListDateBuiltFirst", "")
|
||||
E.set_select(page, "DropDownListRoomInRoofMain", "")
|
||||
E.save_close(page)
|
||||
|
||||
|
||||
def dimensions(page):
|
||||
E.goto(page, "Dimensions", "WebFormDimensions.aspx")
|
||||
E.set_text(page, f"{DIM}TextBoxFloorAreaLowestFloor", "36.86")
|
||||
E.set_text(page, f"{DIM}TextBoxRoomHeightLowestFloor", "2.30")
|
||||
E.set_text(page, f"{DIM}TextBoxWallPerimeterLowestFloor", "13.4")
|
||||
E.set_text(page, f"{DIM}TextBoxPartyWallLengthLowestFloor", "6.8")
|
||||
E.set_text(page, f"{DIM}TextBoxFloorArea1stFloor", "36.86")
|
||||
E.set_text(page, f"{DIM}TextBoxRoomHeight1stFloor", "2.30")
|
||||
E.set_text(page, f"{DIM}TextBoxWallPerimeter1stFloor", "17.4")
|
||||
E.set_text(page, f"{DIM}TextBoxPartyWallLength1stFloor", "6.8")
|
||||
E.save_close(page)
|
||||
|
||||
|
||||
def walls(page):
|
||||
E.goto(page, "Walls", "WebFormWalls.aspx")
|
||||
E.set_select(page, f"{WALL}DropDownListType", "CA Cavity")
|
||||
page.wait_for_timeout(400)
|
||||
print("insulation ->", _pick(page, f"{WALL}DropDownListInsulation", "as built")) # uninsulated
|
||||
# Semi party wall: cavity. Match "masonry filled" → CF (U≈0); avoid loose "filled".
|
||||
pw = _pick(page, f"{PWALL}DropDownListPartyWallType", "masonry filled") \
|
||||
or _pick(page, f"{PWALL}DropDownListPartyWallType", "determine")
|
||||
print("party wall ->", pw)
|
||||
E.save_close(page)
|
||||
|
||||
|
||||
def roofs(page):
|
||||
E.goto(page, "Roofs", "WebFormRoofs.aspx")
|
||||
_pick(page, f"{ROOF}DropDownListType", "access to loft")
|
||||
_pick(page, f"{ROOF}DropDownListInsulationAt", "joists")
|
||||
E.set_select(page, f"{ROOF}DropDownListThickness", "200 mm")
|
||||
E.save_close(page)
|
||||
|
||||
|
||||
def floors(page):
|
||||
E.goto(page, "Floors", "WebFormFloors.aspx")
|
||||
E.set_select(page, f"{FLOOR}DropDownListLocation", "G Ground floor")
|
||||
_pick(page, f"{FLOOR}DropDownListType", "suspended timber")
|
||||
E.set_select(page, f"{FLOOR}DropDownListInsulation", "A As built")
|
||||
E.save_close(page)
|
||||
|
||||
|
||||
def openings(page):
|
||||
E.goto(page, "Openings", "WebFormOpenings.aspx")
|
||||
E.click_tab(page, "TabContainer_TabPanelWindowsPanel")
|
||||
_add_window(page, 11.84, "North", _glazing(page))
|
||||
_delete_zero_rows(page)
|
||||
E.click_tab(page, "TabContainer_TabPanelDoorsPanel")
|
||||
E.set_text(page, f"{DP}TextBoxDoors", "1")
|
||||
E.set_text(page, f"{DP}TextBoxDoorsInsulated", "0")
|
||||
E.set_text(page, f"{DP}TextBoxDraughtProofedDoors", "0")
|
||||
E.save_close(page)
|
||||
|
||||
|
||||
def _glazing(page):
|
||||
for needle in ("unknown install date", "before 2002", "pre 2002"):
|
||||
for opt in _options(page, f"{WP}DropDownListExtGlazing"):
|
||||
low = opt.lower()
|
||||
if needle in low and "triple" not in low and "single" not in low and "known data" not in low:
|
||||
return opt
|
||||
return "Double post or during 2022"
|
||||
|
||||
|
||||
def _add_window(page, area, orientation, glazing):
|
||||
print("glazing ->", glazing)
|
||||
E.set_select(page, f"{WP}DropDownListExtGlazing", glazing)
|
||||
page.wait_for_timeout(400)
|
||||
ft = page.locator(f"#{E.FP}{WP}DropDownListExtFrameType")
|
||||
if ft.count():
|
||||
ft.select_option("PVC")
|
||||
gg = page.locator(f"#{E.FP}{WP}DropDownListExtGlazingGap")
|
||||
if gg.count():
|
||||
gg.select_option("12 mm")
|
||||
wid = f"{E.FP}{WP}TextBoxExtWidth"
|
||||
page.evaluate(
|
||||
"""(a)=>{const e=document.getElementById(a[0]);if(e){e.value=a[1];
|
||||
e.dispatchEvent(new Event('input',{bubbles:true}));
|
||||
e.dispatchEvent(new Event('change',{bubbles:true}));
|
||||
e.dispatchEvent(new Event('blur',{bubbles:true}));}}""", [wid, str(area)])
|
||||
page.locator(f"#{E.FP}{WP}TextBoxExtHeight").fill("1.00")
|
||||
page.locator(f"#{E.FP}{WP}DropDownListExtOrientation").select_option(orientation)
|
||||
page.locator(f"#{E.FP}{WP}DropDownListExtBuildingPartId").select_option("Main")
|
||||
page.locator(f"#{E.FP}{WP}DropDownListExtLocation").select_option("External wall")
|
||||
page.wait_for_timeout(300)
|
||||
before = E.window_row_count(page)
|
||||
page.evaluate("(id)=>{const e=document.getElementById(id); if(e)e.click();}", f"{E.FP}{WP}ButtonAddWindow")
|
||||
for _ in range(25):
|
||||
page.wait_for_timeout(200)
|
||||
if E.window_row_count(page) > before:
|
||||
break
|
||||
|
||||
|
||||
def fix_window(page):
|
||||
"""Edit-in-place: overwrite the shared assessment's leftover window width to ours."""
|
||||
E.goto(page, "Openings", "WebFormOpenings.aspx")
|
||||
E.click_tab(page, "TabContainer_TabPanelWindowsPanel")
|
||||
page.wait_for_timeout(800)
|
||||
wid = f"{E.FP}{WP}TextBoxExtWidth"
|
||||
page.evaluate("""(a)=>{const e=document.getElementById(a[0]);if(e){e.value=a[1];
|
||||
e.dispatchEvent(new Event('input',{bubbles:true}));
|
||||
e.dispatchEvent(new Event('change',{bubbles:true}));
|
||||
e.dispatchEvent(new Event('blur',{bubbles:true}));}}""", [wid, "11.84"])
|
||||
print("window width now:", page.locator(f"#{wid}").input_value())
|
||||
E.save_close(page)
|
||||
|
||||
|
||||
def _grid_rows(page):
|
||||
return page.evaluate(
|
||||
"""()=>{const t=document.querySelector("[id*=GridViewExtendedWidows]");
|
||||
if(!t)return[];return Array.from(t.querySelectorAll('tr')).slice(1)
|
||||
.map(r=>Array.from(r.querySelectorAll('td')).map(c=>c.innerText.trim()));}""")
|
||||
|
||||
|
||||
def _delete_zero_rows(page):
|
||||
g = 0
|
||||
while g < 6 and E.window_row_count(page) > 1:
|
||||
g += 1
|
||||
rows = _grid_rows(page)
|
||||
bad = next((i for i, c in enumerate(rows) if len(c) > 1 and c[1] in ("0.00", "0", "0.0")), None)
|
||||
if bad is None:
|
||||
break
|
||||
_delete_row(page, bad)
|
||||
page.wait_for_timeout(400)
|
||||
|
||||
|
||||
def _delete_row(page, idx):
|
||||
before = E.window_row_count(page)
|
||||
btn = page.evaluate(
|
||||
"""(i)=>{const b=document.querySelectorAll("[id*='GridViewExtendedWidows_DeleteButton_']");return b[i]?b[i].id:null;}""", idx)
|
||||
if not btn:
|
||||
return
|
||||
page.evaluate("(id)=>{const e=document.getElementById(id); if(e)e.click();}", btn)
|
||||
page.wait_for_selector(f"#{E.FP}DeleteWindowDialog_LinkButtonYes", state="visible", timeout=5000)
|
||||
page.evaluate("(id)=>{const e=document.getElementById(id); if(e)e.click();}", f"{E.FP}DeleteWindowDialog_LinkButtonYes")
|
||||
for _ in range(20):
|
||||
page.wait_for_timeout(200)
|
||||
if E.window_row_count(page) < before:
|
||||
break
|
||||
|
||||
|
||||
def ventilation(page):
|
||||
E.goto(page, "VentilationAndCooling", "WebFormVentilationAndCooling.aspx")
|
||||
E.click_tab(page, "TabContainer_TabPanelVentilationPanel")
|
||||
E.set_text(page, f"{VP}TextBoxIntermittentFans", "0")
|
||||
cool = page.locator(f"#{E.FP}{VP}CheckBoxFixedSpaceCooling")
|
||||
if cool.count() and cool.is_checked():
|
||||
E.commit(page, cool.uncheck)
|
||||
E.click_tab(page, "TabContainer_TabPanelMechVent")
|
||||
mv = page.locator(f"#{E.FP}{MV}CheckBoxMechanicalVentilation")
|
||||
if mv.count() and mv.is_checked():
|
||||
E.commit(page, mv.uncheck)
|
||||
E.click_tab(page, "TabContainer_TabPanelAirPressureTest")
|
||||
E.set_select(page, f"{APT}DropDownListTestMethod", "Not available")
|
||||
E.click_tab(page, "TabContainer_TabPanelLighting")
|
||||
E.set_text(page, f"{LP}TextBoxLightsTotal", "10")
|
||||
E.set_text(page, f"{LP}TextBoxLedLightsTotal", "10") # 100% LED
|
||||
E.set_text(page, f"{LP}TextBoxCflLightsTotal", "0")
|
||||
E.save_close(page)
|
||||
|
||||
|
||||
def space_heating(page):
|
||||
# Generic (no-PCDB) mains-gas combi. The shared assessment may carry a prior
|
||||
# cert's storage SEB / PCDB boiler — clear it first if present, then select a
|
||||
# generic gas combi via the boiler dialog. Two passes if a SAP code is bound.
|
||||
E.goto(page, "SpaceHeating", "WebFormSpaceHeating.aspx")
|
||||
page.wait_for_timeout(1000)
|
||||
# Pass 1: clear a leftover SAP-table MainHeatingCode (e.g. SEB) so the boiler
|
||||
# search/dialog is usable.
|
||||
mhc = page.locator(f"#{E.MH1}TextBoxMainHeatingCode")
|
||||
code = mhc.input_value() if mhc.count() else ""
|
||||
if code and code not in ("0",):
|
||||
print(f"clearing leftover MainHeatingCode {code}")
|
||||
page.evaluate("""(id)=>{const e=document.getElementById(id);if(e){e.value='';
|
||||
e.dispatchEvent(new Event('change',{bubbles:true}));}}""", f"{E.MH1}TextBoxMainHeatingCode")
|
||||
page.wait_for_timeout(400)
|
||||
E.save_close(page)
|
||||
return
|
||||
# Generic (no-PCDB) mains-gas boiler via the cascade: Gas → Mains gas →
|
||||
# Boilers → Post 1998 → Condensing (combi vs regular is set by the water-
|
||||
# heating page: "from primary" + no cylinder = combi). ~89% SAP Table 4b.
|
||||
E.set_heating_dialog(page, f"{MH1B}ButtonMainHeatingCode",
|
||||
"^Gas", "Mains gas", "Boilers", "Post 1998", "Condensing",
|
||||
"Combi condens") # L6 = BGW Post 98 Combi condens. (SAP 4b ~89%)
|
||||
print("code:", page.locator(f"#{E.MH1}TextBoxMainHeatingCode").input_value())
|
||||
E.set_heating_dialog(page, f"{MH1B}ButtonMainHeatingControls",
|
||||
"^Boilers", "^Standard", "CBE Programmer, room thermostat and TRVs")
|
||||
print("control:", page.locator(f"#{E.MH1}TextBoxMainHeatingControls").input_value())
|
||||
E.save_close(page)
|
||||
|
||||
|
||||
def secondary(page):
|
||||
# Lodged secondary: SAP 691 = REA electric panel/convector/radiant room heater.
|
||||
E.goto(page, "SpaceHeating", "WebFormSpaceHeating.aspx")
|
||||
page.wait_for_timeout(600)
|
||||
E.set_select(page, "DropDownListSecondaryHeatingPresent", "Yes")
|
||||
page.wait_for_timeout(900)
|
||||
E.set_heating_dialog(page, "ButtonSecondaryHeatingCode",
|
||||
"Electric", "Electric", "Room Heater", "REA Panel")
|
||||
tb = page.locator(f"#{E.FP}TextBoxSecondaryHeatingCode")
|
||||
print("secondary code:", tb.input_value() if tb.count() else "?")
|
||||
E.save_close(page)
|
||||
|
||||
|
||||
def water_heating(page):
|
||||
E.goto(page, "WaterHeating", "WebFormWaterHeating.aspx")
|
||||
E.click_tab(page, "TabContainer_TabPanelWaterHeating")
|
||||
page.wait_for_timeout(400)
|
||||
E.clear_hot_water_cylinder(page)
|
||||
E.set_heating_dialog(page, f"{WH}ButtonWaterHeatingCode",
|
||||
"From Space Heating", "From the primary heating system")
|
||||
print("water code:", page.locator(f"#{E.FP}{WH}TextBoxWaterHeatingCode").input_value())
|
||||
E.save_close(page)
|
||||
|
||||
|
||||
def renewables(page):
|
||||
# 2× PV arrays @1.14 kW. Discover the Renewables/PV page structure.
|
||||
for url in ("WebFormRenewables.aspx", "WebFormPhotovoltaics.aspx", "WebFormSolar.aspx"):
|
||||
try:
|
||||
E.goto(page, url.replace("WebForm", "").replace(".aspx", ""), url)
|
||||
page.wait_for_timeout(800)
|
||||
print("PV page:", url, "->", page.url)
|
||||
inputs = page.evaluate(
|
||||
"""()=>Array.from(document.querySelectorAll("[id*=ContentPlaceHolder1] input,[id*=ContentPlaceHolder1] select")).map(e=>e.id.replace('ContentBody_ContentPlaceHolder1_','')).filter(i=>/pv|photov|solar|peak|orient|pitch|oversh|panel/i.test(i)).slice(0,30)""")
|
||||
print("PV-ish fields:", inputs)
|
||||
return
|
||||
except Exception as e:
|
||||
print(" ", url, "->", type(e).__name__)
|
||||
|
||||
|
||||
_ORDER = ["property_description", "dimensions", "walls", "roofs", "floors",
|
||||
"openings", "fix_window", "ventilation", "space_heating", "secondary",
|
||||
"water_heating", "renewables"]
|
||||
|
||||
|
||||
def main():
|
||||
if len(sys.argv) < 2 or sys.argv[1] not in _ORDER:
|
||||
print("usage: build_22086693.py <" + "|".join(_ORDER) + ">")
|
||||
return 2
|
||||
with E.session() as (ctx, page):
|
||||
globals()[sys.argv[1]](page)
|
||||
print("done:", sys.argv[1], "->", page.url)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
|
|
@ -28,7 +28,7 @@ SESSION_DIR = HERE / ".elmhurst-session"
|
|||
SAMPLE_DIR = (
|
||||
HERE.parent.parent
|
||||
/ "backend/epc_api/json_samples/real_life_examples"
|
||||
/ "SAP-Schema-16.0/uprn_10070004512"
|
||||
/ "RdSAP-Schema-20.0.0/uprn_22086693"
|
||||
)
|
||||
|
||||
ASSESSMENT_GUID = "B44A0DB4-4C08-4241-B818-86F060172105"
|
||||
|
|
|
|||
|
|
@ -11,6 +11,7 @@ from __future__ import annotations
|
|||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Any, cast
|
||||
|
||||
_REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(_REPO_ROOT))
|
||||
|
|
@ -30,6 +31,7 @@ with engine.connect() as conn:
|
|||
st.id AS subtask_id,
|
||||
st.task_id,
|
||||
st.inputs,
|
||||
st.outputs,
|
||||
st.updated_at
|
||||
FROM sub_task st
|
||||
JOIN tasks t ON t.id = st.task_id
|
||||
|
|
@ -45,19 +47,32 @@ with engine.connect() as conn:
|
|||
)
|
||||
exit(0)
|
||||
|
||||
def _error_text(outputs_raw: object) -> str:
|
||||
"""Pull the persisted failure reason out of the subtask outputs JSON."""
|
||||
try:
|
||||
outputs: Any = (
|
||||
json.loads(outputs_raw)
|
||||
if isinstance(outputs_raw, str)
|
||||
else (outputs_raw or {})
|
||||
)
|
||||
except Exception:
|
||||
return str(outputs_raw or "")
|
||||
if isinstance(outputs, dict):
|
||||
return str(cast("dict[str, Any]", outputs).get("error", ""))
|
||||
return ""
|
||||
|
||||
# Collect all property_ids across all rows
|
||||
all_property_ids: list[int] = []
|
||||
parsed: list[tuple[str, str, list[int], str, str]] = []
|
||||
for subtask_id, task_id, inputs_raw, updated_at in subtask_rows:
|
||||
parsed: list[tuple[str, str, list[int], str, str, str]] = []
|
||||
for subtask_id, task_id, inputs_raw, outputs_raw, updated_at in subtask_rows:
|
||||
try:
|
||||
inputs = (
|
||||
inputs: Any = (
|
||||
json.loads(inputs_raw)
|
||||
if isinstance(inputs_raw, str)
|
||||
else (inputs_raw or {})
|
||||
)
|
||||
property_ids: list[int] = [
|
||||
int(p) for p in (inputs.get("property_ids") or [])
|
||||
]
|
||||
raw_ids = cast("list[Any]", cast("dict[str, Any]", inputs).get("property_ids") or [])
|
||||
property_ids: list[int] = [int(p) for p in raw_ids]
|
||||
except Exception:
|
||||
property_ids = []
|
||||
parsed.append(
|
||||
|
|
@ -67,6 +82,7 @@ with engine.connect() as conn:
|
|||
property_ids,
|
||||
str(updated_at),
|
||||
inputs_raw or "",
|
||||
_error_text(outputs_raw),
|
||||
)
|
||||
)
|
||||
all_property_ids.extend(property_ids)
|
||||
|
|
@ -80,23 +96,29 @@ with engine.connect() as conn:
|
|||
).fetchall()
|
||||
uprn_map = {int(r[0]): int(r[1]) for r in uprn_rows}
|
||||
|
||||
def _cell(value: str) -> str:
|
||||
"""Escape table-breaking characters so multi-line errors stay on one row."""
|
||||
return value.replace("|", "\\|").replace("\n", "<br>")
|
||||
|
||||
|
||||
lines: list[str] = [
|
||||
"# Failed modelling_e2e Subtasks\n",
|
||||
f"| Subtask ID | Task ID | Updated At | Property ID | UPRN | Inputs |",
|
||||
f"|-----------|---------|------------|-------------|------|--------|",
|
||||
f"| Subtask ID | Task ID | Updated At | Property ID | UPRN | Error | Inputs |",
|
||||
f"|-----------|---------|------------|-------------|------|-------|--------|",
|
||||
]
|
||||
|
||||
for subtask_id, task_id, property_ids, updated_at, inputs_raw in parsed:
|
||||
inputs_cell = (inputs_raw or "").replace("|", "\\|")
|
||||
for subtask_id, task_id, property_ids, updated_at, inputs_raw, error in parsed:
|
||||
inputs_cell = _cell(inputs_raw or "")
|
||||
error_cell = _cell(error or "")
|
||||
if property_ids:
|
||||
for pid in property_ids:
|
||||
uprn = uprn_map.get(pid, "unknown")
|
||||
lines.append(
|
||||
f"| {subtask_id} | {task_id} | {updated_at} | {pid} | {uprn} | {inputs_cell} |"
|
||||
f"| {subtask_id} | {task_id} | {updated_at} | {pid} | {uprn} | {error_cell} | {inputs_cell} |"
|
||||
)
|
||||
else:
|
||||
lines.append(
|
||||
f"| {subtask_id} | {task_id} | {updated_at} | — | — | {inputs_cell} |"
|
||||
f"| {subtask_id} | {task_id} | {updated_at} | — | — | {error_cell} | {inputs_cell} |"
|
||||
)
|
||||
|
||||
_OUTPUT.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
||||
|
|
|
|||
|
|
@ -20,9 +20,10 @@ PORTFOLIO_ID: int = 796
|
|||
SCENARIO_ID: int = 1268
|
||||
SQS_QUEUE_NAME: str = "modelling_e2e-queue-dev"
|
||||
|
||||
# Number of postcodes to process this run (postcodes where all properties are
|
||||
# already completed are skipped and do not count toward this limit).
|
||||
POSTCODES_LIMIT: int = 1000
|
||||
# Max number of properties to process this run (cost cap). Postcodes are added
|
||||
# whole, smallest-group-first, until adding the next would exceed this — so the
|
||||
# actual count lands at or just under the limit.
|
||||
PROPERTIES_LIMIT: int = 5000
|
||||
|
||||
# True → Lambda runs the full pipeline but skips all DB writes (safe for testing).
|
||||
DRY_RUN: bool = False
|
||||
|
|
@ -94,21 +95,31 @@ def _completed_property_ids() -> set[int]:
|
|||
|
||||
def main() -> None:
|
||||
postcode_map = _load_postcode_map()
|
||||
completed = _completed_property_ids()
|
||||
logger.info(f"{len(completed)} property IDs already completed — skipping")
|
||||
|
||||
# Pending filter disabled — re-run every property regardless of whether it
|
||||
# already has a completed modelling_e2e sub_task for this scenario.
|
||||
batches: list[tuple[str, list[int]]] = []
|
||||
for postcode, ids in postcode_map.items():
|
||||
pending = [pid for pid in ids if pid not in completed]
|
||||
if pending:
|
||||
batches.append((postcode, pending))
|
||||
if ids:
|
||||
batches.append((postcode, ids))
|
||||
|
||||
to_process = batches[:POSTCODES_LIMIT]
|
||||
to_process: list[tuple[str, list[int]]] = []
|
||||
property_count = 0
|
||||
for postcode, ids in batches:
|
||||
if property_count + len(ids) > PROPERTIES_LIMIT:
|
||||
continue
|
||||
to_process.append((postcode, ids))
|
||||
property_count += len(ids)
|
||||
|
||||
if not to_process:
|
||||
logger.info("Nothing left to process.")
|
||||
return
|
||||
|
||||
logger.info(
|
||||
f"selected {len(to_process)} postcodes / {property_count} properties "
|
||||
f"(limit {PROPERTIES_LIMIT})"
|
||||
)
|
||||
|
||||
sqs: Any = cast(
|
||||
Any, boto3.client("sqs", region_name="eu-west-2")
|
||||
) # pyright: ignore[reportUnknownMemberType]
|
||||
|
|
|
|||
|
|
@ -214,16 +214,19 @@ def test_lodged_epc_path_saves_epc_plan_and_marks_modelled(
|
|||
_baseline_orchestrator.return_value.run.assert_called_once_with([PROPERTY_ID])
|
||||
|
||||
|
||||
def test_skipped_cohort_certs_are_surfaced_in_the_outputs() -> None:
|
||||
def test_skipped_cohort_certs_fail_the_subtask_but_the_plan_is_still_saved() -> None:
|
||||
"""Cohort certs the mapper can't consume are skipped (so prediction is not
|
||||
aborted) and surfaced — with cert numbers — in the subtask outputs, so the
|
||||
mapper gaps can be reported and closed."""
|
||||
aborted), then surfaced as a failure — the subtask is marked failed (the
|
||||
cert numbers land in outputs.error via the raised RuntimeError) so the
|
||||
mapper gaps get debugged. The batch still ran to completion first, so the
|
||||
property's plan was committed before the handler raised."""
|
||||
from repositories.comparable_properties.epc_comparable_properties_repository import (
|
||||
SkippedCohortCert,
|
||||
)
|
||||
|
||||
mock_engine = _engine_mock([PROPERTY_ID], [UPRN], [POSTCODE])
|
||||
mock_plan = _plan_mock()
|
||||
mock_uow = MagicMock()
|
||||
skipped = [
|
||||
SkippedCohortCert(
|
||||
certificate_number="8257-7539-1649-0633-4992",
|
||||
|
|
@ -287,24 +290,21 @@ def test_skipped_cohort_certs_are_surfaced_in_the_outputs() -> None:
|
|||
MockUoW = stack.enter_context(
|
||||
patch("applications.modelling_e2e.handler.PostgresUnitOfWork")
|
||||
)
|
||||
MockUoW.return_value.__enter__.return_value = MagicMock()
|
||||
MockUoW.return_value.__enter__.return_value = mock_uow
|
||||
MockUoW.return_value.__exit__.return_value = False
|
||||
|
||||
# Act
|
||||
result = _call_handler(_BODY)
|
||||
# Act — the skipped cert fails the subtask, but only after the batch ran.
|
||||
with pytest.raises(RuntimeError) as excinfo:
|
||||
_call_handler(_BODY)
|
||||
|
||||
# Assert — the handler's return (→ subtask outputs.result) carries the cert
|
||||
# numbers + errors of every skipped cohort cert.
|
||||
assert result == {
|
||||
"skipped_unmappable_cohort_certs": [
|
||||
{
|
||||
"certificate_number": "8257-7539-1649-0633-4992",
|
||||
"error": (
|
||||
"ValueError: RdSapSchema17_1: missing required field 'window'"
|
||||
),
|
||||
}
|
||||
]
|
||||
}
|
||||
# Assert — the cert number + error reach outputs.error (the raised message),
|
||||
# and the property's plan was still committed before the handler raised.
|
||||
message = str(excinfo.value)
|
||||
assert "skipped_unmappable_cohort_certs" in message
|
||||
assert "8257-7539-1649-0633-4992" in message
|
||||
assert "RdSapSchema17_1: missing required field 'window'" in message
|
||||
mock_uow.plan.save.assert_called_once()
|
||||
mock_uow.commit.assert_called_once()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
|
|
|
|||
|
|
@ -492,10 +492,11 @@ def test_suspended_floor_overlay_reproduces_the_relodged_after() -> None:
|
|||
|
||||
|
||||
def test_double_glazing_overlay_reproduces_the_relodged_after_windows() -> None:
|
||||
# The full-SAP pin below is xfail (draught-proofing coupling), but the
|
||||
# overlay's actual job — turning every single-glazed window into the
|
||||
# relodged spec — is deterministic and must hold exactly: it proves the
|
||||
# generator detects BOTH single-glazing codes (1 and 15) on the real cert.
|
||||
# The full-SAP pin below now passes (the draught-proofing + frame-factor
|
||||
# coupling is modelled), but this narrower pin isolates the overlay's actual
|
||||
# job — turning every single-glazed window into the relodged spec — which is
|
||||
# deterministic and must hold exactly: it proves the generator detects BOTH
|
||||
# single-glazing codes (1 and 15) on the real cert.
|
||||
# Arrange
|
||||
before: EpcPropertyData = parse_recommendation_summary(
|
||||
"double_glazing_001431_before.pdf"
|
||||
|
|
@ -528,24 +529,13 @@ def test_double_glazing_overlay_reproduces_the_relodged_after_windows() -> None:
|
|||
assert _window_spec(applied) == _window_spec(after)
|
||||
|
||||
|
||||
_GLAZING_DRAUGHT_COUPLING_REASON: Final[str] = (
|
||||
"Blocked on the glazing measure's draught-proofing coupling. The window "
|
||||
"U/g overlay reproduces the after's 14 windows EXACTLY (all four single-"
|
||||
"glazed panes — codes 1 and 15 — become the relodged double/secondary "
|
||||
"spec). The residual ~0.7 SAP is a secondary effect the overlay does not "
|
||||
"model: replacing the single-glazed (lodged draught_proofed=No) windows "
|
||||
"with sealed units re-lodges percent_draughtproofed 84->100 (~0.3 SAP) and "
|
||||
"lowers fabric heat loss by ~+150 kWh space heating (~0.4 SAP) not yet "
|
||||
"isolated. Flips green once the glazing overlay propagates draught-proofing "
|
||||
"(and the residual fabric coupling is modelled)."
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.xfail(strict=True, reason=_GLAZING_DRAUGHT_COUPLING_REASON)
|
||||
def test_double_glazing_overlay_reproduces_the_relodged_after() -> None:
|
||||
# Arrange — cert 001431 lodges four single-glazed windows (codes 1 and 15,
|
||||
# "single glazing, known data"); the after re-lodges every one as double
|
||||
# (gt=5, U=1.40, g=0.72).
|
||||
# (gt=5, U=1.40, g=0.72). The overlay now also models the two secondary
|
||||
# effects of fitting sealed units: RdSAP 10 §8.1 draught-proofing
|
||||
# 84 → 100 (`percent_draughtproofed`) and the re-lodged FF=0.70 on the
|
||||
# panes that lodged FF 1.00 / 0.50 — so the full-SAP pin closes.
|
||||
before: EpcPropertyData = parse_recommendation_summary(
|
||||
"double_glazing_001431_before.pdf"
|
||||
)
|
||||
|
|
@ -561,7 +551,6 @@ def test_double_glazing_overlay_reproduces_the_relodged_after() -> None:
|
|||
)
|
||||
|
||||
|
||||
@pytest.mark.xfail(strict=True, reason=_GLAZING_DRAUGHT_COUPLING_REASON)
|
||||
def test_secondary_glazing_overlay_reproduces_the_relodged_after() -> None:
|
||||
# Arrange — a planning protection forces secondary glazing; the after
|
||||
# re-lodges every single-glazed window as secondary (gt=11, U=2.90, g=0.85).
|
||||
|
|
|
|||
|
|
@ -67,8 +67,12 @@ def test_single_glazed_dwelling_yields_a_double_glazing_recommendation() -> None
|
|||
option = recommendation.options[0]
|
||||
assert option.measure_type == "double_glazing"
|
||||
assert dict(option.overlay.windows) == {
|
||||
0: WindowOverlay(glazing_type=5, u_value=1.40, solar_transmittance=0.72),
|
||||
2: WindowOverlay(glazing_type=5, u_value=1.40, solar_transmittance=0.72),
|
||||
0: WindowOverlay(
|
||||
glazing_type=5, u_value=1.40, solar_transmittance=0.72, frame_factor=0.70
|
||||
),
|
||||
2: WindowOverlay(
|
||||
glazing_type=5, u_value=1.40, solar_transmittance=0.72, frame_factor=0.70
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
|
|
@ -121,6 +125,10 @@ def test_planning_protection_picks_secondary_glazing_over_double() -> None:
|
|||
option = recommendation.options[0]
|
||||
assert option.measure_type == "secondary_glazing"
|
||||
assert dict(option.overlay.windows) == {
|
||||
0: WindowOverlay(glazing_type=11, u_value=2.90, solar_transmittance=0.85),
|
||||
2: WindowOverlay(glazing_type=11, u_value=2.90, solar_transmittance=0.85),
|
||||
0: WindowOverlay(
|
||||
glazing_type=11, u_value=2.90, solar_transmittance=0.85, frame_factor=0.70
|
||||
),
|
||||
2: WindowOverlay(
|
||||
glazing_type=11, u_value=2.90, solar_transmittance=0.85, frame_factor=0.70
|
||||
),
|
||||
}
|
||||
|
|
|
|||
|
|
@ -55,6 +55,55 @@ def test_apply_writes_targeted_building_part_and_leaves_others_untouched() -> No
|
|||
)
|
||||
|
||||
|
||||
def test_override_for_an_absent_semantic_part_lands_on_the_part_at_that_position() -> (
|
||||
None
|
||||
):
|
||||
# `building_part` is a POSITIONAL index (0=main, 1=extension 1…, ADR-0004). The
|
||||
# gov-API EPC can label its parts differently (e.g. a 2nd part lodged as `other`
|
||||
# rather than `extension_1`). An `extension_1` override must still land on the
|
||||
# part at position 1 — the landlord's correction is applied, not dropped.
|
||||
# Arrange — build_epc() is [MAIN, EXTENSION_1]; relabel the 2nd part to OTHER so
|
||||
# the EXTENSION_1 identifier is absent but position 1 still exists.
|
||||
baseline: EpcPropertyData = build_epc()
|
||||
baseline.sap_building_parts[1].identifier = BuildingPartIdentifier.OTHER
|
||||
simulation = EpcSimulation(
|
||||
building_parts={
|
||||
BuildingPartIdentifier.EXTENSION_1: BuildingPartOverlay(
|
||||
wall_insulation_type=3
|
||||
)
|
||||
}
|
||||
)
|
||||
|
||||
# Act
|
||||
result: EpcPropertyData = apply_simulations(baseline, [simulation])
|
||||
|
||||
# Assert — the override folded onto the part at position 1 (the OTHER part).
|
||||
assert _part(result, BuildingPartIdentifier.OTHER).wall_insulation_type == 3
|
||||
|
||||
|
||||
def test_override_with_no_part_at_that_position_is_skipped() -> None:
|
||||
# When there is genuinely no part at the override's position (the EPC models
|
||||
# fewer parts than the index), the override is skipped rather than crashing —
|
||||
# we cannot model an extension we have no geometry for.
|
||||
# Arrange — build_epc() has 2 parts (positions 0, 1); position 2 is absent.
|
||||
baseline: EpcPropertyData = build_epc()
|
||||
simulation = EpcSimulation(
|
||||
building_parts={
|
||||
BuildingPartIdentifier.MAIN: BuildingPartOverlay(wall_insulation_type=1),
|
||||
BuildingPartIdentifier.EXTENSION_2: BuildingPartOverlay(
|
||||
wall_insulation_type=1
|
||||
),
|
||||
}
|
||||
)
|
||||
|
||||
# Act
|
||||
result: EpcPropertyData = apply_simulations(baseline, [simulation])
|
||||
|
||||
# Assert — the present part got its overlay; nothing was added for position 2.
|
||||
assert _part(result, BuildingPartIdentifier.MAIN).wall_insulation_type == 1
|
||||
assert len(result.sap_building_parts) == len(baseline.sap_building_parts)
|
||||
|
||||
|
||||
def test_flat_roof_construction_type_folds_onto_the_part() -> None:
|
||||
# ADR-0033: a flat-roof landlord override sets `roof_construction_type` so the
|
||||
# calculator's flat path (`"flat" in roof_construction_type`) fires the
|
||||
|
|
|
|||
|
|
@ -603,6 +603,52 @@ _EXPECTATIONS: Final[tuple[RealCertExpectation, ...]] = (
|
|||
"engine prices 100% at the low rate; see elmhurst_worksheet.pdf (243-246)"
|
||||
),
|
||||
),
|
||||
# UPRN 10070004512 → cert 8742-6624-9300-2780-4926. SAP-Schema-16.0, GROUND-
|
||||
# FLOOR FLAT, band B, cavity FILLED, ELECTRIC STORAGE HEATERS (SAP 402 SEB,
|
||||
# manual charge CSA/2401) + electric immersion off-peak (Economy-7 Dual meter)
|
||||
# with a small cylinder (size 1), roof = another dwelling above, floor to
|
||||
# EXTERNAL AIR, double glazed, TFA 33.24. Engine 66 = lodged 66 EXACTLY.
|
||||
# This is the modelling_e2e built_form fix cert: 16.0 omits `built_form`, which
|
||||
# RdSapSchema17_1 requires; the mapper derives it from dwelling_type (flat →
|
||||
# modal 4). built_form is ML-only (the SAP calculator never reads it) so the fix
|
||||
# is SAP-NEUTRAL — the engine reproduces the lodged score regardless. Built in
|
||||
# Elmhurst RdSAP10 (evidence saved: elmhurst_summary.pdf / elmhurst_worksheet.pdf):
|
||||
# worksheet SAP 54, engine on Elmhurst's own parsed inputs 53 ≈ 54 → calculator
|
||||
# faithful. The engine 66 vs Elmhurst 54 (+12) is an input/build gap dominated by
|
||||
# HOT WATER (engine 1272 vs Elmhurst 1948 kWh): the cert lodges a size-1 (small)
|
||||
# cylinder but Elmhurst's RdSAP entry has no "Small" option (Normal/110 L is the
|
||||
# smallest), forcing a larger cylinder + more storage loss; plus the reduced-field
|
||||
# 16.0 floor/party-wall defaults. PINNED to the observed engine 66 (= lodged 66).
|
||||
RealCertExpectation(
|
||||
schema="SAP-Schema-16.0",
|
||||
sample="uprn_10070004512",
|
||||
cert_num="8742-6624-9300-2780-4926",
|
||||
sap_score=66,
|
||||
),
|
||||
# UPRN 22086693 → cert 6102-6227-8000-0083-2292. RdSAP-Schema-20.0.0, SEMI-
|
||||
# DETACHED HOUSE 2-storey, band C, cavity UNINSULATED, mains-gas COMBI (no PCDB
|
||||
# → generic SAP Table-4b BGW post-98 condensing combi), control 2106 (CBE),
|
||||
# pitched 200 mm loft, suspended uninsulated floor, party wall 6.8 m, double
|
||||
# glazed, electric secondary room heater (SAP 691 REA), 2× PV ARRAYS @1.14 kW,
|
||||
# 100% LED, TFA ~74. Engine 66 / lodged 72.
|
||||
# This is the modelling_e2e photovoltaic_supply-AS-LIST fix cert: 20.0.0 typed
|
||||
# `photovoltaic_supply` as the wrapper only, so a measured-array LIST crashed
|
||||
# `from_rdsap_schema_20_0_0` ("'list' object has no attribute none_or_no_details")
|
||||
# and sank the whole prediction cohort. The fix routes it through the dict-tolerant
|
||||
# `_map_schema_21_pv`, capturing the arrays. PV is correctly credited: engine
|
||||
# WITHOUT pv = 61, WITH pv = 66 (+5). Built in Elmhurst RdSAP10 (evidence saved:
|
||||
# elmhurst_summary.pdf / elmhurst_worksheet.pdf) — fabric+heating only (the PV
|
||||
# is a separate "New Technologies" Panel-details grid, deferred): worksheet 55,
|
||||
# engine on Elmhurst's own parsed inputs 55 = 55 EXACTLY → calculator faithful.
|
||||
# The engine-without-pv 61 vs Elmhurst 55 (+6) is the documented engine-vs-
|
||||
# Elmhurst-RdSAP-default residual on a band-C cavity-uninsulated suspended-floor
|
||||
# semi; lodged 72 vs engine 66 (−6) is that plus PV-credit method. PINNED engine 66.
|
||||
RealCertExpectation(
|
||||
schema="RdSAP-Schema-20.0.0",
|
||||
sample="uprn_22086693",
|
||||
cert_num="6102-6227-8000-0083-2292",
|
||||
sap_score=66,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue