Commit graph

4770 commits

Author SHA1 Message Date
Khalim Conn-Kowlessar
eb42cb88a1 slice 14a: ml_training_data pkg + sample.py (CSV filter + random sample)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 17:39:43 +00:00
Khalim Conn-Kowlessar
3abcee6a53 slice 13: to_rows(properties) returns pd.DataFrame
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 16:43:28 +00:00
Khalim Conn-Kowlessar
ebceb4bf2b slice 12: ventilation flat features
Four ventilation features: mechanical_ventilation (categorical
SAP10 code, 0=natural through 6=positive-input-from-outside per
epc_codes.csv mechanical_ventilation enum), mechanical_vent_duct_type
(categorical), blocked_chimneys_count (int), and pressure_test
(int — air-tightness SAP10 code).

Pulled from top-level EpcPropertyData fields; ventilation on SAP10
API EPCs sits on the certificate directly, not on the
sap_ventilation block (which is site-notes-only).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 16:09:53 +00:00
Khalim Conn-Kowlessar
559a2128b9 slice 11b: PV battery, wind turbine, energy source flags
Nine more energy-source features land: has_pv_battery,
pv_battery_count, pv_battery_capacity_kwh (count × per-unit
capacity from pv_batteries.pv_battery, nullable when count=0),
has_wind_turbine, wind_turbine_count, mains_gas (the dominant
fuel-deduction signal), and the three smart-meter / export
booleans (electricity_smart_meter_present, gas_smart_meter_present,
is_dwelling_export_capable).

Closes the PV/solar feature group started in slice 11a.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 16:07:17 +00:00
Khalim Conn-Kowlessar
706d1b5b66 slice 11a: PV array aggregates + capacity_source flag
Fifteen PV features land: has_pv (bool), pv_capacity_source (str
categorical: measured / estimated_from_roof_area / none),
pv_array_count, pv_total_peak_power_kw, eight peak-power-by-octant
columns (pv_peak_power_kw_{N..NW}), peak-power-weighted
pv_avg_pitch and pv_avg_overshading (nullable), and
pv_percent_roof_area (nullable — populated only on the estimated
branch).

Dispatches on the SAP10 EpcPropertyData.SapEnergySource shapes added
in slice 10.5: photovoltaic_arrays populates → measured;
photovoltaic_supply.none_or_no_details.percent_roof_area > 0 →
estimated; everything else → none. percent_roof_area == 0 is the
canonical no-PV payload and surfaces as 'none', not 'estimated'.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 16:04:15 +00:00
Khalim Conn-Kowlessar
b050348927 slice 10.5: PhotovoltaicArray on SAP10 schema + EpcPropertyData
SAP10 EPCs with measured PV carry photovoltaic_supply as a nested
list of arrays (peak_power, pitch, orientation, overshading) rather
than the legacy unmeasured wrapper {none_or_no_details:
{percent_roof_area: N}}. The schema-21 dataclasses now accept both
shapes via Union[PhotovoltaicSupply, List[List[PhotovoltaicArray]]],
and from_dict._coerce now dispatches list values onto list type
variants of multi-type Unions.

EpcPropertyData.SapEnergySource gains
photovoltaic_arrays: Optional[List[PhotovoltaicArray]] — populated
when the measured shape is present, otherwise None. The legacy
photovoltaic_supply field is preserved for the fallback case.
Both schema-21.0.0 and 21.0.1 mappers dispatch via the new
_map_schema_21_pv helper.

Unblocks Slice 11 (PV feature aggregation in EpcMlTransform).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 16:00:25 +00:00
Khalim Conn-Kowlessar
fff6ef3352 slice 10: heating system features (primary + water + secondary)
Fifteen heating features land via hybrid Top-1 + flat fields: the
primary heating slot from main_heating_details[0] gives
main_fuel_type, heat_emitter_type, main_heating_control,
main_heating_category, has_fghrs, fan_flue_present, boiler_flue_type
and central_heating_pump_age (all int-categorical for the SAP10
codes); main_heating_count carries the aggregate. Water heating
adds water_heating_code, water_heating_fuel, cylinder_size, and
cylinder_insulation_thickness_mm. Secondary heating is summarised
by has_secondary_heating (derived) and secondary_fuel_type.

Fuel codes follow the gov api enums in epc_codes.csv (44 main_fuel
values shared with water_heating_fuel). Union[int, str] fields
coerce to int when the value is int, else None.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 15:50:05 +00:00
Khalim Conn-Kowlessar
fb773fa635 slice 9: building parts with main-dwelling carve-out
Thirteen building-parts features land: five cross-all-parts physical
aggregates (count, total_heat_loss_perimeter_m,
total_party_wall_length_m, total_floor_area_from_parts_m2,
avg_room_height_m) and eight Main-Dwelling-specific columns
(heat_loss_perimeter, party_wall_length, total_floor_area,
avg_room_height, has_room_in_roof, construction_age_band,
wall_construction, roof_construction). Main-Dwelling columns are
None when no part has identifier == 'Main Dwelling' — honest about
data quality rather than silently falling back to the first part.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 15:45:21 +00:00
Khalim Conn-Kowlessar
079e6f9a68 slice 8b: window glazed_type and pvc_frame shares
Adds seventeen window-categorical-share features: one float per
SAP10 glazed_type code (1-15) plus a `_other` bucket for anything
outside the enum, and a single `window_pct_pvc_frame` for the
area-weighted PVC-frame share. All shares are area-weighted over
total window area; null pvc_frame share for window-less properties.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 15:36:05 +00:00
Khalim Conn-Kowlessar
dba254e316 slice 8a: window physics and orientation aggregates
Thirteen window-aggregate features land on the transform: count,
total area, eight SAP-octant area columns (N/NE/E/SE/S/SW/W/NW),
area-weighted draught-proofing pct, and area-weighted u_value +
solar transmittance (nullable, populated only when windows carry
transmission_details). Windows with orientation outside 1-8 (0,
NR) contribute to count and total area but no octant.

Also: epc codes CSV (gov api /api/codes export, RdSAP-Schema-21.x +
older versions) moved next to EpcPropertyData as epc_codes.csv —
canonical SAP enum source for upcoming categorical-share slices.
.gitignore exception added so the reference CSV is tracked.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 15:32:45 +00:00
Khalim Conn-Kowlessar
9c8aa75469 slice 7: flat categoricals + ColumnSpec.categorical flag
Adds seven flat categorical features (dwelling_type, tenure,
transaction_type, property_type, built_form, region_code,
country_code) emitted as raw strings. New ColumnSpec.categorical
bool tells the parquet writer to cast these to pd.Categorical at the
I/O boundary, keeping pandas out of the domain/schema module.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 15:14:30 +00:00
Khalim Conn-Kowlessar
e4f9e9e1db slice 6: flat booleans and optional integer indicators
Adds three non-nullable booleans (solar_water_heating,
has_hot_water_cylinder, has_fixed_air_conditioning) and three
optional integer indicators (percent_draughtproofed,
energy_rating_average, environmental_impact_current). All direct
EpcPropertyData field reads.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 15:06:02 +00:00
Khalim Conn-Kowlessar
e9b4dbbfe5 slice 5: room, door and lighting count features
Ten flat int counts added to the transform — door_count,
habitable/heated/wet/insulated_door counts, extensions, open
chimneys, and the three fixed-lighting bulb counts (CFL/LED/
incandescent). All non-nullable; direct EpcPropertyData field reads.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 15:03:58 +00:00
Khalim Conn-Kowlessar
aa00259b1a slice 4: total_floor_area_m2 feature
First feature column lands on the transform: schema() advertises
total_floor_area_m2 as a non-nullable float; to_row() emits the value
from EpcPropertyData.total_floor_area_m2 alongside the six targets.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 14:47:25 +00:00
Khalim Conn-Kowlessar
375b0e895e add missing ucl.py and _fixtures.py from slices 2-3
Previous slice commits used -a-style and missed these new files;
imports in transform.py and test_transform.py would dangle on a
fresh checkout. Re-running pytest after this commit covers all four
EpcMlTransform tests cleanly.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 14:43:09 +00:00
Khalim Conn-Kowlessar
81f6163295 added ucl corrected peui 2026-05-16 14:39:24 +00:00
Khalim Conn-Kowlessar
a64e7e74c5 adding kwh feidls to EpcPropertyData and testing to_row 2026-05-16 14:33:25 +00:00
Khalim Conn-Kowlessar
611ff24eb6 scaffolding for ml pipeline 2026-05-16 14:15:56 +00:00
Khalim Conn-Kowlessar
dfe9e3ddbe added potential file scaffolding: 2026-05-15 10:56:53 +00:00
Khalim Conn-Kowlessar
acb2518235 second grill session updating prd + context 2026-05-15 10:41:47 +00:00
Khalim Conn-Kowlessar
8d6c770da8 grilling session updates to prd 2026-05-14 16:36:22 +00:00
Khalim Conn-Kowlessar
f8bd13cb63 editing per portfolio feature flag 2026-05-14 07:39:18 +00:00
Khalim Conn-Kowlessar
02df38e207 note kwh service not needing predictions 2026-05-13 21:52:02 +00:00
Khalim Conn-Kowlessar
d9c1696085 added architechtural decisions, added to prd 2026-05-13 21:26:18 +00:00
Khalim Conn-Kowlessar
3afeeac1b5 editing prep property ui surfaces logic 2026-05-13 20:10:39 +00:00
Khalim Conn-Kowlessar
fcbaf58a40 initial prd 2026-05-13 19:29:02 +00:00
Daniel Roth
be133ab70f
Merge pull request #1082 from Hestia-Homes/feature/check_routes
remove pandas from app requirements
2026-05-13 16:09:38 +01:00
Jun-te Kim
27b5602608 remove pandas 2026-05-13 15:08:06 +00:00
Daniel Roth
247efc37b0
Merge pull request #1062 from Hestia-Homes/feature/check_routes
basic end point check
2026-05-13 15:48:46 +01:00
Jun-te Kim
51460d1cd3 route at th ebeginnign 2026-05-13 14:47:24 +00:00
Jun-te Kim
54864bf102 resolve merge conflict 2026-05-13 14:22:04 +00:00
Jun-te Kim
fae61cb2c5
Merge pull request #1064 from Hestia-Homes/feature/integrate_new_epc_with_historical_epc
Feature/integrate new epc with historical epc
2026-05-13 15:17:21 +01:00
Jun-te Kim
2fb6a99956 throttle added 2026-05-13 14:02:36 +00:00
Jun-te Kim
ff4ad07a2b retry 2026-05-13 11:41:21 +00:00
Jun-te Kim
c347865b9e retry 2026-05-13 09:34:51 +00:00
Jun-te Kim
27c9752949
Merge pull request #1079 from Hestia-Homes/khalim-tweaks
Khalim tweaks
2026-05-13 09:52:36 +01:00
Khalim Conn-Kowlessar
566c70077a removing redundant code 2026-05-13 08:40:51 +00:00
Khalim Conn-Kowlessar
3fd7321337 remove comment 2026-05-13 08:40:51 +00:00
Jun-te Kim
3bcb94f9e5 Merge branch 'main' into feature/integrate_new_epc_with_historical_epc 2026-05-13 08:38:50 +00:00
Jun-te Kim
153ccb9210
Merge pull request #1077 from Hestia-Homes/feature/task-handler-source-cloud-logs
fix dependency issue
2026-05-12 18:17:56 +01:00
Jun-te Kim
09dbfe2106 fix dependency issue 2026-05-12 17:03:16 +00:00
Daniel Roth
853857436a
Merge pull request #1076 from Hestia-Homes/feature/task-handler-source-cloud-logs
task and sub tasks imrpvoed
2026-05-12 17:37:32 +01:00
Jun-te Kim
e458f0a2b7 task and sub tasks imrpvoed 2026-05-12 16:24:11 +00:00
Jun-te Kim
dfc100f78b rank address similiarity 2026-05-12 16:02:01 +00:00
Daniel Roth
272bfbde13
Merge pull request #1074 from Hestia-Homes/feature/magicplan-trigger
Magicplan service: Correct query URLs and implement pagination in plans response
2026-05-12 16:28:23 +01:00
Daniel Roth
a05dfd38c7 Merge branch 'main' into feature/magicplan-trigger 2026-05-12 14:56:11 +00:00
Daniel Roth
5f77fbf4e4 Fetch all pages in get_plans pagination loop 🟪 2026-05-12 14:54:14 +00:00
Daniel Roth
0d324f99b2 Fetch all pages in get_plans pagination loop 🟩 2026-05-12 14:52:46 +00:00
Daniel Roth
6dfca082f8 Fetch all pages in get_plans pagination loop 🟥 2026-05-12 14:52:31 +00:00
Daniel Roth
f83ddd05a8 Paginate get_plans to return flat list[PlanSummary] 🟩 2026-05-12 14:46:00 +00:00