Commit graph

45 commits

Author SHA1 Message Date
Khalim Conn-Kowlessar
2e416a0221 chore: untrack .claude/settings.json (machine-specific personal config)
`.claude/settings.json` was committed to main by mistake — it holds
per-developer permission allow-lists (npx cache paths, /tmp script paths,
and even hardcoded credentials), not shared project config. Mirror the
existing `.claude/settings.local.json` treatment: remove it from the index
and add it to .gitignore so each developer keeps their own local copy.
Claude Code merges settings.json + settings.local.json at runtime, so no
permissions are lost.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 15:01:26 +00:00
Jun-te Kim
2f0eb49eee Checkpoint: UPRN 10093116543 Elmhurst build + devcontainer VNC/Playwright + perms
- Add SAP-accuracy sample for uprn_10093116543 (epc.json, elmhurst_inputs.md,
  summary/worksheet PDFs)
- Persist hyde viewer stack (xvfb/fluxbox/x11vnc/novnc/websockify) and Playwright
  chromium in the backend devcontainer; forward noVNC 6080
- Broaden .claude/settings.local.json allowlist (display/python/grep/tail)
- In-progress campaign mapper/cert_to_inputs work carried from prior cert

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 15:21:56 +00:00
Jun-te Kim
dad3044740 save skills and automation progress 2026-06-16 12:17:43 +00:00
Daniel Roth
4c707212e7
Merge branch 'main' into improve-sharepoint-renamer 2026-06-12 17:16:15 +01:00
Daniel Roth
2bfbad5ced add local claude settings to gitignore 2026-06-12 16:11:48 +00:00
Jun-te Kim
3b7d26fe34 added test for a 1000 examples 2026-06-09 16:02:21 +00:00
Khalim Conn-Kowlessar
0918dd37ec feat(scripts): run_modelling_e2e — inspect recommendations per property_id
Revives the local recommendation-inspection flow for specific Properties.
`scripts/run_modelling_e2e.py` reads each Property's UPRN from the DB
(read-only), fetches the latest EPC live from the gov EPC API by UPRN, runs the
Modelling stage in memory (all Generators → Optimiser → costed, attributed
Plan), and prints a per-Property plan table + writes a Markdown/CSV summary.
Persists nothing — purely for inspection.

The local DB's Properties have no linked ingested EPC (epc_property.property_id
is NULL for all rows; Ingestion's source clients are stubbed, #1136), so the
EPC must be fetched inline rather than read back. Builds the connection from the
`DB_*` env vars in backend/.env and the EPC token from `EPC_AUTH_TOKEN`.

Threads optional solar insights through harness `run_modelling` (so Solar PV
Options can fire once coordinates are wired) and adds the `solar_pv` catalogue
row. Solar + planning restrictions + DB persistence are noted follow-ups.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 14:25:33 +00:00
Khalim Conn-Kowlessar
afabfa0147 feat(modelling): sample a year from the EPC bulk export, offline-ready
fetch_epc_bulk_sample streams certificates-<year>.json out of the bulk ZIP via
range requests, keeps the first N SAP-version matches, and writes each cert's
inner document to <out>/<cert>.json for run_property_report. Stops after N, so
only the member prefix transfers, not the 15.7 GB archive (RangeFile.bytes_read
reports the true transfer vs the absolute ZIP offset). Verified on 2026: 100
SAP-10.2 certs -> report ran 81 scorable (MAE 2.03), 46 flagged, 19 raises
(11 full-SAP schema 19.1.0, 7 unmapped floor_construction 0/3, 1 missing
post_town) — real shadow-validation signal vs the curated golden 57.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 12:20:57 +00:00
Khalim Conn-Kowlessar
ea3af8d2f4 feat(modelling): CLI to fetch an EPC dump + build the inspection report
run_property_report builds the three-section Markdown+CSV report over a dir of
API-shaped EPC JSON, offline (defaults to the golden 57: 57/57 scorable, MAE
0.54, 6 flagged |Δ|>0.5). fetch_epc_dump pulls raw cert JSON from the live API
by --uprn/--postcode (picking the latest cert per match, skipping existing
files), mirroring fetch_cohort2's proven HTTP shape and reading
OPEN_EPC_API_TOKEN. Report artifacts + epc_dump/ are gitignored.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 11:26:17 +00:00
Khalim Conn-Kowlessar
8b5ab1c59e feat(modelling): turnkey offline cohort script (tables + CSV)
CertResult now carries its Plan (with flat baseline/post-SAP/measures
properties), and `format_cohort_csv` renders one browsable row per cert
(SAP transition, band, measures, cost, bill saving, valuation %, error).
`scripts/run_modelling_cohort.py` is turnkey: no args runs the committed
golden cohort, prints a sense-check table for the first measure-bearing
certs (a capped preview so a large dump doesn't flood the terminal), the
summary, and writes modelling_cohort.csv (gitignored). Point it at the
EPC dump when it lands.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 09:30:53 +00:00
Khalim Conn-Kowlessar
87b6045c97 fixed merge conflicts from main 2026-05-26 11:21:09 +00:00
Jun-te Kim
914a8ed51e postcode splliter working e2e 2026-05-20 11:07:40 +00:00
Khalim Conn-Kowlessar
eb42cb88a1 slice 14a: ml_training_data pkg + sample.py (CSV filter + random sample)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 17:39:43 +00:00
Khalim Conn-Kowlessar
dba254e316 slice 8a: window physics and orientation aggregates
Thirteen window-aggregate features land on the transform: count,
total area, eight SAP-octant area columns (N/NE/E/SE/S/SW/W/NW),
area-weighted draught-proofing pct, and area-weighted u_value +
solar transmittance (nullable, populated only when windows carry
transmission_details). Windows with orientation outside 1-8 (0,
NR) contribute to count and total area but no octant.

Also: epc codes CSV (gov api /api/codes export, RdSAP-Schema-21.x +
older versions) moved next to EpcPropertyData as epc_codes.csv —
canonical SAP enum source for upcoming categorical-share slices.
.gitignore exception added so the reference CSV is tracked.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 15:32:45 +00:00
Khalim Conn-Kowlessar
0955862973 working on integrating new EPC api into address2UPRN 2026-04-27 11:32:44 +00:00
Daniel Roth
e406b11be8 Merge branch 'main' into feature/read-site-notes-pdf 2026-04-23 10:11:14 +00:00
Jun-te Kim
c4f6d77845 implemented onboarding 2026-04-21 20:23:33 +00:00
Daniel Roth
da26e4a4cb extract no extensions 🟥 2026-04-21 10:53:14 +00:00
Jun-te Kim
ec4c870465 added bulk address 2 uprn lmabda 2026-04-17 19:08:34 +00:00
Daniel Roth
a71e7e30d8 correct sharepoint file structure and playwright videos for debugging 2026-03-26 10:33:49 +00:00
Daniel Roth
63c0f84523 update gitignore to ignore terraform outputs 2026-03-17 17:08:59 +00:00
Khalim Conn-Kowlessar
e645f90b0e fixing stict typing issues 2026-02-26 12:45:52 +00:00
Daniel Roth
07cab931e5 correction to gitignore 2026-01-20 11:02:59 +00:00
Daniel Roth
b1aca16be0 Define simple local runner 2026-01-16 17:28:28 +00:00
Michael Duong
723beaf104 fix pipeline for July 2025 data, keep lodgement date to do analysis on new rdsap standard in ML stage 2025-08-26 10:44:31 +01:00
Khalim Conn-Kowlessar
777d5ccf06 updating git ignore with redundant files 2025-04-03 09:12:18 +01:00
Michael Duong
90ba851cc0 fixed merge conflict with refactor 2024-01-16 16:57:45 +00:00
Michael Duong
62ec498d3f add gitignore for data 2023-12-05 11:33:35 +00:00
Michael Duong
2e4403de43 ignore old folder 2023-11-24 12:29:23 +00:00
Khalim Conn-Kowlessar
c86b742af4 Almost deleted model data 2023-10-05 15:09:50 +01:00
Khalim Conn-Kowlessar
64b6b67499 Added s3 download for conservation area data 2023-09-29 10:19:13 +01:00
Khalim Conn-Kowlessar
62f812cd71 Fixing issue with sap model parsing event body 2023-09-12 18:48:33 +01:00
Michael Duong
4f7c2771fe added predictions to gitignore file 2023-08-30 10:22:05 +01:00
Michael Duong
0f2cd9b09d merged in changes, change imports to stay within core for smaller dependency in dockerfile 2023-08-30 10:16:42 +01:00
Khalim Conn-Kowlessar
d6562bfab9 updating imports for MlModel 2023-08-25 15:25:35 +01:00
Khalim Conn-Kowlessar
cb155f1cac Adding database connection to fastapi 2023-07-31 11:02:22 +01:00
Khalim Conn-Kowlessar
46b3ef10ab creating land registry repo 2023-07-20 14:47:19 +01:00
Khalim Conn-Kowlessar
c0f9462377 further breaking up code 2023-07-20 12:24:34 +01:00
Khalim Conn-Kowlessar
b0c161b118 restructuring openUrpn code 2023-07-20 11:41:43 +01:00
Khalim Conn-Kowlessar
b42ba30e4e commit changes to gitignore 2023-07-09 14:24:01 +01:00
Khalim Conn-Kowlessar
f57f9e028c Moved github actions folder 2023-07-05 19:00:12 +01:00
Khalim Conn-Kowlessar
c7075c5432 setting up land registry data 2023-06-15 22:04:21 +01:00
Khalim Conn-Kowlessar
d48a40c64b Project environment setup 2023-06-07 15:57:22 +01:00
Khalim Conn-Kowlessar
3826b0a647 project setup 2023-06-07 14:51:58 +01:00
KhalimCK
1ef30e07e3
Initial commit 2023-06-07 14:39:24 +01:00