LoadReport.from_calico_load reports loaded observations alongside the three drop
reasons (unmatched-to-portfolio, null-uprn, blank-placeholder) so every Calico
row is accounted for.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CalicoParser now tracks how many rows it drops for a blank/N-A covering type
and how many references it drops for not resolving to a portfolio UPRN, exposed
for the load reconciliation report so nothing vanishes silently.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Postgres adapter for PropertyReferenceReader: selects landlord_property_id and
uprn from the property table for a portfolio, feeding PropertyUprnLookup. Thin
I/O over the FE-owned property table; exercised end-to-end where a DB is
available (no DB in this environment).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PropertyUprnLookup builds a landlord_property_id -> uprn map for a portfolio via
an injected PropertyReferenceReader, excluding rows whose UPRN (or reference) is
null so unresolved properties never receive condition rows.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The real Calico export lodges the literal string "N/A" (not an empty cell) for
placeholder rows; _clean_str now treats N/A-family sentinels as no covering, so
the 1,239 placeholders are dropped on parse (5,346 -> 4,107 real observations).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CalicoParser reads the workbook into CalicoAssetCondition rows, drops rows
with no covering type, and groups them by asset reference into CalicoProperty,
resolving each reference to a UPRN via the injected lookup (unmatched dropped).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CalicoMapper maps each roof-covering row onto a ROOF element with a MATERIAL
aspect, setting renewal_year = export_year + remaining_life (Calico's
Decent-Homes-derived years-remaining anchored to the export year, so the
absolute renewal year is stable across re-runs). source="Calico".
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Behaviour-preserving lift-and-shift of the condition module out of legacy
backend/condition/ into domain/condition, infrastructure/condition,
infrastructure/postgres/condition_tables.py, repositories/condition,
applications/condition, and tests/condition. Imports rewritten to the DDD
paths; ConditionPostgres and the ORM models keep the legacy backend.app.db
Base/db_session bridges so the existing suite proves behaviour is unchanged
(SQLModel + session-DI conversion tracked as a follow-up in ADR-0064).
16 condition tests pass at the new location.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the Stock Condition Data / Component Renewal Year / Roof Replacement
Flag glossary terms and ADR-0064, capturing that condition data is ingested
producer-only and never read by the modelling engine.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Locks the gate edges — a lodged pitched loft below 270mm is recommended, a
loft at 270mm is left alone, and the 'Nmm+' form ('300mm+') parses to its
number and stays ineligible. These passed on arrival (they fell out of the
numeric-parse gate added in the previous commit); pinned as regression guards.
ADR-0063.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A lodged numeric loft depth below the 270mm building-regs compliance gate is
now eligible (topped up to the 300mm install depth), regardless of whether the
roof type is lodged, since a measured depth is a positive statement of a real
loft. Sentinels keep their ADR-0047 resolution. Fixes property 724702 (50mm
loft) which previously returned no recommendation. ADR-0063.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The roof generator offered loft insulation only when genuinely uninsulated
(ADR-0021), so a loft with a shallow lodged depth (e.g. 724702's 50mm) got
nothing and never entered the optimiser pool. Grilled with Khalim: the loft
branch becomes eligible when the lodged depth is a known numeric value below
the 270mm building-regs compliance gate, topped up to the 300mm install depth.
Numeric-lodged-depths only; sentinels keep their ADR-0047 resolution. Loft
branch only. CONTEXT 'Roof Insulation Eligibility' updated to match.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Properties 741843/741872 (uprns 100060714155/100021944594) failed
modelling_e2e subtasks 6ac7841a-9338-4efe-97e5-7e0d19b3055d and
c5450f03-5b0d-4068-a3b3-d1470bc0af57 with a bare StopIteration: their
SAP-Schema-15.0 (2011-era LIG-lodged) certs identify the main dwelling's
building part as "Main building" rather than "Main Dwelling", so
from_api_string fell to OTHER and left the property with no MAIN part —
crashing wall_recommendation.py's unguarded next(...).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Thread the UPRN the PashubService has already resolved for a job into the
PasHub site-notes mapper so the EpcPropertyData aggregate is born with its
uprn set, and the existing save_epc_property_data path persists it.
- EpcPropertyDataMapper.from_site_notes gains uprn: Optional[int] = None and
sets it unconditionally (site notes never carry a UPRN natively).
- parse_site_notes_pdf / _parse_pashub forward the uprn; the Elmhurst branch
is untouched.
- PashubService coerces uprn str -> int in one place, carries it on the
internal upload record, and passes it into parse_site_notes_pdf.
- No new lookups; jobs with no known UPRN still persist null, unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The prior commit derived a Python bool (`cylinder_size != 1`), but
`from_rdsap_schema_17_1` reads the field via `schema.has_hot_water_cylinder
== "true"` — every real 15.0/16.x fixture lodges the lowercase string
"true"/"false", not a JSON boolean. A bare bool compares False against
that string check either way, so the derivation silently mapped every
cylinder-present cert to has_hot_water_cylinder=False too.
Caught by the requested true-branch test (mutating sap_16_2.json, which
lodges cylinder_size=2/has_hot_water_cylinder="true", to omit the field)
— it failed under the original bool-typed fix. Now emits "true"/"false"
strings to match the lodged convention.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Task b9fcd354-2a33-4fc4-a64e-388c060a055c (portfolio 824 / scenario 1278)
also failed property_id 749229 (UPRN 100010349400) with "RdSapSchema17_1:
missing required field 'has_hot_water_cylinder'". Cert
9568-3034-6211-6089-5960 (SAP-Schema-15.0) lodges `sap_heating.cylinder_size`
but omits the separate top-level `has_hot_water_cylinder` boolean
RdSapSchema17_1 also requires.
Unlike `multiple_glazed_proportion` (deliberately left un-defaulted a few
lines above — a prior guessed default regressed the accuracy gate),
`cylinder_size` isn't a proxy needing inference: RdSAP 10 Table 28 defines
code 1 as "no cylinder" — literally the same fact `has_hot_water_cylinder`
encodes. Confirmed 1:1 across every real 15.0/16.x fixture that lodges both
fields (cylinder_size == 1 <-> false, every other code <-> true).
Added to the shared `_normalize_sap_schema_16_x`, so it covers the whole
15.0/16.0/16.1/16.2/16.3 reduced-field family, not just 15.0.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Task b9fcd354-2a33-4fc4-a64e-388c060a055c (portfolio 824 / scenario 1278)
still failed 5 of 128 batches with NotNullViolation on
epc_building_part.construction_age_band even after the SAP-Schema-15.0
mapper landed (PR #1531). Cert 8169-6926-5310-0447-8996 (UPRN
100010344955, SAP-Schema-15.0) lodges a bare alternative-wall record
(wall_area/wall_construction/wall_insulation_type, no identifier, age
band, or floor dimensions) as a second sap_building_parts element rather
than under a distinct sap_alternative_wall_1 key. RdSapSchema17_1 (which
the 15.0/16.x reduced-field mappers delegate to) has no alt-wall slot to
route it to, so it persisted as a phantom building part with a null age
band.
_drop_building_parts_without_age_band already solved exactly this shape
of problem for lodging artifacts (ae81a81e), but that commit sits on
fix/drop-building-part-without-age-band, an unmerged branch that has
since drifted far behind main. Reinstating the same filter (fresh, not
cherry-picked) fixes both the original artifact case and this SAP-15
alt-wall case, since RdSapSchema17_1 has nowhere else to put it either
way.
Verified against all 174 properties across the task's originally-failing
batches: zero remaining null-age-band building parts.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
from_api_response now returns None for an unsupported schema_type
instead of raising, so the harness must check before dereferencing
the mapped EpcPropertyData — otherwise an AttributeError replaces
the expected ValueError in calculator_error.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Review findings on PR #1527:
- The overlay constants, ScoredOption builder, ventilation dependency and
selected_types helper come from the shared _optimiser_fixtures module
(landed on the fabric-first base) instead of local copies; the boiler
overlay is the shared BOILER_OVERLAY (SAP Table 4a code 104, a mains-gas
combi) rather than code 201, which is neither a boiler nor a heat pump.
_IWI_OVERLAY (solid-wall internal, type 3) stays local — no shared
equivalent — and the carbon stubs stay bespoke (the shared StubScorer has
no CO2 knob).
- The optimise_package_fabric_first import is lifted to module scope.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review findings on PR #1527:
- The 'which goals are goal-aligned' set lived in two adjacent if-chains
(_objective_for and _require_budget_for_goal_aligned) that had to stay in
sync — a new goal-aligned goal added to one but not the other would slip
the budget guard. Both now read a single _GOAL_ALIGNED_OBJECTIVES table.
- The goal strings are the canonical PortfolioGoal enum values, not
re-declared string constants, so goal-value drift can't silently degrade
a goal to max-SAP; _target_sap reads the enum too.
- _scored_candidate_groups takes objective without a default (its only
caller passes it).
- scoring.py: 'cached: float | None' -> Optional[float] per the CLAUDE.md
'Use Optional over | None' rule.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review findings on PR #1527:
- The objective is required (no sap_rating default) on _repair_to_target,
_best_repair_candidate and _rescored_groups: every caller already passes
it, and a default would let a future call path silently optimise SAP for
a carbon/bill goal while pyright stayed green. The default stays on the
public optimise_package / optimise_package_fabric_first entry points.
- _best_repair_candidate hoists objective(current) out of the candidate
loop: current is loop-invariant, so for the Energy-Savings bill objective
this was one full BillDerivation.derive per candidate per repair iteration
for the same score.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
An unrecognised schema_type used to raise ValueError and abort the whole
call; now from_api_response logs a warning and returns None so one
unmapped cert doesn't break a batch. Schema 15.0 already has a mapper
(PR #1531) so it's unaffected; only genuinely unmapped versions skip.
