Model/backlog/tasks/task-10 - Fix-bulk_address_uploads-SQLModel-—-align-columns-with-real-schema-prevent-rogue-migrations.md
2026-04-20 13:06:31 +00:00

2.2 KiB

id title status assignee created_date updated_date labels dependencies priority ordinal
TASK-10 Fix bulk_address_uploads SQLModel — align columns with real schema, prevent rogue migrations Done
2026-04-20 2026-04-20 12:34
backend
bulk-upload
db
high 7000

Description

backend/app/db/models/bulk_address_uploads.py has several bugs that cause a rogue ALTER TABLE and silent write failures:

1. Wrong column name Model declares combined_csv_s3_uri — real column (drizzle-managed) is combined_output_s3_uri. set_combined_csv_s3_uri() currently writes to a non-existent column.

2. Partial model declared as table=True Model only includes id, task_id, combined_csv_s3_uri, status, updated_at. Missing: portfolio_id, user_id, s3_bucket, s3_key, filename, source_headers, column_mapping, created_at. SQLModel table=True with incomplete columns causes Alembic autogenerate / create_all to try to ALTER or recreate the table.

3. status default mismatch Backend: default="pending". Real table default: 'ready_for_processing'. Triggers ALTER TABLE on migration runs.

4. task_id nullability mismatch Backend: task_id: UUID (NOT NULL). Frontend drizzle schema: nullable (set later, after onboarding starts).

Fix approach:

  • Declare all real columns matching drizzle schema (see src/app/db/schema/bulk_address_uploads.ts in assessment-model repo as source of truth).
  • Rename combined_csv_s3_uricombined_output_s3_uri throughout.
  • task_id: Optional[UUID], status default 'ready_for_processing'.
  • Ensure Alembic env excludes this table from autogenerate — drizzle owns migrations, not backend.

Acceptance Criteria

  • #1 combined_output_s3_uri column name correct throughout model + helper
  • #2 All columns present and match real table schema
  • #3 No rogue ALTER TABLE when backend starts or migrations run
  • #4 task_id nullable, status default 'ready_for_processing'
  • #5 Integration test: combiner runs → combined_output_s3_uri populated → frontend reads it correctly